Inter Process Communication Using Pipes

A pipe is a communication channel between two ends. It is mostly used to communicate between processes running within a computer. As such it is an Inter Process Communication (IPC) mechanism.

The concept of pipe is well known in the Linux (Unix) world. It is used on the command line to direct the output of a command as input of the next command. The so called “pipe character” is used as a syntax gadget for that purpose. Windows is using the same syntax.

More generally, a pipe is an operating system object which is close to a file. An application opens a pipe, then read and writes data and finally closes it. The difference between a pipe and a file is that data is kept in memory and the pipe has two ends. The same or different processes may open the same pipe. What is written at one end becomes readable at the other end. This is why it is named a “pipe”: it fact as a pipe or tunnel between the two ends, data flows from one end and the other. Unlike pipes in the real world, computer pipes are bi-directional. One computer pipe is actually made of two distinct sub-pipes: one for each direction.

We can think of pipes as client/server architecture. The server side opens a pipe and waits for data to be available. The client side opens the pipe and then data may flow into the pipe in both directions independently. If several clients open the same pipe, the server sees it as several independent pipes. At server side, data may be written to individual client’s pipes or broadcasted to all clients.

As seen from Windows API, a pipe can be created with or without a name. Giving a name to a pipe allows processes to share the same pipe easily. When using anonymous pipes, a process knows about it because it has inherited his handle.

The process that creates a pipe is the pipe server. A process that connects to a pipe is a pipe client. One process writes data to the pipe, and then the other process reads the data from the pipe. This is bidirectional: each process can write data that will be read by the other. Read and writes are asynchronous. It means a lot of data may be written before it is read at the other end. Data is stored by the operating system into shared memory. The size of the shared memory area is managed by the operating system using advisory values passed to the pipe’s creation function.

Windows pipe API is relatively complex. It has been encapsulated into Delphi classes by Russell Libby in 2003. Although Russell website is no more available, his code is still around in the internet and has been enhanced by several peoples. I have updated his code for Delphi XE3 and made it available from my website at http://www.overbyte.be/eng/blog_source_code.html

Russell wrote a unit named “pipes.pas” containing 3 components:
- TPipeClient: client component
- TPipeServer: Server component
- TPipConsole: console pipe redirection component

For your convenience, I created two packages: one runtime package and one design time package. The components are installed in the component palette under “Pipes” tab.
Using the components is fairly simple. I made two simple demo programs: PipeClient and PipeServer. You run PipeServer first and then one or more PipeClient instances. PipeClient has an edit box and a “Send” button. When you click on “Send”, the edit box content is sent to the server which in turn displays it in a memo. At server side, the “send” button is replaced by “broadcast” to send the same message to all connected client, if any. Of course you may also send to a specific client.

To send the messages, I actually send strings, taking care of Unicode. When sending (Write or broadcast method), you have to pass a pointer to the first byte to be sent and a count of bytes. At receiver side, data is read from a stream which is also made of bytes.

Warning: a pipe is a byte oriented communication channel. There is no guarantee that each single write at one end will correspond to a single read at the other end. Sending strings has I’ve done in my demo is good for interactive application where a user clicks on a button. In a real world application, you have to design a protocol, just like you do when using socket, so that the reader knows exactly what the writer sent. For example, you send a message made of a byte count and the then actual bytes. Or you design a “line oriented” protocol much like most TCP/IP protocol are. Your messages are made you line of text terminated by a CRLF pair. The reader knows he received a complete message because he received the CRLF pair.

To say it in other words, a pipe is like a TFileStream. The read doesn’t know how the stream has been written. He knows how to read it, for example he knows the stream is made of text lines.

UPDATE 2013-10-04: arno.garrels@gmx.de added 64-bit support and fixed code to compile with Delphi 7 to XE5 (earlier versions may compile however untested).

Follow me on Twitter
Follow me on LinkedIn
Follow me on Google+
Visit my website: http://www.overbyte.be
This article is available from http://francois-piette.blogspot.be
Download source code from my website at
      http://www.overbyte.be/eng/blog_source_code.html