Inter-Process Communication - Part 1

By Hiran Ramankutty

Scope
The purpose of this article is to get the readers familiar with the different mechanisms that are
available for communicating between two or more processes. This may also serve as a tutorial
for the novice programmer. There might be several good tutorials on this subject, but here I will
try to communicate my explorations of this subject. This article may not be technically perfect.
Please send your suggestions and queries to .

Introduction
Inter-Process Communication, which in short is known as IPC, deals mainly with the techniques
and mechanisms that facilitate communication between processes. Now, why do we need special
separate mechanisms or techniques for communicating between processes? Why isn't it possible
to have information shared between two processes without using such special mechanisms?

Let us start from something primitive. Imagine you have two glasses completely filled with
water. One glass contains hot water and the other contains cold water. What can you do to make
the temperature of water in both the glasses equal? The simplest answer will be to mix the water
from both the glasses in a glass with much bigger capacity. Once water is mixed, the temperature
becomes equal after some time. If one can remember, this will be framed as a problem with some
numerical data in a High-School Physics examination. If we go by principles, then the
phenomenon here is conduction. If we go by our topic of IPC, then we can say that since the two
glasses were full, we had to use another glass with a larger capacity to mix the contents in order
to balance their heat energy.

Have you ever wondered about the communication medium used in telephones? What about the
blood transporting system in the human body which communicates blood to different parts of the
body? What about my fingers which are typing this document? My brain is doing so many things
at a time. How is it directing one of my fingers to hit one key and some other finger to hit
another key? How is it synchronizing the typing work that is done by both my hands? How is it
also directing me to type the letters of a word that are actually coming to my mind?

Don't worry. I am not going to give a class in Biology. But it would be good if one can imagine a
few more situations where we are using inter-process communication, though not necessarily in
the human body or in a computer program.

So, where are we now? We know that some medium or other is required for communication
between different processes. Similarly, when it comes to computer programs, we need some
mechanism or medium for communication. Primarily, processes can use the available memory to
communicate with each other. But then, the memory is completely managed by the operating
system. A process will be allotted some part of the available memory for execution. Then each
process will have its own unique user space. In no way will the memory allotted for one process
overlap with the memory allotted for another process. Imagine what would happen otherwise!

So, now the question - how do different processes with unique address space communicate with
each other? The operating system's kernel, which has access to all the memory available, will act
as the communication channel. Similar to our earlier example, where the glass with hot water is
one process address space, the glass with cold water is another, and the glass with the larger
capacity is the kernel address space, so that we pour both hot water and cold water into the glass
with larger capacity.

What next? There are different IPC mechanisms which come into use based on the different
requirements. In terms of our water glasses, we can determine the specifics of both pouring the
water into the larger glass and how it will be used after beign poured.

Basic IPC
OK, enough of glasses and water. The IPC mechanisms can be classified into the following
categories as given below:

a. pipes
b. fifos
c. shared memory
d. mapped memory
e. message queues
f. sockets

Pipes
Pipes were evolved in the most primitive forms of the Unix operating system. They provide
unidirectional flow of communication between processes within the same system. In other
words, they are half-duplex, that is, data flows in only one direction. A pipe is created by
invoking the pipe system call, which creates a pair of file descriptors. These descriptors point to
a pipe inode and the file descriptors are returned through the filedes argument. In the file
descriptor pair, filedes[0] is used for reading whereas filedes[1] is used for writing.

Let me explain a scenario where we can use the pipe system call: consider a keyboard-reader
program which simply exits after any alpha-numeric character is pressed on the keyboard. We
will create two processes; one of them will read characters from the keyboard, and the other will
continuously check for alpha-numeric characters. Let us see how the filedes returned by pipe
can be of use in this scenario: (Text version: kbdread-pipe.c.txt)

/***** KEYBOARD HIT PROGRAM *****/

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <pthread.h>
#include <ctype.h>

int filedes[2];

void *read_char()
{
char c;
printf("Entering routine to read character.........\n");
while(1) {
/* Get a character in 'c' except '\n'. */
c = getchar();
if(c == '\n')
c = getchar();
write(filedes[1], &c, 1);
if(isalnum(c)) {
sleep(2);
exit(1);
}
}
}

void *check_hit()
{
char c;
printf("Entering routine to check hit.........\n");
while(1) {
read(filedes[0], &c, 1);
if(isalnum(c)) {
printf("The key hit is %c\n", c);
exit(1);
} else {
printf("key hit is %c\n", c);
}
}
}

int main()
{
int i;
pthread_t tid1, tid2;
pipe(filedes);
/* Create thread for reading characters. */
i = pthread_create(&tid1, NULL, read_char, NULL);
/* Create thread for checking hitting of any keyboard key. */
i = pthread_create(&tid2, NULL, check_hit, NULL);
if(i == 0) while(1);
return 0;
}

Save and compile the program as cc filename.c -lpthread. Run the program and check the
results. Try hitting a different key every time.
The read_char function simply reads a character other than '\n' from the keyboard and writes it
to filedes[1]. We have the thread check_hit, which continuously checks for the character in
filedes[0]. If the character in filedes[0] is an alpha-numeric character, then the character is
printed and the program terminates.

One major feature of pipe is that the data flowing through the communication medium is
transient, that is, data once read from the read descriptor cannot be read again. Also, if we write
data continuously into the write descriptor, then we will be able to read the data only in the order
in which the data was written. One can experiment with that by doing successive writes or
reads to the respective descriptors.

So, what happens when the pipe system call is invoked? A good look at the manual entry for
pipe suggests that it creates a pair of file descriptors. This suggests that the kernel implements
pipe within the file system. However, pipe does not actually exist as such - so when the call is
made, the kernel allocates free inodes and creates a pair of file descriptors as well as the
corresponding entries in the file table which the kernel uses. Hence, the kernel enables the user to
use the normal file operations like read, write, etc., which the user does through the file
descriptors. The kernel makes sure that one of the descriptors is for reading and another one if
for writing.

I am not going to go into the details of the pipe implementation on the kernel side. For further
reading, one can refer the books mentioned at the end of this article.

FIFOs
FIFOs (first in, first out) are similar to the working of pipes. FIFOs also provide half-duplex
flow of data just like pipes. The difference between fifos and pipes is that the former is
identified in the file system with a name, while the latter is not. That is, fifos are named pipes.
Fifos are identified by an access point which is a file within the file system, whereas pipes are
identified by an access point which is simply an allotted inode. Another major difference
between fifos and pipes is that fifos last throughout the life-cycle of the system, while pipes last
only during the life-cycle of the process in which they were created. To make it more clear, fifos
exist beyond the life of the process. Since they are identified by the file system, they remain in
the hierarchy until explicitly removed using unlink, but pipes are inherited only by related
processes, that is, processes which are descendants of a single process.

Let us see how a fifo can be used to detect a keypress, just as we did with pipes. The same
program where we previously used a pipe can be modified and implemented using a fifo.

(Text version: write-fifo.c.txt)

/***** PROGRAM THAT READS ANY KEY HIT OF THE KEYBOARD*****/

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <pthread.h>
#include <ctype.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>

extern int errno;

void *read_char()
{
char c;
int fd;
printf("Entering routine to read character.........\n");
while(1) {
c = getchar();
fd = open("fifo", O_WRONLY);
if(c == '\n')
c = getchar();
write(fd, &c, 1);
if(isalnum(c)) {
exit(1);
}
close(fd);
}
}

int main()
{
int i;
pthread_t tid1;
i = mkfifo("fifo", 0666);
if(i < 0) {
printf("Problems creating the fifo\n");
if(errno == EEXIST) {
printf("fifo already exists\n");
}
printf("errno is set as %d\n", errno);
}
i = pthread_create(&tid1, NULL, read_char, NULL);
if(i == 0) while(1);
return 0;
}

Compile this program using cc -o write_fifo filename.c. This program reads characters
(keypresses), and writes them into the special file fifo. First the program creates a fifo with read-
write permissions using the function mkfifo. See the manual page for the same. If the fifo exists,
then mkfifo will return the corresponding error, which is set in errno. The thread read_char
continuously tries to read characters from the keyboard.

Note that the fifo is opened with the O_WRONLY (write only) flag .

Once it reads a character other than '\n', it writes the same into the write end of the fifo. The
program that detects it is given below: (text version detect_hit.c.txt):

/***** KEYBOARD HIT PROGRAM *****/

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <unistd.h>
#include <pthread.h>
#include <ctype.h>
#include <errno.h>
#include <fcntl.h>
#include <sys/stat.h>

extern int errno;

void *check_hit()
{
char c;
int fd;
int i;
printf("Entering routine to check hit.........\n");
while(1) {
fd = open("fifo", O_RDONLY);
if(fd < 0) {
printf("Error opening in fifo\n");
printf("errno is %d\n", errno);
continue;
}
i = read(fd, &c, 1);
if(i < 0) {
printf("Error reading fifo\n");
printf("errno is %d\n", errno);
}
if(isalnum(c)) {
printf("The key hit is %c\n", c);
exit(1);
} else {
printf("key hit is %c\n", c);
}
}
}

int main()
{
int i;
i = mkfifo("fifo", 0666);
if(i < 0) {
printf("Problems creating the fifo\n");
if(errno == EEXIST) {
printf("fifo already exists\n");
}
printf("errno is set as %d\n", errno);
}
pthread_t tid2;
i = pthread_create(&tid2, NULL, check_hit, NULL);
if(i == 0) while(1);
return 0;
}
Here, again, it first tries to create a fifo which is created if it does not exist. We then have the
thread check_hit which tries to read characters from the fifo. If the read character is
alphanumeric, the program terminates; otherwise the thread continues reading characters from
the fifo.

Here, the fifo is opened with the flag O_RDONLY.

Compile this program with cc -o detect_hit filename.c. Now run the two executables in
separate terminals, but in the same working directory. Irrespective of the order in which you run,
look for the message fifo already exists on the console. The first program (either of the two) that
you run will not give any error message for creation of the fifo. The second program that you run
will definitely give you the error for creation of the fifo. In the terminal where you run
write_fifo, give input to standard output from your keyboard. You will get the message
regarding the key hit on the keyboard on the terminal running the executable detect_hit.
Analyze the working of the two programs by hitting several keys.

I have used two different programs for exhibiting the usage of fifos. This can be done within a
single program by forking the routines which are called in the two program as threads. But I did
this to show that unlike pipes, fifos can be used for communication between unrelated processes.

Now try running the program again. You will get the message that the fifo already exists even
when you first run either of the two programs. This shows that fifos are persistent as long as the
system lives. That is, the fifos will have to be removed manually - otherwise they will be
permanently recognized by the file system. This is unlike pipes which are inherited as long as
the process that created the pipe is running. Once this process dies, the kernel also removes the
identifiers (file descriptors) for the pipe from the the file tables.

The usage is rather simple and the main advantage is that there is no need for any
synchronization mechanism for accesses to the fifo. There are certain disadvantages: they can
only be used for communication between processes running on the same host machine. Let us
explore other IPC mechanisms to see what have they in store.

Shared Memory
Shared Memory is one of the three kinds of System V IPC mechanism which enables different
processes to communicate with each other as if these processes shared the virtual address space;
hence, any process sharing the memory region can read or write to it. One can imagine some part
of memory being set aside for use by different processes.

The System V IPC describes the use of the shared memory mechanism as consisting of four
steps. Taken in order, they are:

• Fetching an identifier for the shared memory area - shmget (shared memory get)
• Using the identifier to get the shared memory address - shmat (shared memory attach),
• Detaching the shared memory area after use - shmdt (shared memory detach) and
 • Finally using the address to control accesses, permissions, receive information and
destroy the shared memory area - shmctl (shared memory control).

Let us examine the workings of the above system calls. Recall the keyboard hit program; we
shall, once again, see another version of it, this time using the system calls associated with the
shared memory mechanism.

The code given below creates a shared memory area and stores the information of any key hit on
the keyboard. Let us see the code first: (text version: write-shm.c.txt)

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <errno.h>
#include <string.h>
#include <ctype.h>

extern int errno;

#define SIZE 1

char *read_key;
int shmid;

int shared_init()
{
if((shmid = shmget(9999, SIZE, IPC_CREAT | 0666)) < 0) {
printf("Error in shmget. errno is: %d\n", errno);
return -1;
}
if((read_key = shmat(shmid, NULL, 0)) < 0) {
printf("Error in shm attach. errno is: %d\n", errno);
return -1;
}
return 0;
}

void read_char()
{
char c;
while(1) {
c = getchar();
if(c == '\n') {
c = getchar();
}
strncpy(read_key, &c, SIZE);
printf("read_key now is %s\n", read_key);
if(isalnum(*read_key)) {
shmdt(read_key);
shmctl(shmid, IPC_RMID, NULL);
exit(1);
}
 }
}

int main()
{
if(shared_init() < 0) {
printf("Problems with shared memory\n");
exit(1);
}
read_char();
return 0;
}

Here we have a shared memory variable named read_key. The program first initializes the
shared memory area read_key. This is done by generating a shared memory identifier shmid
using the system call shmget. In the context of the program, the first parameter for shmget is
9999, which is the key. This key is used to allocate a shared memory segment. The second
parameter, SIZE (defined as a macro with the value 1), suggests that the shared memory segment
will hold only one of the type of the shared memory variable, that is, only 1 character. The
IPC_CREAT flag (third parameter) suggests that a new shared memory segment has to be created,
with read-write permissions (IPC_CREAT logically OR ed with 0666). This will return a valid
shared memory segment identifier on successful allocation. The identifier will be stored in
shmid. If shared memory segment allocation fails, then -1 is returned and the errno is set
appropriately.

The key which is used to get a shared memory segment can be generated randomly using the
built-in function ftok to get a unique key. Refer to the manual page for the usage.

Once the segment identifier is obtained, we have to attach the shared memory segment to some
address. This is done with the shmat system call. This uses the segment identifier shmid as the
first parameter. The second parameter is the address of the shared memory segment; when this is
given as NULL (as in this program), the kernel will choose a suitable address. The third parameter
is the flag specification which can be set if required or left as zero (see man page of shmdt for
details). On success the shared memory segment is attached to read_key, otherwise -1 is
returned along with the appropriate setting of the errno.

If either shmget or shmat fails, the process terminates. On success from both system calls, we
proceed by invoking the read_char function, which reads keyboard inputs other than '\n'
("Enter" key) and copies them to read_key in the shared memory. If the keyboard input is an
alphanumeric character, the program stops reading inputs from the keyboard and the process
terminates.

We have another program running separately (it does not have to be in the same working
directory) in the local system, which tries to read the data written in the shared memory area. The
code is given below: (text version: read-shm.c.txt)

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#include <errno.h>
#include <string.h>
#include <ctype.h>

extern int errno;

#define SIZE 1

char *detect_key;
int shmid;

int shared_init()
{
if((shmid = shmget(9999, SIZE, 0444)) < 0) {
printf("Error in shmget. errno is: %d\n", errno);
return -1;
}
if((detect_key = shmat(shmid, NULL, SHM_RDONLY)) < 0) {
printf("Error in shm attach. errno is: %d\n", errno);
return -1;
}
// detect_key = NULL;
return 0;
}

void detect_hit()
{
char c;
c = *detect_key;
while(1) {
if(c != *detect_key) {
if(isalnum(detect_key[0])) {
printf("detect_key is %s\n", detect_key);
shmdt(detect_key);
shmctl(shmid, IPC_RMID, NULL);
exit(1);
} else {
printf("detect_key is %s\n", detect_key);
}
c = *detect_key;
}
}
}

int main()
{
if(shared_init() < 0) {
printf("Problems with shared memory\n");
exit(1);
}
detect_hit();
return 0;
}
Here, again, we have a shared memory initialization routine, which in fact does not create a new
shared memory segment, but rather tries to get access to the existing shared memory segment.
Compared to the previous program, the absence of IPC_CREAT flag suggests that we do not have
to create a new shared memory segment. Instead, we simply have to get the corresponding
segment identifier which can be used to attach the existing shared memory segment to some
address. The mode 0444 restricts access to the shared memory segment to 'read only'. If no
shared memory segment with key 9999 exists, we will get an error, which will be returned in
errno. Once we get a valid identifier, we attach the shared memory segment to an address. While
attaching, we use the flag SHM_RDONLY which specifies that the shared memory segment will be
available only for reading.

Next, we have the function detect_hit, which checks whether the pressed key was an
alphanumeric character. The first program obviously has to run first; otherwise, the second
program will show errors during the shared memory initialization, since it would be trying to get
the identifier for a non-existent shared memory segment.

The example shown here doesn't require any synchronization of access to the shared memory
segment. That is because only one program writes into the shared memory and only one program
reads from the shared memory area. But again, there is a problem here. What if the detection
program (second one) is started long after some user has started hitting the keys (running the first
program)? We will not be able to track the previously hit keys. The solution for this is left as an
exercise to the readers.

The entry in /proc/sysvipc/shm gives a list of shared mermory in use. Readers can compare
the entries before running, during running and after running the programs. Try to interpret the
entry in /proc/sysvipc/shm.

Once the two programs identify an alphanumeric character, they will terminate. As part of that
process, the shared memory area is detached by using the system call shmdt. In fact, upon
exiting the detaching is done automatically. But the shared memory segment is not destroyed.
This has to be done by invoking the system call shmctl, which takes the identifier for the shared
memory area as an argument, as well as the command IPC_RMID, which marks the shared
memory segment as destroyed. This has to be done, otherwise the shared memory segment will
persist in memory or in the swap space.

At this point, observation of the entries in /proc/sysvipc/shm can be very useful. If the shared
memory segment is not destroyed, the entries will reflect this. Try this by running the program
without shmctl.

This is the fastest IPC mechanism in the System V IPC services. However, the System V shared
memory mechanism does not have any kind of scheme to ensure that one sees consistent data in
the shared memory region. That is, a process can read a shared memory area at the same time
another process is writing to it. The programmer can then come across inconsistent data while
executing the programs. This suggests that accesses to the shared memory region have to be
mutually exclusive; this is achieved via the use of the semaphore mechanism. We can make the
semaphore access the memory region to lock it and then release the semaphore when done.
The shared memory mechanism can be used when the processes access the shared memory areas
at different times. One may wonder why we can't make the first process store the data in some
file and make another process read the data from the file. But, reading data from a file involves
things like:

• execution of system calls like open, read and close.

• accessing the secondary storage device (generally hard disk) which involves I/O
operations.

These things are not significant if we have a small amount of data to be read. But when we have
large amounts of data stored in a file, then the load of the two activities mentioned above
increases significantly and there is a considerable amount of reduction in the performance of the
"reading program". This, again, has a solution, which is the use of memory mapping - something
I'll discuss at another time.

Conclusion
We have seen the use of the primary IPC mechanisms, and also one of the System V IPC
mechanisms. We have seen some simple uses for pipes, fifos, and the shared memory
mechanism. But one may come across some very complex programs where these mechanisms
will have to be used in a very strict and precise manner. Otherwise, the program along with the
programmer, will be dumped to /dev/null. There are still more things to be learned, not only
for you but also for me. I shall come up with more in the next part, in which we will explore
semaphores, message queues, memory mapping and sockets, and probably try to solve a few
practical problems.