Opensource.

com

A guide to inter-process
communication in Linux

Learn how processes synchronize

with each other in Linux.
By Marty Kalin

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 1

Opensource.com . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

About Opensource.com

What is Opensource.com?

Opensource.com publishes stories about creating,

adopting, and sharing open source
solutions. Visit Opensource.com to learn more about how the open source
way is improving technologies, education, business, government, health, law,
entertainment, humanitarian efforts, and more.

Submit a story idea: opensource.com/story

Email us: [email protected]

2 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . About the Author

Marty Kalin

I’m an academic in computer science (College of Computing and

Digital Media, DePaul University) with wide
experience in software development, mostly in production planning and scheduling
(steel industry) and product configuration (truck and bus manufacturing). Details
on books and other publications are available at:
Marty Kalin’s hompage

Follow Marty Kalin

Twitter: @kalin_martin

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 3

Opensource.com . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Introduction

Introduction 5

Chapters

Shared storage 6
Using pipes and message queues 12
Sockets and signals 19
Wrapping up this guide 24

Get Involved | Additional Resources

Write for Us 25

4 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Introduction

Introduction
This guide is about
the following IPC mechanisms:
interprocess communication (IPC) in Linux. The
guide uses code examples in C to clarify

• Shared files
• Shared memory (with semaphores)
• Pipes (named and unnamed)
• Message queues
• Sockets
• Signals

I’ll introduce you to some core concepts before moving on to the first two of these mech-
anisms: shared files and shared memory.

Core concepts
A process is a program in execution, and each process has its own address space, which
comprises the memory locations that the process is allowed to access. A process has
one or more threads of execution, which are sequences of executable instructions: a
single-threaded process has just one thread, whereas a multi-threaded process has more
than one thread. Threads within a process share various resources, in particular, address
space. Accordingly, threads within a process can communicate straightforwardly through
shared memory, although some modern languages (e.g., Go) encourage a more disci-
plined approach such as the use of thread-safe channels. Of interest here is that different
processes, by default, do not share memory.
There are various ways to launch processes that then communicate, and two ways
dominate in the examples that follow:

• A terminal is used to start one process, and perhaps a different terminal is used to
start another.
• The system function fork is called within one process (the parent) to spawn another
process (the child).

The first examples take the terminal approach. The code examples [1] are available in a
ZIP file on my website.

Links
[1] h
 ttp://condor.depaul.edu/mkalin

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 5

Shared storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Shared
storage
Learn how processes synchronize with each
other in Linux.

Example 1. The producer program

Programmers are
all too familiar with file access, including
#include <stdio.h>
#include <stdlib.h>

the many pitfalls (non-existent files, bad file #include <fcntl.h>

#include <unistd.h>
permissions, and so on) that beset the use
#include <string.h>
of files in programs. Nonetheless, shared
files may be the most basic IPC mecha-
#define FileName "data.dat"
nism. Consider the relatively simple case in
#define DataString "Now is the winter of our discontent\nMade glorious summer by
which one process (producer) creates and
#this sun of York\n"
writes to a file, and another process (con-
sumer) reads from this same file: void report_and_exit(const char* msg) {
perror(msg);
writes +-----------+ reads exit(-1); /* EXIT_FAILURE */
producer-------->| disk file |<-------consumer }
+-----------+
int main() {
The obvious challenge in using this IPC struct flock lock;
mechanism is that a race condition might lock.l_type = F_WRLCK; /* read/write (exclusive versus shared) lock */

arise: the producer and the consumer might lock.l_whence = SEEK_SET; /* base for seek offsets */
lock.l_start = 0; /* 1st byte in file */
access the file at exactly the same time,
lock.l_len = 0; /* 0 here means 'until EOF' */
thereby making the outcome indeterminate.
lock.l_pid = getpid(); /* process id */
To avoid a race condition, the file must be
locked in a way that prevents a conflict be-
int fd; /* file descriptor to identify a file within a process */
tween a write operation and any another
if ((fd = open(FileName, O_RDWR | O_CREAT, 0666)) < 0) /* -1 signals an error */
operation, whether a read or a write. The report_and_exit("open failed...");
locking API in the standard system library
can be summarized as follows: if (fcntl(fd, F_SETLK, &lock) < 0) /** F_SETLK doesn't block, F_SETLKW does **/
• A producer should gain an exclusive lock report_and_exit("fcntl failed to get lock...");
on the file before writing to the file. An ex- else {
clusive lock can be held by one process write(fd, DataString, strlen(DataString)); /* populate data file */
at most, which rules out a race condition fprintf(stderr, "Process %d has written to data file...\n", lock.l_pid);
because no other process can access the }
file until the lock is released.
• A consumer should gain at least a shared /* Now release the lock explicitly. */
lock.l_type = F_UNLCK;
lock on the file before reading from the file.
if (fcntl(fd, F_SETLK, &lock) < 0)
Multiple readers can hold a shared lock at
report_and_exit("explicit unlocking failed...");
the same time, but no writer can access
a file when even a single reader holds a
close(fd); /* close the file: would unlock if needed */
shared lock.
return 0; /* terminating the process would unlock as well */
A shared lock promotes efficiency. If one }
process is just reading a file and not

6 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shared storage

changing its contents, there is no reason to prevent other • If the producer gains the lock, the program writes two text
processes from doing the same. Writing, however, clearly records to the file.
demands exclusive access to a file. • After writing to the file, the producer changes the lock
The standard I/O library includes a utility function named structure’s l_type field to the unlock value:
fcntl that can be used to inspect and manipulate both exclusive
and shared locks on a file. The function works through a file lock.l_type = F_UNLCK;
descriptor, a non-negative integer value that, within a process,
identifies a file. (Different file descriptors in different processes and calls fcntl to perform the unlocking operation. The
may identify the same physical file.) For file locking, Linux pro- program finishes up by closing the file and exiting (see
vides the library function flock, which is a thin wrapper around Example 2).
fcntl. The first example uses the fcntl function to
expose API details (see Example 1). Example 2. The consumer program
The main steps in the producer program above #include <stdio.h>
can be summarized as follows: #include <stdlib.h>
• The program declares a variable of type struct #include <fcntl.h>
flock, which represents a lock, and initializes the #include <unistd.h>
structure’s five fields. The first initialization:
#define FileName "data.dat"

lock.l_type = F_WRLCK; /* exclusive lock */

void report_and_exit(const char* msg) {
perror(msg);
 akes the lock an exclusive (read-write) rather
m exit(-1); /* EXIT_FAILURE */
than a shared (read-only) lock. If the producer }
gains the lock, then no other process will be
able to write or read the file until the produc- int main() {
er releases the lock, either explicitly with the struct flock lock;
appropriate call to fcntl or implicitly by closing lock.l_type = F_WRLCK; /* read/write (exclusive) lock */
lock.l_whence = SEEK_SET; /* base for seek offsets */
the file. (When the process terminates, any
lock.l_start = 0; /* 1st byte in file */
opened files would be closed automatically,
lock.l_len = 0; /* 0 here means 'until EOF' */
thereby releasing the lock.) lock.l_pid = getpid(); /* process id */
• The program then initializes the remaining fields.
The chief effect is that the entire file is to be int fd; /* file descriptor to identify a file within a process */
locked. However, the locking API allows only if ((fd = open(FileName, O_RDONLY)) < 0) /* -1 signals an error */
designated bytes to be locked. For example, if report_and_exit("open to read failed...");
the file contains multiple text records, then a sin-
/* If the file is write-locked, we can't continue. */
gle record (or even part of a record) could be
fcntl(fd, F_GETLK, &lock); /* sets lock.l_type to F_UNLCK if no write lock */
locked and the rest left unlocked.
if (lock.l_type != F_UNLCK)
• The first call to fcntl: report_and_exit("file is still write locked...");

if (fcntl(fd, F_SETLK, &lock) < 0) lock.l_type = F_RDLCK; /* prevents any writing during the reading */
if (fcntl(fd, F_SETLK, &lock) < 0)
tries to lock the file exclusively, checking wheth- report_and_exit("can't get a read-only lock...");
er the call succeeded. In general, the fcntl func-
/* Read the bytes (they happen to be ASCII codes) one at a time. */
tion returns -1 (hence, less than zero) to indicate
int c; /* buffer for read bytes */
failure. The second argument F_SETLK means
while (read(fd, &c, 1) > 0) /* 0 signals EOF */
that the call to fcntl does not block: the function write(STDOUT_FILENO, &c, 1); /* write one byte to the standard output */
returns immediately, either granting the lock or in-
dicating failure. If the flag F_SETLKW (the W at /* Release the lock explicitly. */
the end is for wait) were used instead, the call to lock.l_type = F_UNLCK;
fcntl would block until gaining the lock was pos- if (fcntl(fd, F_SETLK, &lock) < 0)
sible. In the calls to fcntl, the first argument fd is report_and_exit("explicit unlocking failed...");

the file descriptor, the second argument specifies

close(fd);
the action to be taken (in this case, F_SETLK for
return 0;
setting the lock), and the third argument is the ad-
}
dress of the lock structure (in this case, &lock).

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 7

Shared storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The consumer program is more complicated than neces- cess involves reading or writing. As always, programming
sary to highlight features of the locking API. In particular, the comes with tradeoffs. The next example has the upside of
consumer program first checks whether the file is exclusively IPC through shared memory, rather than shared files, with a
locked and only then tries to gain a shared lock. The relevant corresponding boost in performance.
code is:
Shared memory
lock.l_type = F_WRLCK; Linux systems provide two separate APIs for shared mem-
... ory: the legacy System V API and the more recent POSIX
fcntl(fd, F_GETLK, &lock); /* 
sets lock.l_type to F_UNLCK if no one. These APIs should never be mixed in a single appli-
write lock */ cation, however. A downside of the POSIX approach is
if (lock.l_type != F_UNLCK) that features are still in development and dependent upon
report_and_exit("file is still write locked..."); the installed kernel version, which impacts code portabil-
ity. For example, the POSIX API, by default, implements
The F_GETLK operation specified in the fcntl call checks shared memory as a memory-mapped file: for a shared
for a lock, in this case, an exclusive lock given as F_WRLCK memory segment, the system maintains a backing file with
in the first statement above. If the specified lock does not corresponding contents. Shared memory under POSIX
exist, then the fcntl call automatically changes the lock type can be configured without a backing file, but this may im-
field to F_UNLCK to indicate this fact. If the file is exclusively pact portability. My example uses the POSIX API with a
locked, the consumer terminates. (A more robust version of backing file, which combines the benefits of memory ac-
the program might have the consumer sleep a bit and try cess (speed) and file storage (persistence).
again several times.) The shared-memory example has two programs, named
If the file is not currently locked, then the consumer tries memwriter and memreader, and uses a semaphore to co-
to gain a shared (read-only) lock (F_RDLCK). To shorten ordinate their access to the shared memory. Whenever
the program, the F_GETLK call to fcntl could be dropped shared memory comes into the picture with a writer, whether
because the F_RDLCK call would fail if a read-write lock in multi-processing or multi-threading, so does the risk of a
already were held by some other process. Recall that a memory-based race condition; hence, the semaphore is used
read-only lock does prevent any other process from writing to coordinate (synchronize) access to the shared memory.
to the file, but allows other processes to
read from the file. In short, a shared lock Example 3. Source code for the memwriter process
can be held by multiple processes. After /** Compilation: gcc -o memwriter memwriter.c -lrt -lpthread **/
gaining a shared lock, the consumer pro- #include <stdio.h>
gram reads the bytes one at a time from #include <stdlib.h>
the file, prints the bytes to the standard #include <sys/mman.h>
output, releases the lock, closes the file, #include <sys/stat.h>
and terminates. #include <fcntl.h>
Here is the output from the two pro- #include <unistd.h>
grams launched from the same terminal #include <semaphore.h>
with % as the command line prompt: #include <string.h>
#include "shmem.h"
% ./producer
Process 29255 has written to data file...
void report_and_exit(const char* msg) {
perror(msg);
% ./consumer
exit(-1);
Now is the winter of our discontent
}
Made glorious summer by this sun of York

int main() {
In this first code example, the data shared
int fd = shm_open(BackingFile, /* name from smem.h */
through IPC is text: two lines from Shake-
O_RDWR | O_CREAT, /* read/write, create if needed */
speare’s play Richard III. Yet, the shared
AccessPerms); /* access permissions (0644) */
file’s contents could be voluminous, arbi-
if (fd < 0) report_and_exit("Can't open shared mem segment...");
trary bytes (e.g., a digitized movie), which
makes file sharing an impressively flexible
ftruncate(fd, ByteSize); /* get the bytes */
IPC mechanism. The downside is that file
access is relatively slow, whether the ac-

8 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shared storage

The memwriter program should be started first in its own • The memwriter then calls the mmap function:
terminal. The memreader program then can be started
(within a dozen seconds) in its own terminal. The output caddr_t memptr = mmap(NULL, /* let system pick where to
from the memreader is: put segment */
ByteSize, /* how many bytes */
This is the way the world ends... PROT_READ | PROT_WRITE, /* access
protections */
Each source file has documentation at the top explaining the MAP_SHARED, /* mapping visible to other
link flags to be included during compilation. processes */
Let’s start with a review of how semaphores work as a fd, /* file descriptor */
synchronization mechanism. A general semaphore also is 0); /* offset: start at 1st byte */
called a counting semaphore, as it has a value (typically
initialized to zero) that can be incremented. Consider a to get a pointer to the shared memory. (The memreader
shop that rents bicycles, with a hundred of them in stock, makes a similar call.) The pointer type caddr_t starts
with a program that clerks use to do the rentals. Every with a c for calloc, a system function that initializes dy-
time a bike is rented, the semaphore is incremented by namically allocated storage to zeroes. The memwriter
one; when a bike is returned, the semaphore is decre- uses the memptr for the later write operation, using the
mented by one. Rentals can continue until the value hits library strcpy (string copy) function.
100 but then must halt until at least one
bike is returned, thereby decrementing Example 3. Source code for the memwriter process (continued)
the semaphore to 99. caddr_t memptr = mmap(NULL, /* let system pick where to put segment */
A binary semaphore is a special case ByteSize, /* how many bytes */
requiring only two values: 0 and 1. In PROT_READ | PROT_WRITE, /* access protections */
this situation, a semaphore acts as a MAP_SHARED, /* mapping visible to other processes */
mutex: a mutual exclusion construct.
fd, /* file descriptor */
The shared-memory example uses a
0); /* offset: start at 1st byte */
semaphore as a mutex. When the sema-
if ((caddr_t) -1 == memptr) report_and_exit("Can't get segment...");
phore’s value is 0, the memwriter alone
can access the shared memory. After
fprintf(stderr, "shared mem address: %p [0..%d]\n", memptr, ByteSize - 1);
writing, this process increments the
fprintf(stderr, "backing file: /dev/shm%s\n", BackingFile );
semaphore’s value, thereby allowing the
memreader to read the shared memory
/* semaphore code to lock the shared mem */
(see Example 3).
sem_t* semptr = sem_open(SemaphoreName, /* name */
Here’s an overview of how the memwrit-
O_CREAT, /* create the semaphore */
er and memreader programs communi-
cate through shared memory: AccessPerms, /* protection perms */

• The memwriter program, shown above, 0); /* initial value */

calls the shm_open function to get if (semptr == (void*) -1) report_and_exit("sem_open");

a file descriptor for the backing file

that the system coordinates with the strcpy(memptr, MemContents); /* copy some ASCII bytes to the segment */
shared memory. At this point, no mem-
ory has been allocated. The subse- /* increment the semaphore so that memreader can read */
quent call to the misleadingly named if (sem_post(semptr) < 0) report_and_exit("sem_post");
function ftruncate:
sleep(12); /* give reader a chance */
ftruncate(fd, ByteSize); /* get the bytes */
/* clean up */
allocates ByteSize bytes, in this case, munmap(memptr, ByteSize); /* unmap the storage */
a modest 512 bytes. The memwriter close(fd);
and memreader programs access the sem_close(semptr);
shared memory only, not the backing shm_unlink(BackingFile); /* unlink from the backing file */
file. The system is responsible for syn- return 0;
chronizing the shared memory and the }
backing file.

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 9

Shared storage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

 t this point, the memwriter is ready for writing, but it first

• A calls to shm_open are similar in the two programs except that
creates a semaphore to ensure exclusive access to the the memwriter program creates the shared memory, whereas
shared memory. A race condition would occur if the mem- the memreader only accesses this already created memory:
writer were writing while the memreader was reading. If the
call to sem_open succeeds: int fd = shm_open(BackingFile, O_RDWR | O_CREAT, AccessPerms);
/* memwriter */
sem_t* semptr = sem_open(SemaphoreName, /* name */ int fd = shm_open(BackingFile, O_RDWR, AccessPerms);
O_CREAT, /* create the semaphore */ /* memreader */
AccessPerms, /* protection perms */
0); /* initial value */ With a file descriptor in hand, the calls to mmap are the
same:
then the writing can proceed. The SemaphoreName (any
unique non-empty name will do) identifies the semaphore ca
ddr_t memptr = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_
in both the memwriter and the memreader. The initial val- SHARED, fd, 0);
ue of zero gives the semaphore’s creator, in this case, the
memwriter, the right to proceed, in this case, to the write The first argument to mmap is NULL, which means that the
operation. system determines where to allocate the memory in virtual
• After writing, the memwriter increments the semaphore address space. It’s possible (but tricky) to specify an address
value to 1:
Example 4. Source code for the memreader process
if (sem_post(semptr) < 0) .. /** Compilation: gcc -o memreader memreader.c -lrt -lpthread **/
#include <stdio.h>
 ith a call to the sem_post function. In-
w #include <stdlib.h>
crementing the semaphore releases the #include <sys/mman.h>
mutex lock and enables the memreader #include <sys/stat.h>
to perform its read operation. For good #include <fcntl.h>
measure, the memwriter also unmaps the #include <unistd.h>
shared memory from the memwriter ad- #include <semaphore.h>

dress space: #include <string.h>

#include "shmem.h"

munmap(memptr, ByteSize); /* unmap the storage *

void report_and_exit(const char* msg) {
perror(msg);
This bars the memwriter from further access
exit(-1);
to the shared memory (see Example 4). }
In both the memwriter and memreader pro-
grams, the shared-memory functions of main in- int main() {
terest are shm_open and mmap: on success, int fd = shm_open(BackingFile, O_RDWR, AccessPerms); /* empty to begin */
the first call returns a file descriptor for the back- if (fd < 0) report_and_exit("Can't get file descriptor...");
ing file, which the second call then uses to get
a pointer to the shared memory segment. The /* get a pointer to memory */
caddr_t memptr = mmap(NULL, /* let system pick where to put segment */
ByteSize, /* how many bytes */
PROT_READ | PROT_WRITE, /* access protections */
MAP_SHARED, /* mapping visible to other processes */
fd, /* file descriptor */
0); /* offset: start at 1st byte */

10 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Shared storage

instead. The MAP_SHARED flag indicates that the allocated The memreader, like the memwriter, accesses the sema-
memory is shareable among processes, and the last argu- phore through its name in a call to sem_open. But the mem-
ment (in this case, zero) means that the offset for the shared reader then goes into a wait state until the memwriter incre-
memory should be the first byte. The size argument speci- ments the semaphore, whose initial value is 0:
fies the number of bytes to be allocated (in this case, 512),
and the protection argument indicates that the shared mem- if (!sem_wait(semptr)) { /* wait until semaphore != 0 */
ory can be written and read.
When the memwriter program executes successfully, the Once the wait is over, the memreader reads the ASCII bytes
system creates and maintains the backing file; on my sys- from the shared memory, cleans up, and terminates.
tem, the file is /dev/shm/shMemEx, with shMemEx as my The shared-memory API includes operations explicitly to
name (given in the header file shmem.h) for the shared stor- synchronize the shared memory segment and the backing
age. In the current version of the memwriter and memreader file. These operations have been omitted from the example
programs, the statement: to reduce clutter and keep the focus on the memory-sharing
and semaphore code.
shm_unlink(BackingFile); /* removes backing file */ The memwriter and memreader programs are likely to exe-
cute without inducing a race condition even if the semaphore
removes the backing file. If the unlink statement is omitted, code is removed: the memwriter creates the shared memory
then the backing file persists after the program terminates. segment and writes immediately to it; the memreader cannot
even access the shared memory until this has
Example 4. Source code for the memreader process (continued) been created. However, best practice requires
if ((caddr_t) -1 == memptr) report_and_exit("Can't access segment..."); that shared-memory access is synchronized
whenever a write operation is in the mix, and
/* create a semaphore for mutual exclusion */ the semaphore API is important enough to be
sem_t* semptr = sem_open(SemaphoreName, /* name */ highlighted in a code example.
O_CREAT, /* create the semaphore */
AccessPerms, /* protection perms */ Wrapping up
0); /* initial value */ The shared-file and shared-memory exam-
if (semptr == (void*) -1) report_and_exit("sem_open"); ples show how processes can communicate
through shared storage, files in one case and
/* use semaphore as a mutex (lock) by waiting for writer to increment it */ memory segments in the other. The APIs for
if (!sem_wait(semptr)) { /* wait until semaphore != 0 */ both approaches are relatively straightfor-
int i; ward. Do these approaches have a common
for (i = 0; i < strlen(MemContents); i++) downside? Modern applications often deal with
write(STDOUT_FILENO, memptr + i, 1); /* one byte at a time */ streaming data, indeed, with massively large
sem_post(semptr);
streams of data. Neither the shared-file nor the
}
shared-memory approaches are well suited for
massive data streams. Channels of one type or
/* cleanup */
another are better suited. Part 2 thus introduc-
munmap(memptr, ByteSize);
es channels and message queues, again with
close(fd);
code examples in C.
sem_close(semptr);
unlink(BackingFile);
return 0;
}

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 11

Using pipes and message queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Using pipes and

message queues
Learn how processes synchronize with each other in Linux.

This section turns to pipes, which

are channels
that connect processes for communication. A channel has a
one of its threads executes, and if a race condition cannot
arise among threads in the same process, such a condition
cannot arise among threads in different processes. The MT-
write end for writing bytes, and a read end for reading these Safe attribute assures that a race condition does not arise in
bytes in FIFO (first in, first invocations of mq_open. In
out) order. In typical use, one general, channel-based IPC
process writes to the chan- is concurrent-safe, although
nel, and a different process a cautionary note is raised in
reads from this same chan- the examples that follow.
nel. The bytes themselves
might represent anything: Unnamed pipes
numbers, employee records, Let’s start with a contrived
digital movies, and so on. command line example that
Pipes come in two flavors, shows how unnamed pipes
named and unnamed, and work. On all modern sys-
can be used either interac- tems, the vertical bar | rep-
tively from the command line resents an unnamed pipe at
or within programs; examples are forthcoming. This section the command line. Assume % is the command line prompt,
also looks at memory queues, which have fallen out of fash- and consider this command:
ion—but undeservedly so.
The code examples in the first section acknowledged the % 
sleep 5 | echo "Hello, world!" ## writer to the left of |,
threat of race conditions (either file-based or memory-based) reader to the right
in IPC that uses shared storage. The question naturally
arises about safe concurrency for the channel-based IPC, The sleep and echo utilities execute as separate processes,
which will be covered in this section. The code examples and the unnamed pipe allows them to communicate. How-
for pipes and memory queues use APIs with the POSIX ever, the example is contrived in that no communication oc-
stamp of approval, and a core goal of the POSIX standards curs. The greeting Hello, world! appears on the screen; then,
is thread-safety. after about five seconds, the command line prompt returns,
Consider the man pages for the mq_open [1] function, indicating that both the sleep and echo processes have exit-
which belongs to the memory queue API. These pages in- ed. What’s going on?
clude a section on Attributes [2] with this small table: In the vertical-bar syntax from the command line, the pro-
cess to the left (sleep) is the writer, and the process to the
Interface Attribute Value right (echo) is the reader. By default, the reader blocks until
mq_open() Thread safety MT-Safe there are bytes to read from the channel, and the writer—
after writing its bytes—finishes up by sending an end-of-
The value MT-Safe (with MT for multi-threaded) means that stream marker. (Even if the writer terminates prematurely, an
the mq_open function is thread-safe, which in turn implies end-of-stream marker is sent to the reader.) The unnamed
process-safe: A process executes in precisely the sense that pipe persists until both the writer and the reader terminate.

12 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using pipes and message queues

In the contrived example, the sleep process does not parent in case of failure. In the pipeUN example, the
write any bytes to the channel but does terminate after call is:
about five seconds, which sends an end-of-stream mark-
er to the channel. In the meantime, the echo process im- pid_t cpid = fork(); /* called in parent */
mediately writes the greeting to the standard output (the
screen) because this process does not read any bytes The returned value is stored, in this example, in the variable
from the channel, so it does no waiting. Once the sleep cpid of integer type pid_t. (Every process has its own pro-
and echo processes terminate, the unnamed pipe—not cess ID, a non-negative integer that identifies the process.)
used at all for communication—goes away and the com- Forking a new process could fail for several reasons, includ-
mand line prompt returns. ing a full process table, a structure that the system maintains
Here is a more useful example using two unnamed pipes. to track processes. Zombie processes, clarified shortly, can
Suppose that the file test.dat looks like this: cause a process table to fill if these are not harvested.

this Example 1. Two processes communicating through an unnamed pipe.

is
#include <sys/wait.h> /* wait */
the #include <stdio.h>
way #include <stdlib.h> /* exit functions */
the #include <unistd.h> /* read, write, pipe, _exit */
world #include <string.h>
ends
#define ReadEnd 0
#define WriteEnd 1
The command:
void report_and_exit(const char* msg) {
% cat test.dat | sort | uniq
perror(msg);
exit(-1); /** failure **/
pipes the output from the cat (con- }
catenate) process into the sort pro-
cess to produce sorted output, and int main() {
then pipes the sorted output into the int pipeFDs[2]; /* two file descriptors */
uniq process to eliminate duplicate char buf; /* 1-byte buffer */
records (in this case, the two occur- const char* msg = "Nature's first green is gold\n"; /* bytes to write */

rences of the reduce to one):

if (pipe(pipeFDs) < 0) report_and_exit("pipeFD");
pid_t cpid = fork(); /* fork a child process */
ends
if (cpid < 0) report_and_exit("fork"); /* check for failure */
is
the if (0 == cpid) { /*** child ***/ /* child process */
this close(pipeFDs[WriteEnd]); /* child reads, doesn't write */
way
world while (read(pipeFDs[ReadEnd], &buf, 1) > 0) /* read until end of byte stream */
write(STDOUT_FILENO, &buf, sizeof(buf)); /* echo to the standard output */

The scene now is set for a program

close(pipeFDs[ReadEnd]); /* close the ReadEnd: all done */
with two processes that communi-
_exit(0); /* exit and notify parent at once */
cate through an unnamed pipe (see
}
Example 1). else { /*** parent ***/
The pipeUN program above uses close(pipeFDs[ReadEnd]); /* parent writes, doesn't read */
the system function fork to create a
process. Although the program has write(pipeFDs[WriteEnd], msg, strlen(msg)); /* write the bytes to the pipe */
but a single source file, multi-pro- close(pipeFDs[WriteEnd]); /* done writing: generate eof */
cessing occurs during (successful)
execution. Here are the particulars wait(NULL); /* wait for child to exit */

in a quick review of how the library exit(0); /* exit normally */

}
function fork works:
return 0;
• The fork function, called in the
}
parent process, returns -1 to the

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 13

Using pipes and message queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

• If the fork call succeeds, it thereby spawns (creates) a One more aspect of the program needs clarification: the
new child process, returning one value to the parent but a call to the wait function in the parent code. Once spawned,
different value to the child. Both the parent and the child a child process is largely independent of its parent, as even
process execute the same code that follows the call to the short pipeUN program illustrates. The child can execute
fork. (The child inherits copies of all the variables declared arbitrary code that may have nothing to do with the parent.
so far in the parent.) In particular, a successful call to fork However, the system does notify the parent through a sig-
returns: nal—if and when the child terminates.
• Zero to the child process What if the parent terminates before the child? In this
• The child’s process ID to the parent case, unless precautions are taken, the child becomes and
• An if/else or equivalent construct typically is used after a remains a zombie process with an entry in the process table.
successful fork call to segregate code meant for the par- The precautions are of two broad types. One precaution is
ent from code meant for the child. In this example, the con- to have the parent notify the system that the parent has no
struct is: interest in the child’s termination:

if (0 == cpid) { /*** child ***/ signal(SIGCHLD, SIG_IGN); /* in parent: ignore notification */

...
} A second approach is to have the parent execute a wait on
else { /*** parent ***/ the child’s termination, thereby ensuring that the parent out-
... lives the child. This second approach is used in the pipeUN
} program, where the parent code has this call:

If forking a child succeeds, the pipeUN program proceeds as wait(NULL); /* called in parent */
follows. There is an integer array:
This call to wait means wait until the termination of any child
int pipeFDs[2]; /* two file descriptors */ occurs, and in the pipeUN program, there is only one child pro-
cess. (The NULL argument could be replaced with the address
to hold two file descriptors, one for writing to the pipe and of an integer variable to hold the child’s exit status.) There is a
another for reading from the pipe. (The array element more flexible waitpid function for fine-grained control, e.g., for
pipeFDs[0] is the file descriptor for the read end, and the specifying a particular child process among several.
array element pipeFDs[1] is the file descriptor for the write The pipeUN program takes another precaution. When the
end.) A successful call to the system pipe function, made parent is done waiting, the parent terminates with the call to
immediately before the call to fork, populates the array with the regular exit function. By contrast, the child terminates
the two file descriptors: with a call to the _exit variant, which fast-tracks notification
of termination. In effect, the child is telling the system to noti-
if (pipe(pipeFDs) < 0) report_and_exit("pipeFD"); fy the parent ASAP that the child has terminated.
If two processes write to the same unnamed pipe, can the
The parent and the child now have copies of both file de- bytes be interleaved? For example, if process P1 writes:
scriptors, but the separation of concerns pattern means that
each process requires exactly one of the descriptors. In this foo bar
example, the parent does the writing and the child does
the reading, although the roles could be reversed. The first to a pipe and process P2 concurrently writes:
statement in the child if-clause code, therefore, closes the
pipe’s write end: baz baz

close(pipeFDs[WriteEnd]); /* called in child code */ to the same pipe, it seems that the pipe contents might be
something arbitrary, such as:
and the first statement in the parent else-clause code closes
the pipe’s read end: baz foo baz bar

close(pipeFDs[ReadEnd]); /* called in parent code */ The POSIX standard ensures that writes are not interleaved
so long as no write exceeds PIPE_BUF bytes. On Linux sys-
The parent then writes some bytes (ASCII codes) to the un- tems, PIPE_BUF is 4,096 bytes in size. My preference with
named pipe, and the child reads these and echoes them to pipes is to have a single writer and a single reader, thereby
the standard output. sidestepping the issue.

14 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using pipes and message queues

Named pipes mkfifo(pipeName, 0666); /* read/write perms for user/group/others */

An unnamed pipe has no backing file: the system main- int fd = open(pipeName, O_CREAT | O_WRONLY);
tains an in-memory buffer to transfer bytes from the writ-
er to the reader. Once the writer and reader terminate, where pipeName is the name of the backing file passed
the buffer is reclaimed, so the unnamed pipe goes away. to mkfifo as the first argument. The named pipe then is
By contrast, a named pipe has a backing file and a dis- opened with the by-now familiar call to the open function,
tinct API. which returns a file descriptor.
Let’s look at another command line example to get the gist • For a touch of realism, the fifoWriter does not write all the
of named pipes. Here are the steps: data at once, but instead writes a chunk, sleeps a ran-
• Open two terminals. The working directory should be the dom number of microseconds, and so on. In total, 768,000
same for both. 4-byte integer values are written to the named pipe.
• In one of the terminals, enter these two commands (the • After closing the named pipe, the fifoWriter also unlinks
prompt again is %, and my comments start with ##): the file:

% mkfifo tester ## creates a backing file named tester close(fd); /* close pipe: generates end-of-stream marker */
% cat tester ## type the pipe's contents to stdout unlink(pipeName); /* unlink from the implementing file */

 t the beginning, nothing should appear

A Example 2. The fifoWriter program
in the terminal because nothing has #include <sys/types.h>
been written yet to the named pipe. #include <sys/stat.h>
• In the second terminal, enter the com- #include <fcntl.h>
mand: #include <unistd.h>
#include <time.h>
% cat > tester ## redirect keyboard input #include <stdlib.h>
to the pipe #include <stdio.h>

hello, world! ## then hit Return key

#define MaxLoops 12000 /* outer loop */
bye, bye ## ditto
#define ChunkSize 16 /* how many written at a time */
<Control-C> 
## terminate session with a
#define IntsPerChunk 4 /* four 4-byte ints per chunk */
## Control-C
#define MaxZs 250 /* max microseconds to sleep */

 hatever is typed into this terminal is

W int main() {
echoed in the other. Once Ctrl+C is en- const char* pipeName = "./fifoChannel";
tered, the regular command line prompt mkfifo(pipeName, 0666); /* read/write for user/group/others */
returns in both terminals: the pipe has int fd = open(pipeName, O_CREAT | O_WRONLY); /* open as write-only */
been closed. if (fd < 0) return -1; /* can't go on */
• Clean up by removing the file that imple-
ments the named pipe: int i;
for (i = 0; i < MaxLoops; i++) { /* write MaxWrites times */
int j;
% unlink tester
for (j = 0; j < ChunkSize; j++) { /* each time, write ChunkSize bytes */
int k;
As the utility’s name mkfifo implies, a
int chunk[IntsPerChunk];
named pipe also is called a FIFO be-
for (k = 0; k < IntsPerChunk; k++)
cause the first byte in is the first byte chunk[k] = rand();
out, and so on. There is a library write(fd, chunk, sizeof(chunk));
function named mkfifo that creates a }
named pipe in programs and is used usleep((rand() % MaxZs) + 1); /* pause a bit for realism */
in the next example, which consists of }
two processes: one writes to the named
pipe and the other reads from this pipe close(fd); /* close pipe: generates an end-of-stream marker */
(see Example 2). unlink(pipeName); /* unlink from the implementing file */

The fifoWriter program above can be printf("%i ints sent to the pipe.\n", MaxLoops * ChunkSize * IntsPerChunk);

summarized as follows:
return 0;
• The program creates a named pipe for
}
writing:

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 15

Using pipes and message queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

The system reclaims the backing file once every process const char* file = "./fifoChannel";
connected to the pipe has performed the unlink operation. int fd = open(file, O_RDONLY);
In this example, there are only two such processes: the
fifoWriter and the fifoReader, both of which do an unlink The file opens as read-only.
operation. • T
 he program then goes into a potentially infinite loop, try-
The two programs should be executed in different termi- ing to read a 4-byte chunk on each iteration. The read call:
nals with the same working directory. However, the fifoW-
riter should be started before the fifoReader, as the former ssize_t count = read(fd, &next, sizeof(int));
creates the pipe. The fifoReader then accesses the already
created named pipe (see Example 3). r eturns 0 to indicate end-of-stream, in which case the
The fifoReader program above can be summarized as fol- fifoReader breaks out of the loop, closes the named
lows: pipe, and unlinks the backing file before terminating.
• Because the fifoWriter creates the named pipe, the fifoRe- • After reading a 4-byte integer, the fifoReader checks
ader needs only the standard call open to access the pipe whether the number is a prime. This represents the busi-
through the backing file: ness logic that a production-grade reader might perform on
the received bytes. On a sample run, there
Example 3. The fifoReader program were 37,682 primes among the 768,000 in-
#include <stdio.h> tegers received.
#include <stdlib.h> On repeated sample runs, the fifoReader
#include <string.h> successfully read all of the bytes that the
#include <fcntl.h> fifoWriter wrote. This is not surprising. The
#include <unistd.h> two processes execute on the same host,
taking network issues out of the equation.
unsigned is_prime(unsigned n) { /* not pretty, but efficient */
Named pipes are a highly reliable and ef-
if (n <= 3) return n > 1;
if (0 == (n % 2) || 0 == (n % 3)) return 0;
ficient IPC mechanism and, therefore, in
wide use.
unsigned i; Here is the output from the two programs,
for (i = 5; (i * i) <= n; i += 6) each launched from a separate terminal but
if (0 == (n % i) || 0 == (n % (i + 2))) return 0; with the same working directory:

return 1; /* found a prime! */

% ./fifoWriter
}
768000 ints sent to the pipe.

int main() { ###

const char* file = "./fifoChannel"; % ./fifoReader
int fd = open(file, O_RDONLY); Received ints: 768000, primes: 37682
if (fd < 0) return -1; /* no point in continuing */
unsigned count = 0, total = 0, primes_count = 0; Message queues
Pipes have strict FIFO behavior: the first
while (1) {
byte written is the first byte read, the second
int next;
int i; byte written is the second byte read, and so
forth. Message queues can behave in the
ssize_t count = read(fd, &next, sizeof(int)); same way but are flexible enough that byte
if (0 == count) break; /* end of stream */ chunks can be retrieved out of FIFO order.
else if (count == sizeof(int)) { /* read a 4-byte int value */ As the name suggests, a message queue
total++;
is a sequence of messages, each of which
if (is_prime(next)) primes_count++;
has two parts:
}
}
• The payload, which is an array of bytes
(char in C)
close(fd); /* close pipe from read end */ • A type, given as a positive integer value;
unlink(file); /* unlink from the underlying file */ types categorize messages for flexible re-
printf("Received ints: %u, primes: %u\n", total, primes_count); trieval
Consider the following depiction of a mes-
return 0;
sage queue, with each message labeled with
}
an integer type:

16 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Using pipes and message queues

+-+ +-+ +-+ +-+ the message queue allows other retrieval orders. For ex-
sender--->|3|--->|2|--->|2|--->|1|--->receiver ample, the messages could be retrieved by the receiver in
+-+ +-+ +-+ +-+ the order 3-2-1-2.
The mqueue example consists of two programs, the send-
Of the four messages shown, the one labeled 1 is at the er that writes to the message queue and the receiver that
front, i.e., closest to the receiver. Next come two messag- reads from this queue. Both programs include the header file
es with label 2, and finally, a message labeled 3 at the queue.h shown below in Example 4:
back. If strict FIFO behavior were in play, then the mes- The header file defines a structure type named queued-
sages would be received in the order 1-2-2-3. However, Message, with payload (byte array) and type (integer)
fields. This file also defines symbolic constants
Example 4. The header file queue.h (the #define statements), the first two of which
#define ProjectId 123 are used to generate a key that, in turn, is used
#define PathName "queue.h" /* any existing, accessible file would do */ to get a message queue ID. The ProjectId can
#define MsgLen 4 be any positive integer value, and the PathName
#define MsgCount 6 must be an existing, accessible file—in this case,
the file queue.h. The setup statements in both the
typedef struct {
sender and the receiver programs are:
long type; /* must be of type long */
char payload[MsgLen + 1]; /* bytes in the message */
key_t key = ftok(PathName, ProjectId); /* 
generate key
} queuedMessage;
*/
int qid = msgget(key, 0666 | IPC_CREAT); /* 
use key
Example 5. The message sender program to get

#include <stdio.h> queue id */

#include <sys/ipc.h>
#include <sys/msg.h> The ID qid is, in effect, the counterpart of a file
#include <stdlib.h> descriptor for message queues (see Example 5).
#include <string.h> The sender program above sends out six mes-
#include "queue.h" sages, two each of a specified type: the first mes-
sages are of type 1, the next two of type 2, and
void report_and_exit(const char* msg) {
the last two of type 3. The sending statement:
perror(msg);
exit(-1); /* EXIT_FAILURE */
msgsnd(qid, &msg, sizeof(msg), IPC_NOWAIT);
}

int main() { is configured to be non-blocking (the flag IPC_

key_t key = ftok(PathName, ProjectId); NOWAIT) because the messages are so small.
if (key < 0) report_and_exit("couldn't get key..."); The only danger is that a full queue, unlikely in
this example, would result in a sending failure. The
int qid = msgget(key, 0666 | IPC_CREAT); receiver program below also receives messages
if (qid < 0) report_and_exit("couldn't get queue id..."); using the IPC_NOWAIT flag (see Example 6).
The receiver program does not create the mes-
char* payloads[] = {"msg1", "msg2", "msg3", "msg4", "msg5", "msg6"};
sage queue, although the API suggests as much.
int types[] = {1, 1, 2, 2, 3, 3}; /* each must be > 0 */
In the receiver, the call:
int i;
for (i = 0; i < MsgCount; i++) {
/* build the message */ int qid = msgget(key, 0666 | IPC_CREAT);
queuedMessage msg;
msg.type = types[i]; is misleading because of the IPC_CREAT flag, but
strcpy(msg.payload, payloads[i]); this flag really means create if needed, otherwise
access. The sender program calls msgsnd to
/* send the message */ send messages, whereas the receiver calls ms-
msgsnd(qid, &msg, sizeof(msg), IPC_NOWAIT); /* don't block */ grcv to retrieve them. In this example, the sender
printf("%s sent as type %i\n", msg.payload, (int) msg.type);
sends the messages in the order 1-1-2-2-3-3, but
}
the receiver then retrieves them in the order 3-1-
return 0;
}
2-1-3-2, showing that message queues are not
bound to strict FIFO behavior:

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 17

Using pipes and message queues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

% ./sender % ./receiver
msg1 sent as type 1 msg5 received as type 3
msg2 sent as type 1 msg1 received as type 1
msg3 sent as type 2 msg3 received as type 2
msg4 sent as type 2 msg2 received as type 1
msg5 sent as type 3 msg6 received as type 3
msg6 sent as type 3 msg4 received as type 2

Example 6. The message receiver program The output above shows that the sender and
#include <stdio.h> the receiver can be launched from the same
#include <sys/ipc.h> terminal. The output also shows that the mes-
#include <sys/msg.h> sage queue persists even after the sender
#include <stdlib.h> process creates the queue, writes to it, and
#include "queue.h" exits. The queue goes away only after the re-
ceiver process explicitly removes it with the
void report_and_exit(const char* msg) {
call to msgctl:
perror(msg);
exit(-1); /* EXIT_FAILURE */
} if (msgctl(qid, IPC_RMID, NULL) < 0) /* remove
queue */
int main() {
key_t key= ftok(PathName, ProjectId); /* key to identify the queue */ Wrapping up
if (key < 0) report_and_exit("key not gotten...");
The pipes and message queue APIs are fun-
damentally unidirectional: one process writes
int qid = msgget(key, 0666 | IPC_CREAT); /* access if created already */
if (qid < 0) report_and_exit("no access to queue...");
and another reads. There are implementa-
tions of bidirectional named pipes, but my
int types[] = {3, 1, 2, 1, 3, 2}; /* different than in sender */ two cents is that this IPC mechanism is at
int i; its best when it is simplest. As noted earlier,
for (i = 0; i < MsgCount; i++) { message queues have fallen in popularity—
queuedMessage msg; /* defined in queue.h */ but without good reason; these queues are
if (msgrcv(qid, &msg, sizeof(msg), types[i], MSG_NOERROR | IPC_NOWAIT) < 0)
yet another tool in the IPC toolbox. Part 3
puts("msgrcv trouble...");
completes this quick tour of the IPC toolbox
printf("%s received as type %i\n", msg.payload, (int) msg.type);
} with code examples of IPC through sockets
and signals.
/** remove the queue **/
if (msgctl(qid, IPC_RMID, NULL) < 0) /* NULL = 'no flags' */ Links
report_and_exit("trouble removing queue..."); [1] http://man7.org/linux/man-pages/man2/
mq_open.2.html
return 0;
[2] http://man7.org/linux/man-pages/man2/
}
mq_open.2.html#ATTRIBUTES

18 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sockets and signals

Sockets and signals

Learn how processes synchronize with each other in Linux.

The first section focused on IPC through shared

storage (files and memory segments),
and the second section does the same for basic channels:
can flow through the channel until this is closed on either
end, thereby breaking the connection.
An iterative server, which is suited for development only,
pipes (named and unnamed) and message queues. This handles connected clients one at a time to completion: the
section moves from IPC at the high end (sockets) to IPC at first client is handled from start to finish, then the second, and
the low end (signals). Code examples flesh out the details. so on. The downside is that the handling of a particular client
may hang, which then starves all the clients waiting behind.
Sockets A production-grade server would be concurrent, typically
Just as pipes come in two fla- using some mix of multi-pro-
vors (named and unnamed), cessing and multi-thread-
so do sockets. IPC sockets ing. For example, the Nginx
(aka Unix domain sockets) web server on my desktop
enable channel-based com- machine has a pool of four
munication for processes worker processes that can
on the same physical device handle client requests con-
(host ), whereas network currently. The following code
sockets enable this kind of example keeps the clutter to
IPC for processes that can a minimum by using an itera-
run on different hosts, there- tive server; the focus thus re-
by bringing networking into mains on the basic API, not
play. Network sockets need on concurrency.
support from an underlying protocol such as TCP (Transmis- Finally, the socket API has evolved significantly over time
sion Control Protocol) or the lower-level UDP (User Data- as various POSIX refinements have emerged. The current
gram Protocol). sample code for server and client is deliberately simple but
By contrast, IPC sockets rely upon the local system kernel underscores the bidirectional aspect of a stream-based
to support communication; in particular, IPC sockets commu- socket connection. Here’s a summary of the flow of control,
nicate using a local file as a socket address. Despite these with the server started in a terminal then the client started in
implementation differences, the IPC socket and network a separate terminal:
socket APIs are the same in the essentials. The forthcoming • The server awaits client connections and, given a success-
example covers network sockets, but the sample server and ful connection, reads the bytes from the client.
client programs can run on the same machine because the • To underscore the two-way conversation, the server
server uses network address localhost (127.0.0.1), the ad- echoes back to the client the bytes received from the cli-
dress for the local machine on the local machine. ent. These bytes are ASCII character codes, which make
Sockets configured as streams (discussed below) are bi- up book titles.
directional, and control follows a client/server pattern: the cli- • The client writes book titles to the server process and then
ent initiates the conversation by trying to connect to a server, reads the same titles echoed from the server. Both the
which tries to accept the connection. If everything works, server and the client print the titles to the screen. Here is
requests from the client and responses from the server then the server’s output, essentially the same as the client’s:

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 19

Sockets and signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Example 1. The socket server Listening on port 9876 for clients...

#include <string.h> War and Peace

#include <stdio.h> Pride and Prejudice
#include <stdlib.h> The Sound and the Fury
#include <unistd.h>
#include <sys/types.h>
See Example 1.
#include <sys/socket.h>
#include <netinet/tcp.h>
The server program above performs the
#include <arpa/inet.h> classic four-step to ready itself for client re-
#include "sock.h" quests and then to accept individual requests.
Each step is named after a system function
void report(const char* msg, int terminate) {
that the server calls:
perror(msg);
if (terminate) exit(-1); /* failure */
} 1. socket(…): get a file descriptor for the sock-
et connection
int main() { 2. bind(…): bind the socket to an address on
int fd = socket(AF_INET, /* network versus AF_LOCAL */
the server’s host
SOCK_STREAM, /* reliable, bidirectional, arbitrary payload size */
3. listen(…): listen for client requests
0); /* system picks underlying protocol (TCP) */
if (fd < 0) report("socket", 1); /* terminate */ 4. accept(…): accept a particular client request

/* bind the server's local address in memory */ The socket call in full is:
struct sockaddr_in saddr;
memset(&saddr, 0, sizeof(saddr)); /* clear the bytes */
int sockfd = socket(AF_INET, /* versus AF_LOCAL */
saddr.sin_family = AF_INET; /* versus AF_LOCAL */
saddr.sin_addr.s_addr = htonl(INADDR_ANY); /* host-to-network endian */ SOCK_STREAM, /* 
reliable,
saddr.sin_port = htons(PortNumber); /* for listening */ bidirectional */
0); /* 
system picks
if (bind(fd, (struct sockaddr *) &saddr, sizeof(saddr)) < 0) protocol (TCP) */
report("bind", 1); /* terminate */

/* listen to the socket */

The first argument specifies a network sock-
if (listen(fd, MaxConnects) < 0) /* listen for clients, up to MaxConnects */ et as opposed to an IPC socket. There are
report("listen", 1); /* terminate */ several options for the second argument, but
SOCK_STREAM and SOCK_DGRAM (da-
fprintf(stderr, "Listening on port %i for clients...\n", PortNumber);
tagram) are likely the most used. A stream-
/* a server traditionally listens indefinitely */
while (1) {
based socket supports a reliable channel in
struct sockaddr_in caddr; /* client address */ which lost or altered messages are reported;
int len = sizeof(caddr); /* address length could change */ the channel is bidirectional, and the payloads
from one side to the other can be arbitrary
int client_fd = accept(fd, (struct sockaddr*) &caddr, &len); /* accept blocks */
in size. By contrast, a datagram-based sock-
if (client_fd < 0) {
et is unreliable (best try), unidirectional, and
report("accept", 0); /* don't terminate, though there's a problem */
continue; requires fixed-sized payloads. The third ar-
} gument to socket specifies the protocol. For
the stream-based socket in play here, there
/* read from client */ is a single choice, which the zero represents:
int i;
TCP. Because a successful call to socket re-
for (i = 0; i < ConversationLen; i++) {
char buffer[BuffSize + 1]; turns the familiar file descriptor, a socket is
memset(buffer, '\0', sizeof(buffer)); written and read with the same syntax as, for
int count = read(client_fd, buffer, sizeof(buffer)); example, a local file.
if (count > 0) { The bind call is the most complicated, as it
puts(buffer);
reflects various refinements in the socket API.
write(client_fd, buffer, sizeof(buffer)); /* echo as confirmation */
}
The point of interest is that this call binds the
} socket to a memory address on the server
close(client_fd); /* break connection */ machine. However, the listen call is straight-
} /* while(1) */ forward:
return 0;
}
if (listen(fd, MaxConnects) < 0)

20 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sockets and signals

The first argument is the socket’s file descriptor and Example 2. The socket client
the second specifies how many client connections #include <string.h>
can be accommodated before the server issues a #include <stdio.h>
connection refused error on an attempted connec- #include <stdlib.h>
tion. (MaxConnects is set to 8 in the header file #include <unistd.h>
#include <sys/types.h>
sock.h.)
#include <sys/socket.h>
The accept call defaults to a blocking wait: the #include <arpa/inet.h>
server does nothing until a client attempts to connect #include <netinet/in.h>
and then proceeds. The accept function returns -1 to #include <netinet/tcp.h>
indicate an error. If the call succeeds, it returns anoth- #include <netdb.h>
er file descriptor—for a read/write socket in contrast #include "sock.h"

to the accepting socket referenced by the first argu-

const char* books[] = {"War and Peace",
ment in the accept call. The server uses the read/ "Pride and Prejudice",
write socket to read requests from the client and to "The Sound and the Fury"};
write responses back. The accepting socket is used
only to accept client connections. void report(const char* msg, int terminate) {
By design, a server runs indefinitely. Accordingly, perror(msg);
if (terminate) exit(-1); /* failure */
the server can be terminated with a Ctrl+C from the
}
command line (see Example 2).
The client program’s setup code is similar to the int main() {
server’s. The principal difference between the two is /* fd for the socket */
that the client neither listens nor accepts, but instead int sockfd = socket(AF_INET, /* versus AF_LOCAL */
connects: SOCK_STREAM, /* reliable, bidirectional */
0); /* system picks protocol (TCP) */
if (sockfd < 0) report("socket", 1); /* terminate */
if (
connect(sockfd, (struct sockaddr*) &saddr,
sizeof(saddr)) < 0) /* get the address of the host */
struct hostent* hptr = gethostbyname(Host); /* localhost: 127.0.0.1 */
The connect call might fail for several reasons; for if (!hptr) report("gethostbyname", 1); /* is hptr NULL? */
if (hptr->h_addrtype != AF_INET) /* versus AF_LOCAL */
example, the client has the wrong server address or
report("bad address family", 1);
too many clients are already connected to the server.
If the connect operation succeeds, the client writes /* connect to the server: configure server's address 1st */
requests and then reads the echoed responses in a struct sockaddr_in saddr;
for loop. After the conversation, both the server and memset(&saddr, 0, sizeof(saddr));
the client close the read/write socket, although a saddr.sin_family = AF_INET;
saddr.sin_addr.s_addr =
close operation on either side is sufficient to close the
((struct in_addr*) hptr->h_addr_list[0])->s_addr;
connection. The client exits thereafter but, as noted saddr.sin_port = htons(PortNumber); /* port number in big-endian */
earlier, the server remains open for business.
The socket example, with request messages if (connect(sockfd, (struct sockaddr*) &saddr, sizeof(saddr)) < 0)
echoed back to the client, hints at the possibilities report("connect", 1);
of arbitrarily rich conversations between the server
/* Write some stuff and read the echoes. */
and the client. Perhaps this is the chief appeal of
puts("Connect to server, about to write some stuff...");
sockets. It is common on modern systems for client int i;
applications (e.g., a database client) to communi- for (i = 0; i < ConversationLen; i++) {
cate with a server through a socket. As noted ear- if (write(sockfd, books[i], strlen(books[i])) > 0) {
lier, local IPC sockets and network sockets differ /* get confirmation echoed from server and print */
only in a few implementation details; in general, char buffer[BuffSize + 1];
memset(buffer, '\0', sizeof(buffer));
IPC sockets have lower overhead and better per-
if (read(sockfd, buffer, sizeof(buffer)) > 0)
formance. The communication API is essentially the puts(buffer);
same for both. }
}
Signals puts("Client done, about to exit...");
close(sockfd); /* close the connection */
A signal interrupts an executing program and, in this
return 0;
sense, communicates with it. Most signals can be
}
either ignored (blocked) or handled (through desig-

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 21

Sockets and signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

nated code), with SIGSTOP (pause) and SIGKILL (termi- Signals can arise in user interaction. For example, a
nate immediately) as the two notable exceptions. Symbolic user hits Ctrl+C from the command line to terminate a pro-
constants such as SIGKILL have integer values, in this gram started from the command-line; Ctrl+C generates a
case, 9. SIGTERM signal. SIGTERM for terminate, unlike SIGKILL,
can be either blocked or handled. One pro-
Example 3. The graceful shutdown of a multi-processing system cess also can signal another, thereby mak-
#include <stdio.h> ing signals an IPC mechanism.
#include <signal.h> Consider how a multi-processing applica-
#include <stdlib.h> tion such as the Nginx web server might be
#include <unistd.h> shut down gracefully from another process.
#include <sys/wait.h>
The kill function:
void graceful(int signum) {
printf("\tChild confirming received signal: %i\n", signum); in
t kill(pid_t pid, int signum);
puts("\tChild about to terminate gracefully..."); /* declaration */
sleep(1);
puts("\tChild terminating now...");
can be used by one process to terminate
_exit(0); /* fast-track notification of parent */
}
another process or group of processes. If
the first argument to function kill is greater
void set_handler() { than zero, this argument is treated as the pid
struct sigaction current; (process ID) of the targeted process; if the
sigemptyset(&current.sa_mask); /* clear the signal set */
argument is zero, the argument identifies the
current.sa_flags = 0; /* enables setting sa_handler, not sa_action */
group of processes to which the signal send-
current.sa_handler = graceful; /* specify a handler */
sigaction(SIGTERM, &current, NULL); /* register the handler */ er belongs.
} The second argument to kill is either a
standard signal number (e.g., SIGTERM
void child_code() { or SIGKILL) or 0, which makes the call to
set_handler();
signal a query about whether the pid in the
while (1) { /** loop until interrupted **/ first argument is indeed valid. The graceful
sleep(1); shutdown of a multi-processing application
puts("\tChild just woke up, but going back to sleep."); thus could be accomplished by sending a
} terminate signal—a call to the kill function
}
with SIGTERM as the second argument—to
void parent_code(pid_t cpid) {
the group of processes that make up the ap-
puts("Parent sleeping for a time..."); plication. (The Nginx master process could
sleep(5); terminate the worker processes with a call to
kill and then exit itself.) The kill function, like
/* Try to terminate child. */
so many library functions, houses power and
if (-1 == kill(cpid, SIGTERM)) {
perror("kill");
flexibility in a simple invocation syntax (see
exit(-1); Example 3).
} The shutdown program above simulates
wait(NULL); /** wait for child to terminate **/ the graceful shutdown of a multi-processing
puts("My child terminated, about to exit myself...");
system, in this case, a simple one consisting
}
of a parent process and a single child pro-
int main() { cess. The simulation works as follows:
pid_t pid = fork(); • The parent process tries to fork a child. If
if (pid < 0) { the fork succeeds, each process executes
perror("fork"); its own code: the child executes the func-
return -1; /* error */
tion child_code, and the parent executes
}
if (0 == pid) the function parent_code.
child_code(); • The child process goes into a potential-
else ly infinite loop in which the child sleeps
parent_code(pid); for a second, prints a message, goes
return 0; /* normal */
back to sleep, and so on. It is precisely
}
a SIGTERM signal from the parent that

22 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Sockets and signals

causes the child to execute the signal-handling callback puts("Parent sleeping for a time...");
function graceful. The signal thus breaks the child pro- sleep(5);
cess out of its loop and sets up the graceful termination of if (-1 == kill(cpid, SIGTERM)) {
both the child and the parent. The child prints a message ...
before terminating.
• The parent process, after forking the child, sleeps for If the kill call succeeds, the parent does a wait on the
five seconds so that the child can execute for a while; child’s termination to prevent the child from becoming a
of course, the child mostly sleeps in this simulation. The permanent zombie; after the wait, the parent exits.
parent then calls the kill function with SIGTERM as the • The child_code function first calls set_handler and then
second argument, waits for the child to terminate, and goes into its potentially infinite sleeping loop. Here is the
then exits. set_handler function for review:
Here is the output from a sample run:
void set_handler() {
% ./shutdown struct sigaction current; /* current setup */
Parent sleeping for a time... sigemptyset(&current.sa_mask); /* clear the signal set */
Child just woke up, but going back to sleep. current.sa_flags = 0; /* 
for setting sa_handler,
Child just woke up, but going back to sleep. not sa_action */
Child just woke up, but going back to sleep. current.sa_handler = graceful; /* specify a handler */
Child just woke up, but going back to sleep. sigaction(SIGTERM, &current, NULL); /* register the handler */
Child confirming received signal: 15 ## SIGTERM is 15 }
Child about to terminate gracefully...
Child terminating now... The first three lines are preparation. The fourth statement sets
My child terminated, about to exit myself... the handler to the function graceful, which prints some mes-
sages before calling _exit to terminate. The fifth and last state-
For the signal handling, the example uses the sigaction ment then registers the handler with the system through the
library function (POSIX recommended) rather than the call to sigaction. The first argument to sigaction is SIGTERM
legacy signal function, which has portability issues. Here for terminate, the second is the current sigaction setup, and
are the code segments of chief interest: the last argument (NULL in this case) can be used to save a
• If the call to fork succeeds, the parent executes the previous sigaction setup, perhaps for later use.
parent_code function and the child executes the child_ Using signals for IPC is indeed a minimalist approach, but
code function. The parent waits for five seconds before a tried-and-true one at that. IPC through signals clearly be-
signaling the child: longs in the IPC toolbox.

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 23

Wrapping up this guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Wrapping up this guide

This Guide on IPC
anisms through code examples:
has covered the
following mech-
remains a viable—indeed, quite appealing—way to take
advantage of today’s multi-processor machines without
the inherent risk of memory-based race conditions.
• Shared files There is no simple answer, of course, to the question of
• Shared memory (with semaphores) which among the IPC mechanisms is the best. Each involves
• Pipes (named and unnamed) a trade-off typical in programming: simplicity versus function-
• Message queues ality. Signals, for example, are a relatively simple IPC mecha-
• Sockets nism but do not support rich conversations among processes.
• Signals If such a conversion is needed, then one of the other choices
Even today, when thread-centric languages such as Java, is more appropriate. Shared files with locking is reasonably
C#, and Go have become so popular, IPC remains appeal- straightforward, but shared files may not perform well enough
ing because concurrency through multi-processing has an if processes need to share massive data streams; pipes or
obvious advantage over multi-threading: every process, even sockets, with more complicated APIs, might be a better
by default, has its own address space, which rules out choice. Let the problem at hand guide the choice.
memory-based race conditions in multi-processing unless Although the sample code (available on my website [1])
the IPC mechanism of shared memory is brought into play. is all in C, other programming languages often provide thin
(Shared memory must be locked in both multi-processing wrappers around these IPC mechanisms. The code exam-
and multi-threading for safe concurrency.) Anyone who ples are short and simple enough, I hope, to encourage you
has written even an elementary multi-threading program to experiment.
with communication via shared variables knows how chal-
lenging it can be to write thread-safe yet clear, efficient Links
code. Multi-processing with single-threaded processes [1] https://condor.depaul.edu/mkalin/

24 A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com

 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Additional Resources

Write for Us

In 2010, Red Hat CEO Jim Whitehurst announced the launch of Opensource.com
in a post titled Welcome to the conversation on Opensource.com. He explained,
“This site is one of the ways in which Red Hat gives something back to the open
source community. Our desire is to create a connection point for conversations
about the broader impact that open source can have—and is having—even beyond
the software world.” he wrote, adding, “All ideas are welcome, and all participants
are welcome. This will not be a site for Red Hat, about Red Hat. Instead, this will be
a site for open source, about the future.”

By 2013, Opensource.com was publishing an average of 46 articles per month,

and in March 2016, Opensource.com surpassed 1-million page views for the first
time. In 2019, Opensource.com averages more than 1.5 million page views and
90 articles per month.

More than 60% of our content is contributed by members of open source communities,
and additional articles are written by the editorial team and other Red Hat contributors.
A small, international team of staff editors and Community Moderators work closely
with contributors to curate, polish, publish, and promote open source stories from
around the world.

Would you like to write for us? Send pitches and inquiries to [email protected].

To learn more, read 7 big reasons to contribute to Opensource.com.

A guide to inter-process communication in Linux . CC BY-SA 4.0 . Opensource.com 25