Inter-Process Communication — Pipes, Signals and Shared Memory

How processes talk: pipes, signals, message queues, shared memory and sockets, with working examples and their trade-offs.

Est. 35 min Updated Suggest a fix
// Part 01 — Why processes communicate

Separate Processes Still Need to Cooperate

A process has its own virtual address space. That isolation is useful: one program cannot casually overwrite another program's variables. But real systems are made of cooperating processes. A shell connects commands, a browser separates tabs and services, and a delivery platform moves an order from its API process to a worker. They need a deliberate communication channel called inter-process communication (IPC).

Pick a mechanism by asking what data must move, whether the sender and receiver run on one machine, how much data there is, and what should happen when either side is slow or exits. A signal is a small notification, not a way to transfer a document. Shared memory can move large amounts quickly, but requires synchronisation. A socket can connect processes on the same host or across a network, but has a protocol and failure handling to design.

MechanismData shapeTypical useMain responsibility
Anonymous pipeOrdered byte streamParent and child; shell pipelineAgree on framing and close unused ends
Named pipe (FIFO)Ordered byte streamUnrelated local processesCoordinate creation, readers, and writers
SignalSmall event plus optional valueNotify stop, reload, or child exitHandle interruption and safe cleanup
Message queueDiscrete messagesLocal worker coordinationChoose limits, ordering, and delivery policy
Shared memoryCommon memory regionHigh-volume local dataProtect concurrent access and lifecycle
SocketStream or datagramsLocal services or network clientsDefine protocol, timeouts, and authentication
💡 Note
A pipe and a TCP connection both look like byte streams to application code. Neither preserves your message boundaries: if one side writes two records, the receiver still needs a delimiter or a length prefix to tell where each record ends.
// Part 02 — Anonymous pipes

Connect a Producer to a Consumer

An anonymous pipe is a kernel-managed, one-way byte stream. A process creates it before starting a child, then the child inherits the open file descriptors. The parent can connect one command's output to another command's input, which is exactly what a shell does for producer | consumer. A pipe has finite buffering: if the consumer is slower, the producer eventually blocks instead of consuming unlimited memory.

pipeline.sh
bash
# Count successful payment records without saving the whole file in memory.
grep ',SUCCESS,' payments.csv | wc -l

# The pipe carries grep's output to wc; each program has one focused job.

This small C program shows the system-call shape. The parent writes a short message and the child reads until end of file. Compile on a POSIX system with cc pipe_demo.c -o pipe_demo, then run ./pipe_demo. Closing the unused ends matters: the reader sees end-of-file only when every write end has been closed.

pipe_demo.c
c
#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>

int main(void) {
    int channel[2];
    if (pipe(channel) == -1) {
        perror("pipe");
        return 1;
    }

    pid_t child = fork();
    if (child == -1) {
        perror("fork");
        close(channel[0]);
        close(channel[1]);
        return 1;
    }

    if (child == 0) {
        close(channel[1]);
        char buffer[128];
        ssize_t count;
        while ((count = read(channel[0], buffer, sizeof buffer)) > 0) {
            if (write(STDOUT_FILENO, buffer, (size_t)count) == -1) {
                perror("write to stdout");
                close(channel[0]);
                return 1;
            }
        }
        close(channel[0]);
        return count == -1 ? 1 : 0;
    }

    close(channel[0]);
    const char message[] = "Order accepted by the parent process.\n";
    if (write(channel[1], message, sizeof message - 1) == -1) {
        perror("write to pipe");
        close(channel[1]);
        waitpid(child, NULL, 0);
        return 1;
    }
    close(channel[1]);
    if (waitpid(child, NULL, 0) == -1) {
        perror("waitpid");
        return 1;
    }
    return 0;
}
🎯 Pro Tip
Always check read and write return values. Reads may return fewer bytes than requested, and writes can fail; robust stream code loops until it has sent or received a complete application-level record.
// Part 03 — Named pipes

Use a FIFO When Processes Do Not Share a Parent

A FIFO is a named pipe represented in the filesystem. Two unrelated local processes can open the same path and exchange bytes without a network service. The name is a rendezvous point, not a file that stores a durable history. A writer may wait until a reader opens the FIFO, so production code needs a clear startup and shutdown arrangement.

fifo_demo.sh
bash
# Terminal 1: create a FIFO once, then wait for a producer.
mkfifo order-events.fifo
cat order-events.fifo

# Terminal 2: write one line. The reader receives it, then exits.
printf '%s\n' 'order=734 status=packed' > order-events.fifo

# Remove the rendezvous point when finished.
rm order-events.fifo

FIFO permissions are ordinary filesystem permissions, but they do not define a rich message format. If the receiver must survive restarts, track acknowledgements, or process messages from many producers, use a queue or a service backed by durable storage rather than treating a FIFO as a database.

// Part 04 — Signals and queues

Notifications Are Not Payloads

A signal tells a process that an event occurred: a terminal requested interruption, a child exited, or an operator asked a service to reload configuration. Standard signals are notifications, not a reliable event log; repeated occurrences may be coalesced. Keep a handler short and defer complex work to the main loop. Do not assume a handler can safely call every library function.

signal_note.c
c
#include <signal.h>
#include <stdio.h>
#include <unistd.h>

static volatile sig_atomic_t reload_requested = 0;

static void on_hangup(int signal_number) {
    (void)signal_number;
    reload_requested = 1;
}

int main(void) {
    struct sigaction action = {0};
    action.sa_handler = on_hangup;
    sigemptyset(&action.sa_mask);
    if (sigaction(SIGHUP, &action, NULL) == -1) {
        perror("sigaction");
        return 1;
    }

    while (1) {
        if (reload_requested) {
            reload_requested = 0;
            puts("Reload configuration in the normal control flow.");
        }
        sleep(1);
    }
}

Message queues preserve discrete messages and can carry structured payloads. They are a better conceptual fit for "order 734 is ready" than a signal. Decide whether messages may be lost, duplicated, reordered, or delayed, and make consumers idempotent when retries are possible. Queue capacity is finite too; a producer needs backpressure or a failure policy when the queue is full.

⚠️ Important
A signal handler should not print logs, allocate memory, or take an ordinary mutex. Set a safe flag or use a platform event mechanism, then do the actual work outside the handler.
// Part 05 — Shared memory

Fast Data Sharing Needs Explicit Coordination

Shared memory maps the same physical pages into multiple processes. After setup, exchanging a large local data set can avoid repeatedly copying it through a kernel-mediated message channel. But shared bytes do not make a safe protocol: two writers can race, one process can read a half-updated record, and a crashed writer can leave a lock held. Pair the memory region with a process-shared mutex, semaphore, or a carefully designed atomic protocol.

  1. Create or obtain a shared region and agree on its size and layout.
  2. Define who owns each field and how readers know a record is complete.
  3. Use process-shared synchronisation; a normal process-private mutex is not enough.
  4. Define startup, versioning, crash recovery, and who removes the region.
  5. Measure before choosing shared memory: its coordination cost can exceed the copy you hoped to avoid.

For a modest telemetry payload, start with a socket or queue. Choose shared memory when profiling shows that copying is a real bottleneck and you can explain the synchronisation invariants on a whiteboard and in tests.

// Part 06 — Sockets

Use a Local Socket or a Network Protocol

A Unix domain socket connects processes on one host and can use filesystem permissions as one part of access control. TCP sockets can cross machine boundaries and support the same client-server structure. In both cases, the application must define framing, error handling, and authentication as appropriate. A successful connection only means the transport connected; it does not mean a business operation succeeded.

local_socket_pair.py
python
import socket

# socketpair gives two connected local stream endpoints for a small demo.
left, right = socket.socketpair()
try:
    left.sendall(b"order=734\n")
    received = right.recv(128)
    print(received.decode("utf-8").strip())
finally:
    left.close()
    right.close()

A stream socket may deliver part of a message in one read or several messages together in one read. This demo sends a small line and reads once; a real protocol should use newline-delimited records with limits, or a length prefix and a loop that reads the full record. Set timeouts and bound input before parsing it.

// Part 07 — Production choices

Make the Contract Clear at Both Ends

QuestionWhy it mattersA practical design choice
Who owns the channel?Orphaned ends can prevent EOF or keep resources alive.Document which process creates, closes, and removes it.
What is one message?Byte streams do not preserve application records.Specify delimiter or length prefix and maximum size.
What if the receiver is slow?Kernel buffers and queues have limits.Apply backpressure, bounded queues, or a clear reject policy.
Can delivery repeat?Retries after uncertain outcomes may duplicate work.Give operations an idempotency key and track completion.
Can data leave the host?Network IPC crosses a trust boundary.Authenticate peers, validate input, and protect sensitive data.

For a UPI reconciliation service, the API process can publish a compact event while a worker validates and stores it. Include a stable transaction reference, schema version, and timestamp. The worker should tolerate retrying the same event, report failures, and avoid logging full payment credentials. These are application-level responsibilities; picking a pipe or socket does not solve them automatically.

// Part 08 — Pitfalls

Common IPC Mistakes

Assuming one read returns one whole message

Why: Pipes and stream sockets transfer bytes, not records.

Fix: Add framing and loop until a complete bounded message arrives.

Leaving an unused pipe end open

Why: The reader never observes EOF while a writer descriptor remains open.

Fix: Close every descriptor not used by that process, including inherited copies.

Using a signal to carry an event history

Why: Signals can be coalesced and carry little payload.

Fix: Use a queue or durable event store when every event must be processed.

Choosing shared memory only because it is fast

Why: Unsynchronised shared state introduces races and recovery problems.

Fix: Benchmark first, then specify locking, ownership, and crash recovery.

// Part 09 — Practice

Choose and Test an IPC Channel

Practice

Design IPC for a campus order-processing service

  1. Run the pipe example and explain why closing the parent write end lets the child finish.
  2. Use a shell pipeline to count failed delivery records without loading the complete file into memory.
  3. Compare a FIFO, shared memory, and a Unix domain socket for two local services sending small status updates.
  4. Define a newline-delimited message format with a maximum record size and an idempotency key.
  5. Simulate a slow consumer: decide whether the producer blocks, buffers a bounded number of records, or rejects work.

Hint: State the delivery guarantee and what happens on process restart before choosing a mechanism.

🎯 Key Takeaways

  • ✓IPC deliberately shares data or events between otherwise isolated processes.
  • ✓Pipes, FIFOs, and stream sockets carry bytes; your protocol must define message boundaries.
  • ✓Signals are useful for notifications but are not a durable event queue.
  • ✓Shared memory can reduce copying, but requires explicit synchronisation and lifecycle management.
  • ✓Every IPC design needs a policy for slow consumers, process failure, retries, and resource cleanup.
  • ✓Use local sockets or a network protocol when client-server communication needs a clear boundary.
  • For system-call details, consult the POSIX and Linux manual pages for pipe, sigaction, shm_open, and socket.

Next

What comes next

Now that you know how processes exchange work, learn how the OS decides which runnable process gets the CPU next.

Lesson 04 · CPU Scheduling →

HELP US IMPROVE

How was this lesson?

Your note is shared with the Chaduvuko team for review. Please don’t include personal or sensitive information.

Choose your feedback

Spotted a mistake or something unclear? Suggest a fix on GitHub or email us. Select the text first and it is quoted for you.

Share

Discussion

0

Have a better approach? Found something outdated? Share it — your knowledge helps everyone learning here.

Continue with GitHub
Loading...