Pipes — Beej's Guide to Unix IPC, Chapter 4

Chapter 0: Why Pipes?

You've used fork() to create two processes. They can't share memory. How do you get data from one to the other? The simplest answer: a pipe.

A pipe is a one-way data channel. One end writes, the other reads. Data flows in order — first in, first out. The shell uses pipes every time you type something like ls | grep foo | wc -l. Each | creates a pipe connecting stdout of the left command to stdin of the right.

The mental model: Imagine a physical pipe connecting two rooms. You pour water in one end, it comes out the other. You can't push it backwards. You can't look at what's inside without draining it. It's a one-way stream of bytes.

Pipe Data Flow

Click "Send" to push data through the pipe from writer to reader. Watch bytes flow through the buffer.

Buffer: empty

In what direction does data flow through a pipe?

Both directions simultaneously One direction only: writer to reader Either direction, depending on configuration

Chapter 1: File Descriptors

Before diving into pipes, you need to understand file descriptors. They're just integers that refer to open I/O channels. Every process starts with three:

FD	Name	Purpose
0	stdin	Standard input (keyboard)
1	stdout	Standard output (terminal)
2	stderr	Standard error (terminal)

When you call open(), you get a new file descriptor (usually 3, 4, 5, ...). When you call pipe(), you get a pair of file descriptors — one for reading, one for writing. The pipe() call is the low-level equivalent of the FILE* streams from stdio.

The connection: fopen() returns a FILE*. Under the hood, it uses a file descriptor. fileno(fp) gives you the integer fd. Pipes work at the fd level — with read() and write(), not fread() and fwrite().

What is file descriptor 1?

Standard input Standard output Standard error

Chapter 2: pipe() — Creating a Pipe

The pipe() system call creates a pipe and gives you two file descriptors:

c
int pfds[2];
pipe(pfds);
/* pfds[0] = read end  */
/* pfds[1] = write end */

Now you can write to pfds[1] and read from pfds[0]:

c
write(pfds[1], "hello", 5);  /* put bytes in */

char buf[10];
read(pfds[0], buf, 5);     /* take bytes out */
/* buf now contains "hello" */

Key insight: pfds[0] is the read end (index 0 = input), pfds[1] is the write end (index 1 = output). Think of it as: data goes in at [1] and comes out at [0]. On most systems, the pipe buffer holds about 64KB before blocking.

This example is trivial — writing to yourself through a pipe is pointless. The real power comes when you combine pipe() with fork().

After calling pipe(pfds), which descriptor do you write to?

pfds[0] pfds[1] Either one

Chapter 3: fork() + pipe() — The Power Combo

Here's where it gets interesting. You create the pipe before forking. The child inherits copies of both file descriptors. Now the child can write to pfds[1] and the parent can read from pfds[0] — data flows between processes!

c
int pfds[2];
pipe(pfds);

if (!fork()) {
    /* CHILD: write to pipe */
    close(pfds[0]);           /* close unused read end */
    write(pfds[1], "test", 5);
    exit(0);
} else {
    /* PARENT: read from pipe */
    char buf[30];
    close(pfds[1]);           /* close unused write end */
    read(pfds[0], buf, 5);
    printf("Got: %s\n", buf); /* prints "test" */
    wait(NULL);
}

Critical rule: Always close the unused end. If the parent doesn't close pfds[1], the read() will never get EOF — it'll block forever, waiting for data from a write end that's technically still open. Close what you don't use.

Child

write(pfds[1], data, len)

↓ bytes flow through pipe

Parent

read(pfds[0], buf, len)

Why must the parent close pfds[1] (the write end) when it only wants to read?

To save memory It's optional but good practice Otherwise read() will never see EOF and will block forever

Chapter 4: dup() — Redirecting I/O

What if you want a child's stdout to go through a pipe instead of to the terminal? You need dup() (or dup2()). It clones a file descriptor into a new number.

The trick: close fd 1 (stdout), then call dup(pfds[1]). Since dup() always uses the lowest available fd, and you just freed fd 1, the pipe's write end becomes the new stdout. Now anything the child printf()s goes into the pipe!

c
/* In the child: redirect stdout to pipe */
close(1);          /* free up fd 1 (stdout) */
dup(pfds[1]);       /* clone pipe write end into fd 1 */
close(pfds[0]);     /* close unused read end */

/* Now stdout IS the pipe */
printf("This goes into the pipe!\n");

dup2() is cleaner: Instead of close+dup, you can use dup2(pfds[1], 1) which atomically copies pfds[1] into fd 1, closing the old fd 1 if needed. Same result, one call.

What does close(1); dup(pfds[1]); accomplish?

Makes stdout point to the pipe's write end Closes the pipe Creates a new pipe

Chapter 5: Implementing ls | wc -l

Now we can build the shell's pipe operator in C. The plan: fork two children. Child 1 runs ls with its stdout redirected to the pipe. Child 2 runs wc -l with its stdin redirected from the pipe.

c
int pfds[2];
pipe(pfds);

if (!fork()) {
    /* Child 1: ls */
    close(1);            /* close stdout */
    dup(pfds[1]);         /* pipe write end -> stdout */
    close(pfds[0]);
    execlp("ls", "ls", NULL);
} else {
    /* Child 2: wc -l */
    close(0);            /* close stdin */
    dup(pfds[0]);         /* pipe read end -> stdin */
    close(pfds[1]);
    execlp("wc", "wc", "-l", NULL);
}

exec() replaces: execlp() replaces the current process with a new program. After execlp("ls", ...), the child IS ls. It inherits all open file descriptors — including our redirected stdout. This is how the shell implements pipelines.

ls (Child 1)

stdout → pipe write end

↓ bytes flow through pipe

wc -l (Child 2)

stdin ← pipe read end

How does the shell implement the "|" pipe operator?

pipe() + fork() + dup() to redirect stdout/stdin + exec() Shared memory between the two commands A temporary file on disk

Chapter 6: Blocking Behavior

Pipes have a built-in buffer (typically 64KB on Linux). Two things can cause blocking:

read() blocks when the pipe is empty but writers still exist. It waits for data. Returns 0 (EOF) only when all write ends are closed.

write() blocks when the pipe buffer is full. It waits until the reader drains some data. If no readers exist, write() triggers SIGPIPE.

This blocking behavior is what makes pipes work as a synchronization mechanism too. The reader naturally waits for the writer, and the writer naturally waits if the reader is slow. It's like a conveyor belt that pauses when the other end isn't keeping up.

What happens when you read() from a pipe that has data in its buffer?

read() returns immediately with the available data read() blocks until the buffer is completely full read() fails with an error

Chapter 7: Pipe Pipeline Simulator

The showcase simulation. Build a multi-stage pipeline like the shell does. Watch data flow through each pipe, see buffers fill and drain, and observe blocking when a stage falls behind.

Shell Pipeline Simulator

Data (shown as colored blocks) flows from left to right through pipes. Click "Produce" to generate data and watch it flow through the pipeline. Adjust processing speed with the slider.

Speed 5

What happens when a pipe's buffer is full and the writer tries to write more?

Data is silently dropped The write() call blocks until the reader consumes some data The pipe automatically expands

Chapter 8: Beyond Pipes

Pipes are elegant but limited. They only work between related processes (parent-child via fork). You can't connect two unrelated programs with a pipe() call. For that, you need FIFOs (named pipes) — the topic of the next chapter.

Feature	Pipes	FIFOs
Direction	Unidirectional	Unidirectional
Relationship	Parent-child only	Any processes
Persistence	In-memory only	File on disk
Creation	pipe()	mknod() or mkfifo
Typical use	Shell pipelines	IPC between daemons

Coming up: FIFOs (named pipes), message queues for typed messages, shared memory for raw speed, and Unix sockets for full-duplex communication.

"There is no form of IPC that is simpler than pipes." — Beej

What is the main limitation of anonymous pipes?

They can only transfer strings They only work between related (parent-child) processes They are too slow for practical use

Unix Pipes