Pipes and Redirection in Linux: stdin, stdout, stderr

Most people learn redirection the same way: someone shows them > file, then 2>&1, they copy it into a cron job and it works, and that’s the end of it. Until the day it doesn’t. The log file is empty, or stderr is still spraying across the terminal, or a variable set inside a loop is mysteriously blank after the loop ends, and now you’re staring at 2>&1 >/dev/null trying to remember why the order matters.

The fix for all of that is one idea: every process has a small table of numbered file handles, and redirection is just editing that table before the program starts. Once you can picture the table, every operator on this page becomes obvious rather than memorised.

Everything below was run on Ubuntu 24.04 (bash 5.2) and Red Hat Enterprise Linux 9.7 (bash 5.1). The two differ in exactly one place that matters, and it bites cron jobs, so it gets its own section.

The three streams every process gets

When the kernel starts a program, it hands it three open file descriptors, numbered 0, 1 and 2. Every C library, every language runtime, every shell agrees on what they mean:

• 0, stdin: where the program reads input from
• 1, stdout: where it writes normal output
• 2, stderr: where it writes errors and diagnostics

These aren’t abstract. You can look at them. $$ is the PID of your current shell, and /proc shows you its open descriptors:

$ ls -l /proc/$$/fd
total 0
lrwx------ 1 asif asif 64 Apr  8 17:12 0 -> /dev/pts/3
lrwx------ 1 asif asif 64 Apr  8 17:12 1 -> /dev/pts/3
lrwx------ 1 asif asif 64 Apr  8 17:12 2 -> /dev/pts/3
$ tty
/dev/pts/3

All three point at the same place: your terminal. That’s why typing produces text on screen and why errors and normal output look identical when you’re sitting at a shell. They’re different streams that happen to share a destination.

Now start something with all three redirected and look at its table:

$ sleep 30 > out.txt 2> err.txt < /dev/null &
$ ls -l /proc/$!/fd
total 0
lr-x------ 1 asif asif 64 Apr  8 17:11 0 -> /dev/null
l-wx------ 1 asif asif 64 Apr  8 17:11 1 -> /tmp/pipesdemo/out.txt
l-wx------ 1 asif asif 64 Apr  8 17:11 2 -> /tmp/pipesdemo/err.txt

Same three slots, different destinations. sleep has no idea. It never sees the filenames. It just writes to “1” and reads from “0” like always, and the shell arranged for those numbers to point somewhere else before sleep ran. That’s all redirection is.

Redirecting output: >, >> and 2>

Here’s a command that produces one line on stdout and one on stderr:

$ ls /etc/hostname /nope
ls: cannot access '/nope': No such file or directory
/etc/hostname

Redirect stdout to a file and watch the error stay on screen:

$ ls /etc/hostname /nope > out.txt
ls: cannot access '/nope': No such file or directory
$ cat out.txt
/etc/hostname

That’s the single most common surprise with >. It only touches descriptor 1. The error went to descriptor 2, which is still your terminal. > is shorthand for 1>, and if you want the errors instead, say so:

$ ls /etc/hostname /nope 2> err.txt
/etc/hostname
$ cat err.txt
ls: cannot access '/nope': No such file or directory

Now the listing is on screen and the error is in the file. Two operators, two descriptors, no overlap.

> truncates the file to zero bytes and then writes. >> opens it for append instead:

$ echo one > f.txt; echo two > f.txt; cat f.txt
two
$ echo one > f.txt; echo two >> f.txt; cat f.txt
one
two

Log files want >>. Generated config files want >. Getting that backwards is how you end up with a 40 GB file or a log that only ever contains the last run.

2>&1, and why the order matters

You’ll want both streams in one file more often than not. The operator for that is 2>&1, and it reads as: “make descriptor 2 point wherever descriptor 1 points right now“. The “right now” is the whole story.

$ ls /etc/hostname /nope > both.txt 2>&1
$ cat both.txt
ls: cannot access '/nope': No such file or directory
/etc/hostname

The shell processes redirections left to right. First > both.txt points 1 at the file. Then 2>&1 copies 1’s current target into 2. Both end up at the file.

Flip the order and it silently does something else:

$ ls /etc/hostname /nope 2>&1 > only-stdout.txt
ls: cannot access '/nope': No such file or directory
$ cat only-stdout.txt
/etc/hostname

This time 2>&1 ran first, when descriptor 1 was still the terminal. So 2 got a copy of “the terminal”. Then > moved 1 to the file, but 2 had already been set and doesn’t follow. Error on screen, output in file. Not a syntax error, not a warning, just quietly wrong.

If you want to see the shell doing this rather than take my word for it, strace shows the system calls. The whole mechanism is two dup2 calls:

$ strace -f -e trace=openat,dup2 -o trace.txt bash -c 'ls /nope > both.txt 2>&1'
$ grep -E "both.txt|dup2" trace.txt
4295  openat(AT_FDCWD, "both.txt", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 3
4295  dup2(3, 1)                        = 1
4295  dup2(1, 2)                        = 2

Open the file (it lands on the next free number, 3). Copy 3 onto 1. Copy 1 onto 2. Then run ls. Reverse the last two lines and you’ve got the broken version. That’s genuinely everything there is to 2>&1.

The other form you’ll see everywhere is discarding errors:

find / -name "*.conf" 2>/dev/null
apt list --installed 2>/dev/null | wc -l

/dev/null is a device that accepts writes and throws them away. Pointing stderr at it is how you run find across the whole filesystem without a wall of “Permission denied” burying the results. The find examples article leans on this constantly.

The shortcuts, and the one place they break

Bash has two abbreviations for “both streams”:

$ ls /etc/hostname /nope &> all.txt        # same as > all.txt 2>&1
$ ls /etc/hostname /nope |& sort            # same as 2>&1 | sort
/etc/hostname
ls: cannot access '/nope': No such file or directory

Fine in bash. But here’s the trap. On Ubuntu and Debian, /bin/sh is not bash:

$ ls -l /bin/sh          # Ubuntu 24.04
lrwxrwxrwx 1 root root 4 Aug 15  2023 /bin/sh -> dash
$ ls -l /bin/sh          # RHEL 9.7
lrwxrwxrwx 1 root root 4 Oct  2  2023 /bin/sh -> bash

dash is a small, fast, strictly POSIX shell, and it does not know these operators. Watch what happens to each under sh on Ubuntu:

$ sh -c 'ls /etc/hostname /nope &> dash.txt'; cat dash.txt
---
ls: cannot access '/nope': No such file or directory
/etc/hostname
$ sh -c 'ls /nope |& cat'
sh: 1: Syntax error: "&" unexpected
$ sh -c 'cat <<< hello'
sh: 1: Syntax error: redirection unexpected

The |& and <<< cases at least fail loudly. The &> one is nasty: dash parses cmd &> file as cmd & (run in background) followed by > file (truncate the file). The command runs with its output going to the terminal, the file gets created empty, and nothing errors. Run that same script on RHEL and it works perfectly, because sh is bash there.

Where does this actually bite? Cron runs jobs with /bin/sh unless you set SHELL=/bin/bash in the crontab. So does system() from most languages, and so does any script with #!/bin/sh at the top. A cron line like backup.sh &> /var/log/backup.log works on your RHEL test box and silently logs nothing on the Ubuntu production box. Use the long form, > file 2>&1, and it works in every shell ever made.

Redirecting input: <, here-docs and here-strings

Input redirection is the same table edit in the other direction. < file points descriptor 0 at the file:

$ wc -l < /etc/passwd
31
$ wc -l /etc/passwd
31 /etc/passwd

Spot the difference in the output. With <, wc was reading from stdin and never saw a filename, so it can’t print one. Trivial here, but it matters for tools like sort -o and anything that behaves differently for stdin versus a named file.

A here-document feeds a block of text from the script itself into stdin. The delimiter word is anything you like; EOF is just convention:

$ cat <<EOF
Host: $HOSTNAME, user: $USER
EOF
Host: web1, user: asif

Variables expand inside. Quote the delimiter and they don’t, which is what you want when writing config files that contain literal dollar signs:

$ cat > motd.txt <<'EOF'
Welcome to $HOSTNAME
No expansion in here because EOF is quoted
EOF
$ cat motd.txt
Welcome to $HOSTNAME
No expansion in here because EOF is quoted

That cat > file <<'EOF' pattern is the cleanest way to write a multi-line file from a script or an Ansible shell task. <<-EOF with a dash strips leading tabs so you can indent the block inside a function.

A here-string is the one-line version, and it’s the tidy way to feed a variable to a command without echo and a pipe:

$ grep -c o <<< "foo boo zoo"
1
$ read -r a b <<< "hello world"; echo "b=$b"
b=world

Bash and zsh only, as shown above. dash doesn’t have it.

Pipes: what actually happens

A pipe connects descriptor 1 of the left command to descriptor 0 of the right one, through a small kernel buffer. Nothing touches the disk.

$ cut -d: -f7 /etc/passwd | sort | uniq -c | sort -rn
     27 /usr/sbin/nologin
      2 /bin/bash
      1 /bin/sync
      1 /bin/false
$ du -sh /usr/* 2>/dev/null | sort -rh | head -5
5.6G	/usr/lib
385M	/usr/bin
291M	/usr/share
129M	/usr/local
25M	/usr/sbin

Three things about pipes that most guides skip, all of which explain behaviour you’ve probably seen and not understood.

Every command in a pipeline starts at the same time. The left side does not run to completion first:

$ (sleep 2; echo left done) | (echo right started; cat)
right started
left done

The right side printed before the left side had produced anything. This is why tail -f log | grep ERROR works as a live filter, and it’s why a pipeline’s memory use stays flat even when you push gigabytes through it. The buffer between them is 64 KB on Linux (ulimit -p says 8, but that’s 8 × 512 bytes = 4 KB, which is the largest write guaranteed to be atomic, not the capacity). When it’s full, the writer blocks until the reader catches up.

Only stdout goes through the pipe. stderr still points at the terminal:

$ ls /etc/hostname /nope | wc -l
ls: cannot access '/nope': No such file or directory
1

wc counted one line. The error skipped the pipe entirely and landed on screen. That’s usually what you want (errors don’t corrupt the data), and when it isn’t, 2>&1 | sends both through.

The pipeline’s exit status is the last command’s. Which brings us to the section that fixes the most broken backup scripts.

Exit codes in a pipeline: $?, PIPESTATUS and pipefail

$ false | true; echo $?
0
$ cat /nope | wc -l; echo $?
cat: /nope: No such file or directory
0
0

cat failed. wc happily counted zero lines and exited 0. The pipeline reports 0. Any script doing if pg_dump db | gzip > backup.gz; then echo "backup OK" will say “backup OK” when pg_dump can’t connect, because gzip compressed an empty stream without complaint.

Bash keeps the individual codes in an array:

$ false | true; echo "${PIPESTATUS[@]}"
1 0

And pipefail makes the pipeline’s status the rightmost non-zero code instead of the last command’s:

$ set -o pipefail; false | true; echo $?
1

In a script:

$ cat backup.sh
#!/bin/bash
set -o pipefail
tar -czf - /etc/nope 2>/dev/null | gzip -t
echo "exit: $?  PIPESTATUS: ${PIPESTATUS[@]}"
$ bash backup.sh
exit: 2  PIPESTATUS: 2 0

set -euo pipefail at the top of every bash script. It’s not a style choice, it’s the difference between a backup job that fails loudly and one that emails you “success” for six months while writing empty archives.

Exit code 141 and SIGPIPE

Once you start looking at PIPESTATUS you’ll see this:

$ seq 1 1000000 | head -1; echo "${PIPESTATUS[@]}"
1
141 0

seq “failed” with 141. It didn’t really. head read one line and exited, which closed the read end of the pipe. seq tried to write more, and the kernel sent it SIGPIPE (signal 13; 128 + 13 = 141). That’s the normal, correct way a pipeline shuts down early, and it’s why yes | head -3 doesn’t run forever.

The gotcha: with pipefail on, something | head now returns 141 and set -e kills your script. If you’re deliberately consuming partial output, either don’t use pipefail on that line, or accept 141 explicitly: cmd | head -1 || [ $? -eq 141 ].

The variable that vanishes after a pipe

This one costs people hours. Count lines with a while read loop fed by a pipe:

$ count=0; printf "a\nb\nc\n" | while read -r l; do count=$((count+1)); done; echo $count
0

Zero. The loop ran three times. Where did the count go?

Each side of a pipe runs in its own process. The while loop ran in a subshell, incremented its copy of count to 3, then exited. The parent shell’s count was never touched. Not a bug, just how processes work, and it applies to every variable, array, cd and export inside the right-hand side of a pipe.

Three fixes, in the order I’d reach for them:

# 1. Process substitution: the loop runs in the current shell
$ count=0; while read -r l; do count=$((count+1)); done < <(printf "a\nb\nc\n"); echo $count
3

# 2. lastpipe: tell bash to run the last pipeline stage in the current shell
$ shopt -s lastpipe; count=0; printf "a\nb\nc\n" | while read -r l; do count=$((count+1)); done; echo $count
3

# 3. Here-string, when the input is already in a variable
$ count=0; while read -r l; do count=$((count+1)); done <<< "$data"

lastpipe only works when job control is off, which means in scripts, not at an interactive prompt (bash silently ignores it there). Process substitution works everywhere bash does. Both are bash-only; in a #!/bin/sh script, write the output to a temp file and < it in.

Five traps that look correct

1. Redirecting a file onto itself

$ cat data.txt
b
a
c
$ cat data.txt | sort > data.txt
$ cat data.txt
$ ls -l data.txt
-rw-r--r-- 1 asif asif 0 Apr  8 17:12 data.txt

Gone. The shell sets up redirections before running the command, and > data.txt truncates the file to zero bytes. By the time cat opens it, it’s empty. This applies to grep -v x file > file, sed ... file > file, all of them.

Use the tool’s own in-place option where it has one (sort -o data.txt data.txt, sed -i), or sponge from the moreutils package, which soaks up all its input before writing:

$ sort data.txt | sponge data.txt; cat data.txt
a
b
c

2. sudo echo > /etc/something

$ sudo echo "10.0.0.5 db1" >> /etc/hosts
bash: /etc/hosts: Permission denied

Same reason. The redirection is done by your shell, as you, before sudo even starts. echo would have run as root, but the file was opened as you. Send the data through a pipe to a process that is root instead:

echo "10.0.0.5 db1" | sudo tee -a /etc/hosts > /dev/null

The trailing > /dev/null is optional; tee echoes what it writes, and you may or may not want that on screen. If you’re editing several system files, sudo tee is worth the muscle memory. The file management guide covers the rest of the ways to create and edit files safely.

3. ls | xargs rm and filenames with spaces

$ touch "file one.txt" "file two.txt"
$ ls file*.txt | xargs rm
rm: cannot remove 'file': No such file or directory
rm: cannot remove 'one.txt': No such file or directory
rm: cannot remove 'file': No such file or directory
rm: cannot remove 'two.txt': No such file or directory

xargs splits on whitespace by default, so “file one.txt” became two arguments. Nothing was deleted here, which is the lucky outcome; the unlucky one is when a file called one.txt also exists. Use null-separated input, which find can produce and xargs -0 can consume, and never parse ls in a script:

$ find . -name "file*.txt" -print0 | xargs -0 rm -v
removed './file one.txt'
removed './file two.txt'

4. Overwriting a file you meant to keep

noclobber makes > refuse to truncate an existing file. Some people put it in their .bashrc:

$ set -o noclobber
$ echo y > f.txt
bash: f.txt: cannot overwrite existing file
$ echo y >| f.txt; cat f.txt
y

>| is the “yes, I mean it” override. It’s a personal-preference setting rather than something to put in scripts, because scripts that assume > works will start failing in surprising places.

5. Output that arrives late, or not at all

Run this and nothing appears for three seconds, then all three lines land at once:

$ (for i in 1 2 3; do echo line $i; sleep 1; done) | grep line | cat
line 1
line 2
line 3

When stdout is a terminal, most programs flush after every line. When stdout is a pipe, the C library switches to block buffering, collecting a few KB (typically 4 KB) before writing anything. grep is sitting on “line 1” waiting for more. Kill the pipeline before the buffer fills and that output is lost forever, which is exactly what happens when a monitoring script gets killed by a timeout and the log is empty.

Tools that expect to sit in a live pipeline usually have a flag: grep --line-buffered, sed -u, awk with fflush(), python -u. For anything else, stdbuf -oL cmd from coreutils forces line buffering on a program that doesn’t offer the option.

$ (for i in 1 2 3; do echo line $i; sleep 1; done) | grep --line-buffered line | cat
line 1        # appears immediately
line 2
line 3

tee: see it and save it

tee reads stdin, writes it to a file and to stdout, so the pipeline continues:

$ ls /etc | head -3 | tee list.txt
NetworkManager
PackageKit
X11
$ wc -l list.txt
3 list.txt

The everyday one is make 2>&1 | tee build.log: watch the build scroll past and have the full log afterwards. tee -a appends rather than truncates. And as above, | sudo tee is how you write to a root-owned file from a pipeline.

The journalctl equivalent that comes up a lot on servers: journalctl -u nginx -f | tee -a /tmp/nginx-debug.log while you reproduce a problem.

Process substitution: a pipe that looks like a file

<(command) runs the command and gives you a path you can hand to anything expecting a filename:

$ echo <(true)
/dev/fd/63
$ ls -l <(true)
lr-x------ 1 asif asif 64 Apr  8 17:11 /dev/fd/63 -> pipe:[44279]

It’s a pipe, dressed up as a file in /dev/fd. This is how you diff two commands without temp files:

$ diff <(ls /etc | head -5) <(ls /usr | head -5)
1,5c1,5
< NetworkManager
< PackageKit
< X11
< adduser.conf
< alternatives
---
> bin
> games
> include
> lib
> lib64

Real uses: diff <(ssh web1 cat /etc/nginx/nginx.conf) <(ssh web2 cat /etc/nginx/nginx.conf) to compare config across servers, or comm -23 <(sort a) <(sort b) for set differences.

The output form, >(command), is rarer but solves one specific problem beautifully: sending stdout and stderr to two different processes:

$ ls /etc/hostname /nope 2> >(sed "s/^/ERR: /") > >(sed "s/^/OUT: /")
OUT: /etc/hostname
ERR: ls: cannot access '/nope': No such file or directory

Tag errors, timestamp output, ship stderr to a different log; all without a temp file.

exec: redirect the shell itself

Everything so far redirects one command. exec with a redirection and no command redirects the current shell, permanently, for everything that follows. Put this at the top of a script and every line of output goes to both the terminal and a log:

$ cat script.sh
#!/bin/bash
exec > >(tee -a script.log) 2>&1
echo "starting"
ls /nope
echo "done"
$ ./script.sh
starting
ls: cannot access '/nope': No such file or directory
done
$ cat script.log
starting
ls: cannot access '/nope': No such file or directory
done

No 2>&1 | tee on every line, no wrapper script, and it works for output from anything the script calls. This is the idiom for cron jobs and deploy scripts that need a full log.

exec also opens extra descriptors. You’re not limited to 0, 1 and 2. Open a file on 3, write to it several times, close it:

$ exec 3> notes.txt
$ echo "first" >&3
$ echo "second" >&3
$ exec 3>&-
$ cat notes.txt
first
second

The file stays open between writes, so this is faster than repeated >> in a loop and keeps the writes atomic relative to each other. exec 3< file does the same for reading (read -r line <&3), which lets you read one file line by line while stdin is still the terminal, so read -p "Continue?" inside the loop works.

And the party trick, swapping stdout and stderr using 3 as scratch space:

$ ls /etc/hostname /nope 3>&1 1>&2 2>&3 3>&- | sed "s/^/stderr via pipe: /"
/etc/hostname
stderr via pipe: ls: cannot access '/nope': No such file or directory

Read it left to right with the table in mind: save stdout on 3, point stdout at stderr, point stderr at the saved stdout, close 3. Now only the errors go down the pipe. Occasionally useful for filtering a noisy stderr while leaving real output alone.

/dev/null, /dev/stdout and named pipes

$ ls -l /dev/null /dev/stdin /dev/stdout /dev/stderr
crw-rw-rw- 1 root root 1, 3 Apr  8 17:04 /dev/null
lrwxrwxrwx 1 root root   15 Apr  8 17:04 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 root root   15 Apr  8 17:04 /dev/stdin -> /proc/self/fd/0
lrwxrwxrwx 1 root root   15 Apr  8 17:04 /dev/stdout -> /proc/self/fd/1

/dev/null is a real device. The other three are symlinks straight back into the process’s own fd table, which is why echo oops > /dev/stderr and echo oops >&2 do the same thing. The >&2 form is the one to use in scripts: it’s POSIX, and it works even in odd environments where /dev/stderr isn’t writable.

Many tools accept - as a filename meaning “stdin here”, which is handy when you want to mix a file and piped data:

$ echo "from stdin" | cat /etc/hostname -
web1
from stdin

A named pipe (FIFO) is a pipe with a path in the filesystem, so two unrelated processes can use it:

$ mkfifo myfifo; ls -l myfifo
prw-r--r-- 1 asif asif 0 Apr  8 17:11 myfifo
$ (echo "through the fifo" > myfifo &); cat myfifo
through the fifo

Note the p in the permissions. Writers block until a reader shows up and vice versa. You’ll meet these when a service wants to be fed from a script it doesn’t control, or when streaming a database dump straight into a restore on another host without staging it on disk.

Which one do I reach for?

If you take one thing from this page, make it the table of three numbered slots. > edits slot 1. 2> edits slot 2. < edits slot 0. 2>&1 copies slot 1 into slot 2, at that moment, in that order. A pipe wires slot 1 of one process to slot 0 of the next. Everything else on this page, and everything you’ll ever see in a Makefile or a cron entry or someone’s dotfiles, is a combination of those five moves.

For the commands that sit on either end of the pipes, the 50 essential Linux commands guide is the companion piece to this one.