Linux Redirection and Pipelines: stdin, stdout, stderr, and tee
I composed single-line data pipelines using redirection, pipes, and tee to filter, transform, and log output in one pass. Understanding the three standard streams let me capture stderr separately and split a pipeline to both a file and the screen.
Objective & Context
The Unix philosophy is small tools composed via streams. This lab masters file descriptors 0/1/2, pipe composition, and tee for branching, the glue behind every filter and scripting lab.
Learning Objectives
- Distinguish file descriptors 0 (stdin), 1 (stdout), and 2 (stderr).
- Redirect stdout and stderr to separate files or merge them.
- Compose multi-tool pipelines where each stage transforms the stream.
- Branch a stream to both a file and the next pipe stage with tee.
- Use process substitution and here-strings where pipes fall short.
Environment & Prerequisites
Prerequisites
- Any Bash shell with GNU coreutils (standard on all Linux distributions).
- A command that writes to both stdout and stderr (make, gcc, or curl -v).
- A log destination file for tee practice.
- No extra packages needed.
Step-by-Step Execution
The outputs on this page come from a re-run on 14 November 2026 in an Ubuntu 24.04 container (arm64). Sample files, users, and hosts were created for the run, so names, numbers, and timestamps reflect that run.
1. Separate stdout and stderr
make 2> build-errors.log 1> build-out.logOutput:
$ wc -l build-out.log build-errors.log
42 build-out.log
3 build-errors.log
45 total
What this teaches: 2> captures only the error stream while 1> captures normal output; separating them keeps error triage clean. Order matters: 2>&1 must come after the stdout redirect to merge correctly.
2. Filter through a pipe
journalctl -u nginx | grep -i error | tail -20Output:
Nov 14 17:06:38 srv01 nginx[430]: 2026/11/14 22:06:38 [error] 430#430: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:9/", host: "localhost"
Nov 14 17:06:38 srv01 nginx[432]: 2026/11/14 22:06:38 [error] 432#432: *3 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:9/", host: "localhost"
Nov 14 17:06:38 srv01 nginx[431]: 2026/11/14 22:06:38 [error] 431#431: *5 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:9/", host: "localhost"
What this teaches: Each pipe stage filters the stream further: journalctl emits, grep keeps error lines, tail windows the last 20. Data flows left to right with no temporary files.
3. Split output with tee
dmesg | tee dmesg.log | grep -i failOutput:
[ 0.000000] NUMA: Failed to initialise from firmware
What this teaches: tee duplicates stdin to a file and stdout simultaneously, so you archive the raw dmesg output while grep shows only failures live. Add -a to append instead of overwriting.
4. Feed two live streams to one command with process substitution
diff <(sort hosts-a.txt) <(sort hosts-b.txt)Output:
3d2
< web02
What this teaches: <(...) runs a command and presents its output as a readable file path (a /dev/fd entry), so tools like diff that demand two named file arguments can consume live command output. This covers the case pipes cannot: a command that needs two inputs at once. The >(...) form does the reverse, presenting a writable path.
How It Works
Every Linux process starts with three open streams: stdin (0) for input, stdout (1) for normal output, and stderr (2) for diagnostics. Redirection operators rebind these numbers to files before the command runs, which is why 2>file captures errors while leaving normal output on the terminal. The shell performs redirection itself; the command never knows its output went to a file.
A pipeline connects stdout of one process directly to stdin of the next through a kernel pipe buffer, so data streams through without touching disk. This is what makes long chains fast: each stage processes lines as they arrive. Merging with 2>&1 duplicates descriptor 2 onto wherever descriptor 1 currently points, which is why the ordering of redirections changes the result.
tee exists for the cases where you want both persistence and live processing: it reads stdin once and writes it to every named file plus stdout. Combined with process substitution, >(cmd), you can even fan one stream out to several processing commands at once, turning a linear pipeline into a small dataflow graph.
Validation & Testing
Run a command that emits both streams and confirm errors land in their own file while stdout is filtered. Pass criteria: stderr captured separately, the pipeline filters correctly, and tee writes the full stream to a file while passing it onward.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| 2>&1 produced an empty file | The merge was placed before the stdout redirect | Write cmd >file 2>&1 so stderr duplicates the already-redirected stdout |
| Pipeline shows output but the file is empty | Buffering: the writer holds data until its buffer fills or it exits | Use stdbuf -o0 or the tool's --line-buffered flag for live streams |
| tee overwrote yesterday's log | tee truncates by default like > | Add -a to append instead of overwriting |
| sudo cmd > /root/file gives permission denied | Redirection runs as your user, before sudo elevates | Use sudo tee: cmd | sudo tee /root/file >/dev/null |
| Pipe to head kills the producer with SIGPIPE | head exits early and the writer gets a broken pipe signal | Usually harmless; add 2>/dev/null if the noise bothers you |
| Here-string adds an unwanted trailing newline | <<< always appends a newline to the word | Use printf without \n piped in, or strip it downstream |
Advanced: Troubleshooting
- Errors not captured: redirect
2>explicitly; pipes only carry stdout by default. - Both streams merged: use
2>&1after the stdout redirect, order matters. - tee not writing: add
-ato append rather than overwrite.
Key Results
- Captured stdout and stderr to separate files in one command.
- Built filter pipelines that reduce noisy logs to relevant lines.
- Branched output to file and screen simultaneously with tee.
- Internalized the three standard streams for reliable scripting.
Key Takeaways
- Descriptors 0, 1, 2 are stdin, stdout, stderr; redirection rebinds them before the command runs.
- Separate streams for triage; merge with >file 2>&1 in that order when you want everything.
- Pipelines stream through kernel buffers with no temporary files.
- tee archives and displays simultaneously; -a appends.
- Redirection happens as your user, so sudo needs tee for privileged files.
Practice Challenges
Challenge 1: Silent build log
Goal: run a build so normal output goes to build-out.log, errors go to build-errors.log, and your terminal shows only a one-line summary.
Hint: redirect both streams, then echo based on the exit code.
Challenge 2: Fan-out monitoring
Goal: send one journalctl stream to three places at once: a full archive file, a live error-only view, and a count of critical lines.
Hint: combine tee with process substitution >( ).
Frequently Asked Questions
How do I fix '2>&1 produced an empty file'?
Likely cause: The merge was placed before the stdout redirect. Fix: Write cmd >file 2>&1 so stderr duplicates the already-redirected stdout.
How do I fix 'Pipeline shows output but the file is empty'?
Likely cause: Buffering: the writer holds data until its buffer fills or it exits. Fix: Use stdbuf -o0 or the tool's --line-buffered flag for live streams.
How do I fix 'tee overwrote yesterday's log'?
Likely cause: tee truncates by default like >. Fix: Add -a to append instead of overwriting.
How do I fix 'sudo cmd > /root/file gives permission denied'?
Likely cause: Redirection runs as your user, before sudo elevates. Fix: Use sudo tee: cmd | sudo tee /root/file >/dev/null.
How do I fix 'Pipe to head kills the producer with SIGPIPE'?
Likely cause: head exits early and the writer gets a broken pipe signal. Fix: Usually harmless; add 2>/dev/null if the noise bothers you.
Cite This Page
Citation formats
Plain text
Taki Sadik, "Linux Redirection and Pipelines: stdin, stdout, stderr, and tee", TYFSADIK, https://tyfsadik.org/blog/labs/linux-fundamentals/redirection-pipelines (accessed 2026-10-22).
BibTeX
@misc{sadik-redirection-pipelines,
author = {Sadik, MD. Taki Yasir Faraji},
title = {Linux Redirection and Pipelines: stdin, stdout, stderr, and tee},
year = {2026},
url = {https://tyfsadik.org/blog/labs/linux-fundamentals/redirection-pipelines},
urldate = {2026-10-22}
}
APA
Sadik, M. T. Y. F. (2026). Linux Redirection and Pipelines: stdin, stdout, stderr, and tee. TYFSADIK. https://tyfsadik.org/blog/labs/linux-fundamentals/redirection-pipelines