Shell: pipes and redirections

In short: In the shell a pipe (|) connects the output of one program to the input of the next, and redirections (>, <) steer input and output into files. From these you build powerful command chains out of small tools.

In more detail: Every program has three standard channels: input (stdin, 0), output (stdout, 1) and error output (stderr, 2). The shell can rewire them.

In Depth

• Pipe: cat log.txt | grep ERROR | sort | uniq -c | sort -rn | head filters, counts and ranks error messages. The programs run concurrently.
• Output to a file: command > file overwrites, command >> file appends.
• Input from a file: command < file.
• Separating errors: command 2> errors.txt, both together command > all.txt 2>&1 or short &> all.txt (bash). Mind the order: 2>&1 copies where 1 currently points.
• Discarding: command > /dev/null 2>&1.
• Show and save at once: command | tee file.txt.
• Command substitution: $(command) inserts one command’s output into another line.
• Here-strings and here-docs: <<< "text" and <<EOF ... EOF pass text as input.
• Success in pipes: By default only the last command’s return value counts. set -o pipefail reports earlier failures too.

See also: writing shell scripts, Bash