Tutorial

An Introduction to Linux I/O Redirection

Updated on June 3, 2022
Default avatar

By David Collazo

An Introduction to Linux I/O Redirection

Introduction

The redirection capabilities built into Linux provide you with a robust set of tools to optimize many workflows. The “Unix philosophy” of software development was to make tools that each do one thing well, and this philosophy has been carried forward to modern command-line tools, which are individually powerful, and exponentially more so when combined. Whether you’re writing complex software or just working on the command line, knowing how to manipulate the different I/O streams in your environment will greatly increase your productivity.

Prerequisites

To follow along with this guide, you will need to have access to a Linux server. If you need information about connecting to your server for the first time, you can follow our guide on connecting to a Linux server using SSH.

Streams

Input and output in the Linux environment is distributed across three streams. These streams are:

  • standard input (stdin)

  • standard output (stdout)

  • standard error (stderr)

The streams are also numbered:

  • stdin (0)

  • stdout (1)

  • stderr (2)

During standard interactions between the user and the terminal, standard input comes from the user’s keyboard. Standard output and standard error are displayed on the user’s terminal as text. Collectively, the three streams are referred to as the standard streams.

Standard Input

The standard input stream typically carries data from a user to a program. Programs that expect standard input usually receive input from a device, such as a keyboard. Later in this tutorial, you will see examples of using one program’s output as Standard Input to another.

Standard Output

Standard output is the output that is generated by a program. When the standard output stream is not redirected, it will output text directly to the terminal. Try to output some arbitrary text, using echo:

  1. echo Sent to the terminal
Output
Sent to the terminal

When used without any additional options, the echo command outputs any argument that is passed to it on the command line.

Run echo without any arguments:

  1. echo

It will return an empty line. Some programs do not do anything without provided arguments.

Standard Error

Standard error contains errors generated by a program that has failed in some way. Like standard output, the default destination for this stream is the terminal display.

Let’s see a basic example of standard error using the ls command. ls lists a directory’s contents.

When run without an argument, ls lists the contents within the current directory. If ls is run with a directory as an argument, it will list the contents of the provided directory.

  1. ls %

Since % is not an existing directory, this will send the following text to standard error:

Output
ls: cannot access %: No such file or directory

A program does not have to crash or finish running in order to generate Standard Error, and whether some output is sent to either Standard Output or Standard Error is down to the behavior of the program. They are not technically different from one another in any way — just that one output stream is supposed to be reserved for error messages, and some tools will assume that Standard Error being empty means that a program ran successfully. Some programs will even output minor errors to Standard Error without crashing or failing to also produce the intended output. It is only used as a convention to separate intended output from unintended output.

Stream Redirection

Linux includes redirection commands for each stream. These can be used to write standard output or standard error to a file. If you write to a file that does not exist, a new file with that name will be created prior to writing.

Commands with a single bracket overwrite the destination’s existing contents.

Overwrite

  • > - standard output

  • < - standard input

  • 2> - standard error

Commands with a double bracket do not overwrite the destination’s existing contents.

Append

  • >> - standard output

  • << - standard input

  • 2>> - standard error

Pipes

Pipes are used to redirect a stream from one program to another. When a program’s standard output is sent to another through a pipe, the first program’s output will be used as input to the second, rather than being printed to the terminal. Only the data returned by the second program will be displayed.

The Linux pipe is represented by a vertical bar: |

Here is an example of a command using a pipe:

  1. ls | less

This takes the output of ls, which displays the contents of your current directory, and pipes it to the less program. less displays the data sent to it one line at a time.

ls normally displays directory contents across multiple rows. When you run it through less, each entry is placed on a new line.

Though the functionality of the pipe may appear to be similar to that of > and >>, the distinction is that pipes redirect data from one command to another, while > and >> are used to redirect exclusively to files.

Filters

Filters are are a class of programs that are commonly used with output piped from another program. Many of them are also useful on their own, but they illustrate piping behavior especially well.

  • find - returns files with filenames that match the argument passed to find.

  • grep - returns text that matches the string pattern passed to grep.

  • tee - redirects standard input to both standard output and one or more files.

  • tr - finds-and-replaces one string with another.

  • wc - counts characters, lines, and words.

Examples

Now that you have been introduced to redirection, piping, and basic filters, let’s look at some common redirection patterns and examples.

The command > file pattern redirects the standard output of a command to a file.

  1. ls ~ > root_dir_contents.txt

The command above passes the contents of your home directory (~) as standard output, and writes the output to a file named root_dir_contents.txt. It will delete any prior contents in the file, as it is a single-bracket command.

The command > /dev/null pattern redirects standard output to nowhere. /dev/null is a special file that is used to trash any data that is redirected to it. It is used to discard standard output that is not needed, and that might otherwise interfere with the functionality of a command or a script. Any output that is sent to /dev/null is discarded.

  1. ls > /dev/null

This command discards the standard output stream returned from the command ls by passing it to /dev/null.

This command 2> file pattern redirects the standard error stream of a command to a file, overwriting existing contents.

  1. mkdir '' 2> mkdir_log.txt

This redirects the error raised by the invalid directory name '', and writes it to log.txt. Note that the error is still sent to the terminal and displayed as text.

The command >> file pattern redirects the standard output of a command to a file without overwriting the file’s existing contents.

  1. echo Written to a new file > data.txt
  2. echo Appended content to an existing file >> data.txt

This pair of commands first redirects the text inputted by the user through echo to a new file. It then appends the text received by the second echo command to the existing file, without overwriting its contents.

The command 2>> file pattern above redirects the standard error stream of a command to a file without overwriting the file’s existing contents. This pattern is useful for creating error logs for a program or service, as the log file will not have its previous content wiped each time the file is written to.

  1. find '' 2> stderr_log.txt
  2. wc '' 2>> stderr_log.txt

The above command redirects the error message caused by an invalid find argument to a file named stderr_log.txt. It then appends the error message caused by an invalid wc argument to the same file.

The command | command pattern redirects the standard output from the first command to the standard input of the second command.

  1. find /var lib | grep deb

This command searches through /var and its subfolders for filenames and extensions that match the string deb, and returns the file paths for the files, with the matching portion in each path highlighted in red.

The command | tee file pattern (which includes the tee command) redirects the standard output of the command to a file and overwrites its contents. Then, it displays the redirected output in the terminal. It creates a new file if the file does not already exist.

In the context of this pattern, tee is typically used to view a program’s output while simultaneously saving it to a file.

  1. wc /etc/magic | tee magic_count.txt

This pipes the counts for characters, lines, and words in the /etc/magic file (used by the Linux shell to determine file types) to the tee command, which then splits wc’s output in two directions, and sends it to the terminal display and the magic_count.txt file. For the tee command, imagine the letter T. The bottom part of the letter is the initial data, and the top part is the data being split in two different directions (standard output and the terminal).

Multiple pipes can be used to redirect output across multiple commands and/or filters.

Conclusion

Learning how to use the redirection capabilities built into the Linux command line is a crucial skill. Now that you have seen the basics of how redirections and pipes work, you’ll be able to begin your journey into the world of shell scripting, which makes frequent use of the programs and patterns highlighted in this guide.

Searching for specific commands, or for something that you would like to do in the command line (e.g. “delete all files in a directory that begin with an uppercase letter”) can also prove helpful when you need to accomplish a specific task using the command line.

Thanks for learning with the DigitalOcean Community. Check out our offerings for compute, storage, networking, and managed databases.

Learn more about us


Tutorial Series: Getting Started with Linux

If you are new to Linux and its command line interface, it may seem like a daunting task to get started with it. This series will bring you up to speed with essential Linux basics, and provide a solid foundation for working with Linux servers. If you have little to no knowledge about using Linux, this is where you will want to start.

About the authors
Default avatar
David Collazo

author

Still looking for an answer?

Ask a questionSearch for more help

Was this helpful?
 
10 Comments


This textbox defaults to using Markdown to format your answer.

You can type !ref in this text area to quickly search our full set of tutorials, documentation & marketplace offerings and insert the link!

man tee | less is bit redundant and also strip the colours and bold formatting. man already use less pager by default.

Is there a way to create a custom stream? Like this:

$ mksth /dev/mystrm
$ (/bin/some_program < /dev/mystrm > /output.log &)
$ echo "some_code" > /dev/mystrm #Pass some code to 'some_program' that it recognizes via custom stream
$ tail /output.log #Get results for 'some_code'

in the examples part, in the second last example, maybe you meant

find /var/lib | grep deb

not

find /var lib | grep deb

it was pretty confusing to me as a beginner, but then I realized you are giving two arguments to the find command which doesn’t make a lot of sense here

This pattern redirects the standard error stream of a command to a file, overwriting existing contents.

mkdir ‘’ 2> mkdir_log.txt

This redirects the error raised by the invalid directory name ‘’, and writes it to log.txt. Note that the error is still sent to the terminal and displayed as text.

This is an error.

As soon as you redirect with 2> , all the error messages are gone to that device or file. It won’t display anything on the terminal (i.e. the stdout) .

When I run the command ‘find /var lib | grep deb’ as demonstrated in the ‘command | command’ section, it shows files containing the string ‘deb’ in /var directory, followed by the error message ‘find: ‘lib’: No such file or directory’. What is missing is a ‘/’ between ‘/var’ and ‘lib’ in the command. To get the error-free result, execute the following command: ‘find /var/lib | grep deb’. (Note the ‘/’ symbol between ‘var’ and ‘lib’.)

Thank you so much for great tutorial!

echo Written to a new file > data.txt
echo Appended to an existing file's contents >> data.txt

This doesn’t seem to work unless you remove the

Thank you very much! Awesome explanation.

Etel Sverdlov
DigitalOcean Employee
DigitalOcean Employee badge
January 24, 2014

Thank you for catching that. We have updated the article :)

Thanks for the useful info but there is an error in your article: you mention “Commands with a single bracket do not overwrite the destination’s existing contents.” but this should be “Commands with double brackets”.

Try DigitalOcean for free

Click below to sign up and get $200 of credit to try our products over 60 days!

Sign up

Join the Tech Talk
Success! Thank you! Please check your email for further details.

Please complete your information!

Get our biweekly newsletter

Sign up for Infrastructure as a Newsletter.

Hollie's Hub for Good

Working on improving health and education, reducing inequality, and spurring economic growth? We'd like to help.

Become a contributor

Get paid to write technical tutorials and select a tech-focused charity to receive a matching donation.

Welcome to the developer cloud

DigitalOcean makes it simple to launch in the cloud and scale up as you grow — whether you're running one virtual machine or ten thousand.

Learn more
DigitalOcean Cloud Control Panel