|
| 1 | +# Introduction to Redirection |
| 2 | + |
| 3 | +In the world of the command-line interface, programs often interact with three standard data streams: |
| 4 | + |
| 5 | +1. **Standard Input** (`stdin`): Where a program receives its input (typically from the keyboard). |
| 6 | +1. **Standard Output** (`stdout`): Where a program sends its normal output (typically to the terminal screen). |
| 7 | +1. **Standard Error** (`stderr`): Where a program sends error messages (typically to the terminal screen). |
| 8 | + |
| 9 | +Recall that we introduced these in the [Pipelines and Command Lists][pipelines] chapter. |
| 10 | + |
| 11 | +~~~~exercism/note |
| 12 | +Pipelines can be considered a form of redirection: the standard output from one program is sent directly to the standard input of another. |
| 13 | +We've already covered pipelines. |
| 14 | +This concept will cover redirecting to and from files. |
| 15 | +~~~~ |
| 16 | + |
| 17 | +Shell redirection is a powerful mechanism that allows you to change the destination of these streams. |
| 18 | +Instead of a program reading from the keyboard or writing to the screen, you can redirect input from a file or send output to a file. |
| 19 | +This provides considerable flexibility and control over how programs interact with data. |
| 20 | + |
| 21 | +## Key Redirection Operators |
| 22 | + |
| 23 | +Here are the most common redirection operators and what they do: |
| 24 | + |
| 25 | +### `>` (Output Redirection) |
| 26 | + |
| 27 | +* Purpose: Redirect the standard output (`stdout`) of a command to a file. |
| 28 | +* Behavior: |
| 29 | + * If the file doesn't exist, it is created. |
| 30 | + * If the file exists, its contents are overwritten. |
| 31 | +* Example: |
| 32 | + |
| 33 | + ```bash |
| 34 | + # Sends the output of 'ls -l' to 'file_listing.txt' |
| 35 | + ls -l > file_listing.txt |
| 36 | + ``` |
| 37 | + |
| 38 | +~~~~exercism/caution |
| 39 | +Redirection is performed **before** the command is executed. |
| 40 | +This means you cannot redirect to the same file you need to read from. |
| 41 | +
|
| 42 | +```bash |
| 43 | +date > date.txt # populate the file with the current date |
| 44 | +cat date.txt > date.txt # 'date.txt' is now empty! |
| 45 | +``` |
| 46 | +
|
| 47 | +In the second command of the above example, |
| 48 | +
|
| 49 | +1. Bash first truncates the file named in the redirection. |
| 50 | + This destroys the previous contents, leaving an empty file. |
| 51 | +2. Then bash reads the newly empty file. |
| 52 | +3. Then bash writes the contents (i.e., nothing) back to the file. |
| 53 | +~~~~ |
| 54 | + |
| 55 | +### `>>` (Append Output Redirection) |
| 56 | + |
| 57 | +* Purpose: Append the standard output (`stdout`) of a command to a file. |
| 58 | +* Behavior: |
| 59 | + * If the file doesn't exist, it is created. |
| 60 | + * If the file exists, the output is added to the end of the file. |
| 61 | +* Example: |
| 62 | + |
| 63 | + ```bash |
| 64 | + # Adds "Another line" to the end of the file |
| 65 | + echo "Another line" >> file_listing.txt |
| 66 | + ``` |
| 67 | + |
| 68 | +### `<` (Input Redirection) |
| 69 | + |
| 70 | +* Purpose: Redirect the standard input (`stdin`) of a command from a file. |
| 71 | +* Behavior: The command reads its input from the specified file instead of the keyboard. |
| 72 | +* Example: |
| 73 | + |
| 74 | + ```bash |
| 75 | + # 'tr' reads from 'lowercase.txt' and |
| 76 | + # output is redirected to 'uppercase.txt' |
| 77 | + tr 'a-z' 'A-Z' < lowercase.txt > uppercase.txt |
| 78 | + ``` |
| 79 | + |
| 80 | +### `2>` (Error Redirection): |
| 81 | + |
| 82 | +* Purpose: Redirect the standard error (`stderr`) of a command to a file. |
| 83 | +* Behavior: Error messages are sent to the specified file instead of the terminal. |
| 84 | +* Example: |
| 85 | + ```bash |
| 86 | + # Error message from 'rm' is sent to 'error_log.txt' |
| 87 | + rm non_existent_file 2> error_log.txt |
| 88 | + ``` |
| 89 | + |
| 90 | +You can _append_ standard error to a file with `2>>`. |
| 91 | + |
| 92 | +### Additional Redirections |
| 93 | + |
| 94 | +There are also redirections that can merge both `stdin` and `stderr`. |
| 95 | +See the [manual][manual] for details. |
| 96 | + |
| 97 | +## `/dev/null` |
| 98 | + |
| 99 | +`/dev/null` is a special file that |
| 100 | + |
| 101 | +* is an empty file when you read from it, and |
| 102 | +* discards all data written to it. |
| 103 | + |
| 104 | +It is sometimes referred to as "the bitbucket". |
| 105 | +It is useful for suppressing output or errors. |
| 106 | + |
| 107 | +```bash |
| 108 | +some_command > /dev/null # Discards stdout |
| 109 | +some_command 2> /dev/null # Discards stderr |
| 110 | + |
| 111 | +# Using /dev/null as input will prevent the command |
| 112 | +# from reading data from the keyboard |
| 113 | +some_command < /dev/null |
| 114 | +``` |
| 115 | + |
| 116 | +## Combining redirections |
| 117 | + |
| 118 | +Multiple redirections can be given at once. |
| 119 | + |
| 120 | +```bash |
| 121 | +some_command < input.txt > output.txt 2> error.txt |
| 122 | +``` |
| 123 | + |
| 124 | +A redirection can be copied. |
| 125 | + |
| 126 | +```bash |
| 127 | +# send stderr to the same destination as stdout |
| 128 | +some_command > output.txt 2>&1 |
| 129 | +``` |
| 130 | + |
| 131 | +When you see `2>&1`, read that like "redirect stderr to _whatever stdout is **currently** pointing to_". |
| 132 | +Bash processes redirection instructions strictly from left to right, which means you can do: |
| 133 | + |
| 134 | +```bash |
| 135 | +# redirect stdout to 'output1.txt' |
| 136 | +# redirect stderr to the same place |
| 137 | +# then change stdout to 'output2.txt' |
| 138 | +some_command > output1.txt 2>&1 > output2.txt |
| 139 | +``` |
| 140 | + |
| 141 | +~~~~exercism/advanced |
| 142 | +## File Descriptors |
| 143 | +
|
| 144 | +In the context of shell programming and operating systems like Linux/Unix, a file descriptor (FD) is a non-negative integer that serves as an identifier or handle for an open file or I/O (Input/Output) resource. |
| 145 | +Think of it like a label or a reference number that the operating system uses to keep track of open files and data streams. |
| 146 | +
|
| 147 | +The numbers you see on the left-hand side of redirections are file descriptors. |
| 148 | +
|
| 149 | +The standard I/O streams are given the first 3 file descriptors: |
| 150 | +
|
| 151 | +* standard input is FD 0 |
| 152 | +* standard output is FD 1 |
| 153 | +* standard error is FD 2 |
| 154 | +
|
| 155 | +New file descriptors can be set or created with the `exec` command and redirection operators. |
| 156 | +
|
| 157 | +```bash |
| 158 | +# create FD 3 as a copy of stdout |
| 159 | +exec 3>&1 |
| 160 | +# send stdout to a log file |
| 161 | +exec > log.txt |
| 162 | +
|
| 163 | +# then redirect both stdout and file descriptor 3 |
| 164 | +echo "some debug message" >&3 # displayed in the terminal |
| 165 | +some_command 2>&1 # stdout and stderr send to 'log.txt' |
| 166 | +``` |
| 167 | +
|
| 168 | +This technique looks quite convoluted, but it can be very useful in situations where your script is being run by some other software package, and standard output is automatically captured by the software. |
| 169 | +The `bats` command that is used to run unit tests for this track works like this: `bats` captures a test's stdout and stderr, but debug messages sent to FD 3 will be displayed on the terminal. |
| 170 | +~~~~ |
| 171 | + |
| 172 | +## In Summary |
| 173 | + |
| 174 | +Why is redirection useful? |
| 175 | + |
| 176 | +* Saving Output: Store the results of a command for later review or processing. |
| 177 | +* Logging: Capture error messages for debugging. |
| 178 | +* Automation: Chain commands together, using the output of one as the input of another. |
| 179 | +* Filtering: Discard unwanted output. |
| 180 | +* Batch Processing: Process data from files instead of typing it manually. |
| 181 | +* Scripting: Redirections are essential for writing robust shell scripts. |
| 182 | + |
| 183 | +Shell redirections give you fine-grained control over how programs interact with data, enabling you to build complex workflows and manage information effectively. |
| 184 | + |
| 185 | +[pipelines]: https://exercism.org/tracks/bash/concepts/pipelines |
| 186 | +[manual]: https://www.gnu.org/software/bash/manual/bash.html#Redirections |
0 commit comments