Christopher Negus - Linux Bible

A truly powerful feature of the shell is the capability to redirect the input and output of commands to and from other commands and files. To allow commands to be strung together, the shell uses metacharacters. A metacharacter is a typed character that has special meaning to the shell for connecting commands or requesting expansion.

Metacharacters include the pipe character ( |), ampersand ( &), semicolon ( ;), right parenthesis ( )), left parenthesis ( (), less than sign ( <), and greater than sign ( >). The next sections describe how to use metacharacters on the command line to change how commands behave.

The pipe ( |) metacharacter connects the output from one command to the input of another command. This lets you have one command work on some data and then have the next command deal with the results. Here is an example of a command line that includes pipes:

$ cat /etc/passwd | sort | less

This command lists the contents of the /etc/passwdfile and pipes the output to the sortcommand. The sortcommand takes the usernames that begin each line of the /etc/passwdfile, sorts them alphabetically, and pipes the output to the lesscommand (to page through the output).

Pipes are an excellent illustration of how UNIX, the predecessor of Linux, was created as an operating system made up of building blocks. A standard practice in UNIX was to connect utilities in different ways to get different jobs done. For example, before the days of graphical word processors, users created plain-text files that included macros to indicate formatting. To see how the document really appeared, they would use a command such as the following:

$ gunzip < /usr/share/man/man1/grep.1.gz | nroff -c -man | less

In this example, the contents of the grepman page ( grep.1.gz) are directed to the gunzipcommand to be unzipped. The output from gunzipis piped to the nroffcommand to format the man page using the manual macro ( -man). To display the output, it is piped to the lesscommand. Because the file being displayed is in plain text, you could have substituted any number of options to work with the text before displaying it. You could sort the contents, change or delete some of the content, or bring in text from other documents. The key is that, instead of all of those features being in one program, you get results from piping and redirecting input and output between multiple commands.

Sometimes, you may want a sequence of commands to run, with one command completing before the next command begins. You can do this by typing several commands on the same command line and separating them with semicolons ( ;):

$ date ; troff -me verylargedocument | lpr ; date

In this example, I was formatting a huge document and wanted to know how long it would take. The first command ( date) showed the date and time before the formatting started. The troffcommand formatted the document and then piped the output to the printer. When the formatting was finished, the date and time were printed again (so I knew how long the troffcommand took to complete).

Another useful command to add to the end of a long command line is mail. You could add the following to the end of a command line:

; mail -s "Finished the long command" chris@example.com

Then, for example, a mail message is sent to the user you choose after the command completes.

Some commands can take a while to complete. Sometimes, you may not want to tie up your shell waiting for a command to finish. In those cases, you can have the commands run in the background by using the ampersand ( &).

Text formatting commands (such as nroffand troff, described earlier) are examples of commands that can be run in the background to format a large document. You also might want to create your own shell scripts that run in the background to check continuously for certain events to occur, such as the hard disk filling up or particular users logging in.

The following is an example of a command being run in the background:

$ troff -me verylargedocument | lpr &

Don't close the shell until the process is completed or that kills the process. Other ways to manage background and foreground processes are described in Chapter 6, “Managing Running Processes.”

With command substitution, you can have the output of a command interpreted by the shell instead of by the command itself. In this way, you can have the standard output of a command become an argument for another command. The two forms of command substitution are $(command)and `command`(backticks, not single quotes).

The command in this case can include options, metacharacters, and arguments. The following is an example of using command substitution:

$ vi $(find /home | grep xyzzy)

In this example, the command substitution is done before the vicommand is run. First, the findcommand starts at the /homedirectory and prints out all of the files and directories below that point in the filesystem. The output is piped to the grepcommand, which filters out all files except for those that include the string xyzzyin the filename. Finally, the vicommand opens all filenames for editing (one at a time) that include xyzzy. (If you run this and are not familiar with vi, you can type :q!to exit the file.)

This particular example is useful if you want to edit a file for which you know the name but not the location. As long as the string is uncommon, you can find and open every instance of a filename existing beneath a point you choose in the filesystem. (In other words, don't use grepfrom the root filesystem or you'll match and try to edit several thousand files.)

Sometimes, you want to pass arithmetic results to a command. There are two forms that you can use to expand an arithmetic expression and pass it to the shell: $ [expression] or $ (expression) . The following is an example:

$ echo "I am $[2020 - 1957] years old." I am 63 years old.

The shell interprets the arithmetic expression first ( 2020 - 1957) and then passes that information to the echocommand. The echocommand displays the text with the results of the arithmetic ( 63) inserted.

Here's an example of the other form:

$ echo "There are $(ls | wc -w) files in this directory." There are 14 files in this directory.

This lists the contents of the current directory ( ls) and runs the word count command to count the number of files found ( wc -w). The resulting number (14, in this case) is echoed back with the rest of the sentence shown.

Variables that store information within the shell can be expanded using the dollar sign ( $) metacharacter. When you expand an environment variable on a command line, the value of the variable is printed instead of the variable name itself, as follows:

$ ls -l $BASH -rwxr-xr-x. 1 root root 1219248 Oct 12 17:59 /usr/bin/bash