Linux Basics – sed
As Linux commands go, this one has a very confusing name. That is, until someone tells you that like many Linux commands, it is yet another abbreviation. This time,
sed is short for stream editor. Many editors for text files operate in a sort of point-click-and-save fashion. They cannot edit a real-time stream of data such as a log file.
sed, on the other hand, is able to modify text streaming through it in real-time. The
sed command is a wonderful utility. The language is very simple, but the documentation is terrible. We asked our friendly Linux guru at HOSTAFRICA to explain it to us.
The most important feature of sed
Used for daily tasks specifically, the most useful feature of
sed is the ‘s‘ function. The ‘s‘ stands for substitution. The
substitute command changes all occurrences of the regular expression into a new value. A simple example, using our
sample.txt file from the previous Linux Basics blog (see below) is changing
dog in the
sample.txt file to
mouse in the
~$: cat sample.txt The quick brown fox jumped over the lazy dog .
Now to use the stream editor
sed to change
~$ cat sample.txt |sed s/dog/mouse/ The quick brown fox jumped over the lazy mouse .
Notice the command output content of the file
sample.txt, but we pipe it into the
sed command. We then instruct
sed to substitute
mouse, which looks as such
s/dog/mouse/. We can also do this without the
cat command as
sed is clever enough to read files without assistance from a
cat. Also, note that we have not yet modified anything.
~$ sed s/dog/mouse/ sample.txt The quick brown fox jumped over the lazy mouse .
Now we will redirect the output from
~$ sed s/dog/mouse/ sample.txt >new.txt ~$ cat new.txt The quick brown fox jumped over the lazy mouse .
Check content of
~$ cat sample.txt The quick brown fox jumped over the lazy dog .
Now you can see,
sed stood in the data stream between
new.txt and changed
mouse. In general, it is better to quote your
sed expression like this:
~$ echo "dog"
~$ echo "dog" |sed 's/dog/mouse/'
Let’s be creative
~$ echo "hounddog" |sed 's/dog/mouse/'
Another important concept is that
sed is line oriented. Suppose you have the input file:
one two three, one two three four three two one one hundred
and you used the command
~$ sed 's/one/ONE/' <file
The output would be:
ONE two three, one two three four three two ONE ONE hundred
Note that this changed
ONE once on each line. The first line had
one twice, but only the first occurrence was changed. That’s the default behaviour. If you want something different, you’ll have to use some of the options that are available.
Let’s return to the command:
There’re four parts to this substitute command:
s Substitute command
one Regular Expression Pattern Search Pattern
ONE Replacement string
The search pattern is on the left-hand side and the replacement string is on the right-hand side. The search pattern may be matched using REGULAR EXPRESSIONS, just as we did with the
The character after the
s is the delimiter. It is conventionally a slash because this is what
vi use. It can be anything you want. If you want to change a pathname that contains a slash – say
/usr/local/bin to /common/bin – you could use the backslash to quote the slash as below:
but it is easier to read if you use an underline instead of a slash as a delimiter:
Use any delimiter you like, as long as it does not occur in the search string. Always remember that you need 3 delimiters.
There are many, many more options we won’t mention here
sed command has many more options and is incredibly powerful. As this is a Linux Basics series, we will not delve into the magic of
sed any further in this post.
However, if you’re looking for a deeper understanding, one of the best comprehensive guides to
sed is by Bruce Barnet at grymoire.com.