Linux Basics – sed
As Linux commands go, this one has a very confusing name. That is, until someone tells you that like many Linux commands, it is yet another abbreviation. This time, sed
is short for stream editor. Many editors for text files operate in a sort of point-click-and-save fashion. They cannot edit a real-time stream of data such as a log file. sed
, on the other hand, is able to modify text streaming through it in real-time. The sed
command is a wonderful utility. The language is very simple, but the documentation is terrible. We asked our friendly Linux guru at HOSTAFRICA to explain it to us.
The most important feature of sed
Used for daily tasks specifically, the most useful feature of sed
is the ‘s‘ function. The ‘s‘ stands for substitution. The substitute
command changes all occurrences of the regular expression into a new value. A simple example, using our sample.txt
file from the previous Linux Basics blog (see below) is changing dog
in the sample.txt
file to mouse
in the new.txt
file.
First, our sample.txt
file:
~$: cat sample.txt The quick brown fox jumped over the lazy dog .
Now to use the stream editor sed
to change dog
to mouse
.
~$ cat sample.txt |sed s/dog/mouse/ The quick brown fox jumped over the lazy mouse .
Notice the command output content of the file sample.txt
, but we pipe it into the sed
command. We then instruct sed
to substitute s
dog
with mouse
, which looks as such s/dog/mouse/
. We can also do this without the cat
command as sed
is clever enough to read files without assistance from a cat
. Also, note that we have not yet modified anything.
~$ sed s/dog/mouse/ sample.txt The quick brown fox jumped over the lazy mouse .
Now we will redirect the output from sed
to new.txt
.
~$ sed s/dog/mouse/ sample.txt >new.txt ~$ cat new.txt The quick brown fox jumped over the lazy mouse .
Check content of sample.txt
~$ cat sample.txt The quick brown fox jumped over the lazy dog .
Now you can see, sed
stood in the data stream between sample.txt
and new.txt
and changed dog
to mouse
. In general, it is better to quote your sed
expression like this:
sed 's/dog/mouse/'
SAMPLE
~$ echo "dog"
Output
dog
~$ echo "dog" |sed 's/dog/mouse/'
Output
mouse
Let’s be creative
~$ echo "hounddog" |sed 's/dog/mouse/'
Output
houndmouse
Another important concept is that sed
is line oriented. Suppose you have the input file:
one two three, one two three four three two one one hundred
and you used the command
~$ sed 's/one/ONE/' <file
The output would be:
ONE two three, one two three four three two ONE ONE hundred
Note that this changed one
to ONE
once on each line. The first line had one
twice, but only the first occurrence was changed. That’s the default behaviour. If you want something different, you’ll have to use some of the options that are available.
Let’s return to the command: sed 's/one/ONE/'
There’re four parts to this substitute command:
s
Substitute command
/../../
Delimiter
one
Regular Expression Pattern Search Pattern
ONE
Replacement string
The search pattern is on the left-hand side and the replacement string is on the right-hand side. The search pattern may be matched using REGULAR EXPRESSIONS, just as we did with the grep
or egrep
command.
The character after the s
is the delimiter. It is conventionally a slash because this is what ed
, more
, and vi
use. It can be anything you want. If you want to change a pathname that contains a slash – say /usr/local/bin to /common/bin
– you could use the backslash to quote the slash as below:
sed 's/\/usr\/local\/bin/\/common\/bin/'
but it is easier to read if you use an underline instead of a slash as a delimiter:
sed 's_/usr/local/bin_/common/bin_'
Use any delimiter you like, as long as it does not occur in the search string. Always remember that you need 3 delimiters.
There are many, many more options we won’t mention here
The sed
command has many more options and is incredibly powerful. As this is a Linux Basics series, we will not delve into the magic of sed
any further in this post.
However, if you’re looking for a deeper understanding, one of the best comprehensive guides to sed
is by Bruce Barnet at grymoire.com.
Happy Hosting!