Hack Like a Pro: Linux Basics for the Aspiring Hacker, Part 10 (Manipulating Text)

Welcome back, my aspiring hackers!

As mentioned several times in previous Linux tutorials, nearly everything in Linux is a file, and very often they are text files. For instance, all of the configuration files in Linux are text files. To reconfigure an application in Linux, we simply need to open the configuration file, change the text file, re-save, and then restart the application and our reconfiguration is applied.

With so many text files, manipulating text becomes crucial in managing Linux and Linux applications. In this tutorial, we'll look at several of the commands and techniques for manipulating text in Linux. For demonstration purposes, we'll use files from the world's best NIDS, Snort.

Step 1: Cat That File

As demonstrated in an earlier tutorial, cat is probably the most basic text display command. Let's cat the Snort config file found in /etc/snort.

• cat /etc/snort/snort.conf

As you can see, the snort.conf is displayed on our screen until it comes to the end of the file. Not the most convenient way to work with this file.

Step 2: Take the Head

If we just want to view the beginning of a file, we can use the head command. This command displays the first 10 lines of a file, by default.

• head /etc/snort/snort.conf

If we want to see more or less than the default 10 lines, we can tell head how many lines we want to see by putting the number of lines we want to see (with the - switch) between the command and the file name.

• head -30 /etc/snort/snort.conf

Here we can see that only the first 30 lines of snort.conf are displayed.

Step 3: Grab That Tail

Similar to the head command, we view the last lines of a file by using the tail command. Let's use it on the snort.conf.

• tail /etc/snort/snort.conf

Notice that it displays some of the last "includes" of the rules files, but not all of them. Let's now see if we can display all the rule "includes" by grabbing the last 40 lines of the snort.conf.

• tail -40 /etc/snort/snort.conf

Now we can view nearly all the rule includes all on one screen.

Step 4: Numbering Those Lines

Sometimes—especially with very long files—we may want the file displayed with line numbers. This is probably the case with the snort.conf, as it has 838 lines. This makes it easier to reference changes and come back to the same place within a file. To display a file with line number, we simply type:

• nl snort.conf

Note that each line now has a number making referencing much easier.

Step 5: I Grep That

After cat, grep is probably the most widely used text manipulation command. It's a filtering command; in other words, it enables us to filter the content of a file for display. If for instance, we wanted to see all the instances of where the word "database" occurs in our snort.conf file, we could ask cat to only display those lines where it occurs by typing:

• cat /etc/snort/ snort.conf | grep database

This command will first grab the snort.conf and then "pipe" it (|) to grep which will take it as input and then look for the occurrences of the word "database" and only display those lines. Grep is a powerful and essential command for working in Linux as it can save us hours searching for every occurrence of a word or command.

Step 6: I Sed That Works

The sed command essentially allows us to search for occurrences of a word or text pattern and then do some work on it. The name comes from the concept of a stream editor and is a contraction of those two words. In its most basic form, sed operates like the find and replace function in Windows. Let's search for the word "mysql" in the snort.conf file using grep.

• cat /etc/snort/snort.conf | grep mysql

We can see that the grep command found five occurrences of the word mysql.

Let's say we want sed to replace every occurrence of mysql and with MySQL (remember, Linux is case sensitive) and then save the new file to snort2.conf. We could do this by typing:

• sed s/mysql/MySQL/g snort.conf > snort2.conf

This command says, "search (s) for the word mysql and replace it with the word MySQL globally (i.e. wherever you find it in the file)."

Now, when we grep snort2.conf for mysql, we see that none were found and when we grep for MySQL, we find five occurrences of MySQL.

• cat /etc/snort/snort.conf | grep MySQL

If we just want to replace only the first occurrence of the word mysql, we could leave out the trailing g and it would only replace the first occurrence.

• sed s/mysql/MySQL/ snort.conf > snort2.conf

The sed command can also be used to find and replace any specific occurrence of a word. For instance, if I want to only replace the third occurrence of the word mysql, I can simply place the number of the occurrence at the end of the command and sed will only replace the third occurrence of the word "mysql" with "MySQL".

• sed s/mysql/MySQL/3 snort.conf > snort2.conf

Stay Tuned for More

That's it for this lesson, but there are many more to come, so check out our section on learning Linux basics to stay up to date. If you have any questions on using sed, cat, head, tail, nl, or grep, ask away in the comments below. You can also visit the Null Byte forum for help on unrelated matters.