Hack Like a Pro: Linux Basics for the Aspiring Hacker, Part 10 (Manipulating Text)

Linux Basics for the Aspiring Hacker, Part 10 (Manipulating Text)

Hack Like a Pro: Linux Basics for the Aspiring Hacker, Part 10 (Manipulating Text)

Welcome back, my aspiring hackers!

As mentioned several times in previous Linux tutorials, nearly everything in Linux is a file, and very often they are text files. For instance, all of the configuration files in Linux are text files. To reconfigure an application in Linux, we simply need to open the configuration file, change the text file, re-save, and then restart the application and our reconfiguration is applied.

With so many text files, manipulating text becomes crucial in managing Linux and Linux applications. In this tutorial, we'll look at several of the commands and techniques for manipulating text in Linux. For demonstration purposes, we'll use files from the world's best NIDS, Snort.

Step 1: Cat That File

As demonstrated in an earlier tutorial, cat is probably the most basic text display command. Let's cat the Snort config file found in /etc/snort.

  • cat /etc/snort/snort.conf

As you can see, the snort.conf is displayed on our screen until it comes to the end of the file. Not the most convenient way to work with this file.

Step 2: Take the Head

If we just want to view the beginning of a file, we can use the head command. This command displays the first 10 lines of a file, by default.

  • head /etc/snort/snort.conf

If we want to see more or less than the default 10 lines, we can tell head how many lines we want to see by putting the number of lines we want to see (with the - switch) between the command and the file name.

  • head -30 /etc/snort/snort.conf

Here we can see that only the first 30 lines of snort.conf are displayed.

Step 3: Grab That Tail

Similar to the head command, we view the last lines of a file by using the tail command. Let's use it on the snort.conf.

  • tail /etc/snort/snort.conf

Notice that it displays some of the last "includes" of the rules files, but not all of them. Let's now see if we can display all the rule "includes" by grabbing the last 40 lines of the snort.conf.

  • tail -40 /etc/snort/snort.conf

Now we can view nearly all the rule includes all on one screen.

Step 4: Numbering Those Lines

Sometimes—especially with very long files—we may want the file displayed with line numbers. This is probably the case with the snort.conf, as it has 838 lines. This makes it easier to reference changes and come back to the same place within a file. To display a file with line number, we simply type:

  • nl snort.conf

Note that each line now has a number making referencing much easier.

Step 5: I Grep That

After cat, grep is probably the most widely used text manipulation command. It's a filtering command; in other words, it enables us to filter the content of a file for display. If for instance, we wanted to see all the instances of where the word "database" occurs in our snort.conf file, we could ask cat to only display those lines where it occurs by typing:

  • cat /etc/snort/ snort.conf | grep database

This command will first grab the snort.conf and then "pipe" it (|) to grep which will take it as input and then look for the occurrences of the word "database" and only display those lines. Grep is a powerful and essential command for working in Linux as it can save us hours searching for every occurrence of a word or command.

Step 6: I Sed That Works

The sed command essentially allows us to search for occurrences of a word or text pattern and then do some work on it. The name comes from the concept of a stream editor and is a contraction of those two words. In its most basic form, sed operates like the find and replace function in Windows. Let's search for the word "mysql" in the snort.conf file using grep.

  • cat /etc/snort/snort.conf | grep mysql

We can see that the grep command found five occurrences of the word mysql.

Let's say we want sed to replace every occurrence of mysql and with MySQL (remember, Linux is case sensitive) and then save the new file to snort2.conf. We could do this by typing:

  • sed s/mysql/MySQL/g snort.conf > snort2.conf

This command says, "search (s) for the word mysql and replace it with the word MySQL globally (i.e. wherever you find it in the file)."

Now, when we grep snort2.conf for mysql, we see that none were found and when we grep for MySQL, we find five occurrences of MySQL.

  • cat /etc/snort/snort.conf | grep MySQL

If we just want to replace only the first occurrence of the word mysql, we could leave out the trailing g and it would only replace the first occurrence.

  • sed s/mysql/MySQL/ snort.conf > snort2.conf

The sed command can also be used to find and replace any specific occurrence of a word. For instance, if I want to only replace the third occurrence of the word mysql, I can simply place the number of the occurrence at the end of the command and sed will only replace the third occurrence of the word "mysql" with "MySQL".

  • sed s/mysql/MySQL/3 snort.conf > snort2.conf

Stay Tuned for More

That's it for this lesson, but there are many more to come, so check out our section on learning Linux basics to stay up to date. If you have any questions on using sed, cat, head, tail, nl, or grep, ask away in the comments below. You can also visit the Null Byte forum for help on unrelated matters.

Original penguin eye image via Shutterstock

21 Comments

Hey i am new at this , i dont now how to do anytihng yet :) and probably i am using window not linux, i watched all your videos so can you help me on windows i downloaded metasploit aswell.

Can you have grep display the number of what line the instances of the word we search for occur in?

or you could do something like cat snort.conf | grep -n mysql
-n makes grep display line numbers

Yes, use nl and then grep. For instance,

nl snort.conf | grep mysql

OTW

I don't really know how to word this though so it may be a bit long of a comment here, but, how do you... put grep'd text side by side i guess.

Like say I have a list called A and looks like this:

A.txt
A B C
A D G
G F A
J K L
P P W

And I grep'd it for A's such that:

cat A.txt | grep A

A B C
A D G
G F A

And I have another list called 1 and well let's speed it up and say I grep'd it for 1's so it looks like this:

cat 1.txt | grep 1

1 5 4
1 3 2
1 6 6

The question is, how would I combine them such that the output would look like this:

A B C 1 5 4
A D G 1 3 2
G F A 1 6 6

Or even nicer if possible, like this with the colon.

A B C : 1 5 4
A D G : 1 3 2
G F A : 1 6 6

Eight:

Consider using a For each loop. First, grep the first file and direct (>) to a file and then grep the second file and append (>>) the results to the first output. Do this line by line.

That's the general strategy I would use.

Hope this helps.

OTW

i am currently using kali-linux but all of your tutorials are with BT5, should i revert to bt5 as well?

Once more, as I read through your articles, I try to do them myself, step by step, in the beginning of Step 4, after typing in (and variations of)

'nl snort.conf'
I discovered that similarly to earlier mentions, the command that worked was
'nl /etc/snort/.conf'

Similarly to this, in step 6, you have written the command of
'sed s/mysql/MySQL/g snort.conf > snort2.conf'
when in my practise, that command does not work, whereas
'sed s/mysql/MySQL/g /etc/snort/snort.conf > /etc/snort/snort/snort2.conf'
is accepted, but does not appear to change
'mysql'
to
'MySQL'

It would appear, to me, that the reason is the conflict between the written command and the command in the screenshot where the written command is searching in

'/etc/snort/snort.conf'
and the screenshot searches in
'/etc/snort/snort2.conf'
Where the alteration seems to have been made by the earlier written command.

If there is something that I am missing then I apologise, and I am interested in knowing what it is and how to fix this.

Furthermore my curiosity has piqued to which possible shortenings would work.

Moreover, in the beginning of Step 5, there is a space between
'snort/'
and
'snort.conf'
This seems to stop the command from working.

Thank you,

Nemesis1512.

Nemesis:

As for the first, 'nl snort.conf', it will work as long as you are in the /etc/snort directory. The same is true for your second error.

It is important to note what directory you are in when executing a command. If I am in the directory when the object file resides, I don't have to give the full path (snort.conf vs. /etc/snort/snort.conf).

As for your third error in Step 5, you are indeed correct, there should not be a space there.

OTW

Thank you very much for taking your time to explain the reasons for my uncertainties, this greatly helps my learning in the ways of Linux of which I am slowly getting more accustomed to.

Thank you again, OTW,

Nemesis1512.

Nemesis:

You are very welcome.

Hope to hear more from you in the future.

OTW

Sometimes the spacing on these commands gets "botched" in the editing process. Try to stick with the commands in the screenshots, if there is a conflict.

OTW

So then I'm new to this, has any one got any thing easy and fun for me I can do please

Thanks a lot for these tutorials. I have learned a lot from them.

Good tutorial. Thanks a lot!

If I had a text file with the following contents,
seed
aspseed
germinating seeds
seeder
another seed.

Is there any way to use sed command to replace only the word "seed" with something else?( ie. the "seed" in the 1st line and 5th line)

2 years have been now since these basic starting tutorials were posted, many views, many minds past by here, still passing, and as one of them, i can't help but wonder where are they now, what kind of level have they achieved, what kind of change happened to their thinking, and did they find the answers they were looking for, and most importantly will i ever be able to do the same.

special thanks to the author "OTW".

Share Your Thoughts

  • Hot
  • Latest