Linux Command Line Cheat Sheet: Awk Checksums Cut File Grep Head Sed Sor T WC XXD
Linux Command Line Cheat Sheet: Awk Checksums Cut File Grep Head Sed Sor T WC XXD
Abstract
The following examples may be typed in the terminal, but copy/paste will work fine (be sure
to omit the prompt).
Many of these examples will use the "cat example.txt | command" syntax. This is safer than
the equivalent syntax of "command < example.txt".
Why? Most everyone learning the Unix/Linux commandline has accidentally reversed the "
<" sign (read) with the ">" sign (write), accidentally overwriting a file. The syntax of "cat
example.txt | command" is therefore safer. Please feel free to use whatever syntax you are
most comfortable with.
On a related note, "There is more than one way to do it," as Larry Wall once said. You may
come up with different ways to perform the following, and perhaps better ways as well. Feel
free to share your CLI Kung Fu with us for possible inclusion!
Where to Acquire
These tools are installed natively in most Unix/Linux distributions, as well as OS X.
Examples/Use Case
awk
checksums
cut
file
grep
head
sed
sort
wc
xxd
awk
Print the length of each line of a file (/etc/passwd in this case), followed by the line itself:
1/7
Print the 2nd field from a file using the string 'Mozilla/' as a delimiter:
checksums
$ md5sum /etc/passwd
Generate the SHA1 checksum of a file. The three following commands are equivalent:
$ sha1sum /etc/passwd
$ shasum /etc/passwd
$ shasum -a1 /etc/passwd
cut
Cut the 2nd field from a file, using the space as a delimiter:
2/7
$ cat /var/log/dpkg.log | cut -d' ' -f2
Cut the 6th field from a file, using the colon as a delimiter:
Cut the 2nd and 3rd field from a file, use the comma as a delimiter:
Cut beginning at the 7th field, to end of line, using the space as a delimiter:
Cut the 6th field, using the double-quote (") as a delimiter, and escaping it to treat it as a
literal character:
file
$ file /usr/local/bin/*
grep
3/7
$ grep bash /etc/passwd
Search for lines that do not contain the string "bash", case insensitive:
Search for lines containing the string "root", case sensitive, plus print the next 5 lines:
head
$ head -n 10 /etc/passwd
sed
grep for lines containing "Mozilla", then delete all characters up to and including "Mozilla":
grep for lines containing "Mozilla", then delete all characters that precede "Mozilla":
4/7
$ grep Mozilla /var/log/apache2/access.log | sed
"s/^.*Mozilla/Mozilla/g"
sort
The following examples will run strings on a file, search for user-agent (ignore case), and use
various sort options
Sort and unique lines. The two following sets of commands are equivalent:
Get a numeric count of each unique entry, perform a numeric sort of that count:
Sort and unique lines, print the length of each unique line followed by the line itself,
perform a reverse numeric sort of that count:
5/7
wc
Determine number of lines in a file (the flag is the letter "ell", not the number one):
$ wc -l /etc/passwd
xxd
xxd creates a hexdump, or converts a hexdump into binary. A lot of malware hex-encodes
web traffic or malicious payloads (such as DOS executables) in order to avoid signature
matching. Useful hex patterns to look for are 4d5a90 (the magic bytes for a DOS
executable: "MZ<90>"), and "DOS mode" (444f53206d6f6465, see commands below).
Convert the string "DOS mode" to hex, grouped in sets of 4 hex characters (default):
Use sed to remove the percent signs from the percent-encoded hex string
"%63%67%69%2D%62%69%6E", then translate to binary:
6/7
Additional Info
A printable PDF version of this cheatsheet is available here:
LinuxCLI
7/7