05-Searching and Manipulating Files

more
more – To display file contents one screen at a
time
$more file1
[SpaceBar] – To display the next screen
[Enter] - advance 1 line
q - quit
 more
[alex@server1 ~]$ more /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
gopher:x:13:30:gopher:/var/gopher:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
 less
less – It is similar to more , but which allows
backward movement in the file as well as forward
movement.

$less file1
[SpaceBar] – To display the next screen
[Enter] - advance 1 line
b - go backwards
q - quit
 less
[alex@server1 ~]$ less /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin
games:x:12:100:games:/usr/games:/sbin/nologin
gopher:x:13:30:gopher:/var/gopher:/sbin/nologin
ftp:x:14:50:FTP User:/var/ftp:/sbin/nologin
nobody:x:99:99:Nobody:/:/sbin/nologin
usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
avahi-autoipd:x:170:170:Avahi IPv4LL Stack:/var/lib/avahi-autoipd:/sb
 head
head – To display lines from the beginning of a
file ,the default is 10 lines.
$head /etc/passwd
displays the first 10 lines of /etc/passwd

$head -5 /etc/passwd
displays the first 5 lines of /etc/passwd
 head
[alex@server1 ~]$ head -5 /etc/passwd
root:x:0:0:root:/root:/bin/bash
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
[alex@server1 ~]$
 tail
tail – To display lines from the end of a file, the
default is 10 lines.
$tail /etc/passwd
displays the last 10 lines of /etc/passwd

$tail -5 /etc/passwd
displays the last 5 lines of /etc/passwd
 tail
[alex@server1 ~]$ tail -5 /etc/passwd
gdm:x:42:42::/var/lib/gdm:/sbin/nologin
alex:x:500:500:alex:/home/alex:/bin/bash
user1:x:501:501::/home/user1:/bin/bash
user2:x:502:502::/home/user2:/bin/bash
user3:x:503:503::/home/user3:/bin/bash
[alex@server1 ~]$
 grep

The grep command searches the contents of a

file for lines that contain a certain pattern.
Global Regular Expression Point

[alex@server1 ~]$ grep alex /etc/passwd

alex:x:500:500:alex:/home/alex:/bin/bash
[alex@server1 ~]$
 Redirecting output

When you redirect standard output, the data that

would normally appear on the screen is stored in a
file instead. Redirect output to a file using > or >>
$ ls -l > file1
stores the output from ls in file1; if file1 already exists, it
is overwritten
$ ls -l >> file1
appends the output from ls to the file file1
 Redirecting output
[alex@Server1 ~]$ cat computers
HP Laptop
Dell Laptop
IBM Laptop
HP Server
Dell Server
IBM Server
HP Printer
Canon Printer
Brother Printer

[alex@Server1 ~]$
 Redirecting output
[alex@Server1 ~]$ grep HP computers >list1
[alex@Server1 ~]$ cat list1
HP Laptop
HP Server
HP Printer
[alex@Server1 ~]$
Redirecting output
[alex@Server1 ~]$ grep Dell computers >>list1
[alex@Server1 ~]$ cat list1
HP Laptop
HP Server
HP Printer
Dell Laptop
Dell Server
[alex@Server1 ~]$
 Pipes

A pipe ( | ) takes the output from one command

and uses it as input to the next command.
$ ls  -l  /etc  |more

display a long listing of the /etc directory and pipe the

output through the more command so that it is
displayed one screen at a time.
 
Remember:
to send the output of a command to a file, use
redirection (> or >>)
to send the output of a command to another command,
use piping ( | )
  
 Pipes
[alex@server1 ~]$ ls -l /etc |more
total 2268
drwxr-xr-x. 3 root root 4096 Mar 27 05:44 abrt
drwxr-xr-x. 4 root root 4096 Mar 27 05:47 acpi
-rw-r--r--. 1 root root 46 Sep 23 03:53 adjtime
-rw-r--r--. 1 root root 1512 Jun 29 2010 aliases
-rw-r-----. 1 root smmsp 12288 Mar 27 05:47 aliases.db
drwxr-xr-x. 2 root root 4096 Mar 27 05:51 alsa
drwxr-xr-x. 2 root root 4096 Mar 27 05:51 alternatives
-rw-r--r--. 1 root root 541 Aug 13 2010 anacrontab
-rw-r--r--. 1 root root 245 Oct 6 2010 anthy-conf
-rw-r--r--. 1 root root 148 Sep 10 2008 asound.conf
-rw-------. 1 root root 1 Mar 15 2010 at.deny
drwxr-x---. 3 root root 4096 Mar 27 05:43 audisp
drwxr-x---. 2 root root 4096 Mar 27 05:43 audit
drwxr-xr-x. 4 root root 4096 Mar 27 05:46 avahi
drwxr-xr-x. 2 root root 4096 Mar 27 05:47 bash_completion.d
-rw-r--r--. 1 root root 2615 Jun 29 2010 bashrc
drwxr-xr-x. 2 root root 4096 Oct 11 2010 blkid
drwxr-xr-x. 2 root root 4096 Mar 27 05:50 bluetooth
drwxr-xr-x. 2 root root 4096 Mar 27 05:42 bonobo-activation
--More--
 Pipes
Commands that are used in pipelines to modify
the output from other commands are being used
as filters
[alex@server1 ~]$ ls -l |grep list
-rw-rw-r--. 1 alex alex 21 Sep 23 05:11 list1
-rw-rw-r--. 1 alex alex 10 Sep 23 05:10 list2
[alex@server1 ~]$
 Pipes
[alex@server1 ~]$ ls -l
total 56
drwxr-xr-x. 2 alex alex 4096 Mar 27 06:03 Desktop
drwxr-xr-x. 2 alex alex 4096 Mar 27 06:01 Documents
drwxr-xr-x. 2 alex alex 4096 Mar 27 06:01 Downloads
-rw-rw-r--. 1 alex alex 654 Sep 23 05:06 file1
drwxrwxr-x. 2 alex alex 4096 Sep 19 11:15 folder1
drwxrwxr-x. 6 alex alex 4096 Sep 22 15:01 Foods
-rw-rw-r--. 1 alex alex 21 Sep 23 05:11 list1
-rw-rw-r--. 1 alex alex 10 Sep 23 05:10 list2
drwxr-xr-x. 2 alex alex 4096 Mar 27 06:01 Music
drwxr-xr-x. 2 alex alex 4096 Mar 27 06:01 Pictures
drwxr-xr-x. 2 alex alex 4096 Mar 27 06:01 Public
drwxr-xr-x. 2 alex alex 4096 Mar 27 06:01 Templates
drwxrwxr-x. 4 alex alex 4096 Sep 22 12:07 Trips
drwxr-xr-x. 2 alex alex 4096 Mar 27 06:01 Videos
[alex@server1 ~]$ ls -l |grep list
-rw-rw-r--. 1 alex alex 21 Sep 23 05:11 list1
-rw-rw-r--. 1 alex alex 10 Sep 23 05:10 list2
[alex@server1 ~]$
 find
The find command is used to search files and directories
$ find [path] -type fd -name “pattern”

[alex@server1 ~]$ find /home/alex -type f -name "list*"

/home/alex/list2
/home/alex/list1
/home/alex/Foods/October/list_Octfoods
/home/alex/Foods/September/list_SepFoods
/home/alex/Foods/list_foods
[alex@server1 ~]$

The search would be started in the directory /home/alex and

carry on within that directory and its subdirectories only.
 find
[alex@server1 ~]$ find ~/Foods/ -type f -name "list*"
/home/alex/Foods/October/list_Octfoods
/home/alex/Foods/September/list_SepFoods
/home/alex/Foods/list_foods
If you just want to search from the current directory and its
subdirectories replace the path with a '.‘
[alex@server1 Foods]$ find ./ -type f -name "list*"
./October/list_Octfoods
./September/list_SepFoods
./list_foods
 find
The -name option is case sensitive
To search names in lowercase and uppercase , we
should use the option -iname

[alex@server1 Foods]$ find ./ -type f -iname "list*"

./List_Fruits
./October/list_Octfoods
./September/list_SepFoods
./list_foods
 find
You can specify as many places to search as you wish:
[alex@server1 Foods]$ find ./September ./October -type f -iname "list*"

./September/list_SepFoods
./October/list_Octfoods

If no path is given, find looks in the current directory and its
subdirectories

[alex@server1 Foods]$ find -type f -name "list*“

./October/list_Octfoods
./September/list_SepFoods
./list_foods
 find
As the command find scans through the directory tree,
there will be directories which you do not have permissions
to search. The find command will not scan these directories
but will display permission denied messages on standard
error.
To suppress the permission denied messages, redirect
standard error to /dev/null.
$ find […][…][…] 2> /dev/null
2: Standard Error Ouput (0: Standard Input, 1: Standard output)
/dev/null: the null device is a special file that discards all
data written to it.
 find
Find all files under the /etc directory with the filename

passwd
[alex@server1 ~]$ find /etc -type f -name "passwd"
find: `/etc/lvm': Permission denied
/etc/passwd
/etc/pam.d/passwd
find: `/etc/pki/CA/private': Permission denied
find: `/etc/sssd': Permission denied
find: `/etc/cups/ssl': Permission denied
find: `/etc/audisp': Permission denied
find: `/etc/ntp/crypto': Permission denied
find: `/etc/selinux/targeted/modules/active': Permission denied
find: `/etc/audit': Permission denied
find: `/etc/dhcp': Permission denied
find: `/etc/polkit-1/localauthority': Permission denied
find: `/etc/sudoers.d': Permission denied
[alex@server1 ~]$
 find
Find all files under the /etc directory with the filename
passwd and suppress the display of the "permission
denied" messages.
[alex@server1 ~]$ find /etc -type f -name "passwd" 2> /dev/null
/etc/passwd
/etc/pam.d/passwd
[alex@server1 ~]$
 find
To redirect the output to a file use
an other >
[alex@server1 ~]$ find /etc -type f -name "passwd" 2> /dev/null >pass

/etc/passwd
/etc/pam.d/passwd
[alex@server1 ~]$
 find
Find is an excellent tool for generating reports on basic
information regarding files or directories but what if you
want more than just reports?
Execute option

$ find [path] -type fd -name “pattern” -exec comand {} \;

 find
[alex@RH9Server alex]$ find ./ -type f -name "t*" -exec mv {} dir2 \;
[alex@server1 Foods]$ find ./ -type f -iname "list*"
./List_Fruits
./October/list_Octfoods
./September/list_SepFoods
./list_foods
[alex@server1 Foods]$ find ./ -type f -iname "list*" -exec cp {} December \;
[alex@server1 Foods]$ ls December/
Fruits_Dec List_Fruits list_SepFoods
list_foods list_Octfoods Vegetables_Dec
[alex@server1 Foods]$
 wc

Short for word count, wc displays

a count of lines, words, and
characters in a file.
$ wc [-c ] [-l] [-w] file
-c : to count characters
-l : to count lines
-w: to count words
 wc
[alex@server1]$ cat Vegetables
Carrots Celery
Cucumbers Potatoes
Tomatoes
[alex@server1]$ wc Vegetables
3 5 42 Vegetables
=================
3 = Lines
5 = Words
42 = Characters
 wc
[alex@server1 ~]$ who
alex pts/0 2012-09-23 04:55 (192.168.1.196)
alex pts/1 2012-09-23 09:18 (192.168.1.196)
user3 pts/2 2012-09-23 09:19 (192.168.1.196)
user1 pts/3 2012-09-23 09:20 (192.168.1.196)
user3 pts/4 2012-09-23 09:20 (192.168.1.196)
[alex@server1 ~]$ who |grep user3
user3 pts/2 2012-09-23 09:19 (192.168.1.196)
user3 pts/4 2012-09-23 09:20 (192.168.1.196)

[alex@server1 ~]$ who |grep user3 |wc -l

2
[alex@server1 ~]$
The user user3 is logged to the system from two

terminals.
 sort
The command sort sorts the lines in a text file

$ sort file

[alex@server1 ~]$ sort /etc/passwd |tail -5

usbmuxd:x:113:113:usbmuxd user:/:/sbin/nologin
user1:x:501:501::/home/user1:/bin/bash
user2:x:502:502::/home/user2:/bin/bash
user3:x:503:503::/home/user3:/bin/bash
uucp:x:10:14:uucp:/var/spool/uucp:/sbin/nologin
[alex@server1 ~]$
 sort
To sort in reverse order
$ sort – r file
[alex@server1 ~]$ sort -r /etc/passwd |tail -5
avahi:x:70:70:Avahi mDNS/DNS-SD
Stack:/var/run/avahi-daemon:/sbin/nologin
avahi-autoipd:x:170:170:Avahi IPv4LL
Stack:/var/lib/avahi-autoipd:/sbin/nologin
apache:x:48:48:Apache:/var/www:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
abrt:x:499:498::/etc/abrt:/sbin/nologin
[alex@server1 ~]$
 sort
sort assumes that the lines of a file are just words
separated by blanks
If you want to sort a file by column ,use the option -k
followed by the number of column
$ sort -k2 file1
Will sort file1 by the second column
If you want to sort a file by column in numerical
order ,use the option -k followed by the number of
column followed by n
$ sort -k2n file1
Will sort file1 by the second column in numerical order
 sort
[alex@server1 ~]$ cat studentslist
Fred Clark 10
Tony Harper 17
Richard Tremblay 25
Mary Remington 209
Carole Parizeau 3
[alex@server1 ~]$
 sort
[alex@server1 ~]$ sort studentslist >list1
[alex@server1 ~]$ cat list1
Carole Parizeau 3
Fred Clark 10
Mary Remington 209
Richard Tremblay 25
Tony Harper 17
[alex@server1 ~]$
 sort
[alex@server1 ~]$ sort -k2 studentslist >list2
[alex@server1 ~]$ cat list2
Fred Clark 10
Tony Harper 17
Carole Parizeau 3
Mary Remington 209
Richard Tremblay 25
[alex@server1 ~]$
 sort
[alex@server1 ~]$ sort -k3 studentslist >list3
[alex@server1 ~]$ cat list3
Fred Clark 10
Tony Harper 17
Mary Remington 209
Richard Tremblay 25
Carole Parizeau 3
[alex@server1 ~]$
 sort
[alex@server1 ~]$ sort -k3n studentslist >list4
[alex@server1 ~]$ cat list4
Carole Parizeau 3
Fred Clark 10
Tony Harper 17
Richard Tremblay 25
Mary Remington 209
[alex@server1 ~]$
 sed
The Unix utility sed (stream editor) is a text
editing tool that can be used to edit text
files without having to open them.
You can use the sed command to change
one string to another within a file

$ sed 's/oldstring/newstring/' file

In the quoted string, the "s" means

substitute,
 sed

[alex@server1 ~]$ cat file1

this text1
this text1
this text1
[alex@server1 ~]$ sed 's/text1/newtext/' file1
this newtext
this newtext
this newtext
 sed
To save the change you need to redirect the display to
a new file
$ sed 's/oldstring/newstring/' file >newfile

[alex@server1 ~]$ sed 's/text1/newtext/' file1 >file2

[alex@server1 ~]$ cat file2
this newtext
this newtext
this newtext
 sed
you can do multiple pattern matches and replacing
with sed .
[alex@server1 ~]$ sed 's/this/that/;s/new/old/' file2 >file3
[alex@server1 ~]$ cat file3
that oldtext
that oldtext
that oldtext
 sed
If a word you want to replace is repeated several
times in the same line
[alex@server1 ~]$ cat file1
this text1 text1 text1
this text1 file text1
this file text1 text1
this text1 text1 file
[alex@server1 ~]$ sed 's/text1/text2/' file1
this text2 text1 text1
this text2 file text1
this file text2 text1
this text2 text1 file
 sed
[alex@server1 ~]$ sed 's/text1/text2/g' file1
this text2 text2 text2
this text2 file text2
this file text2 text2
this text2 text2 file