Ost Unit II Material
Ost Unit II Material
2.1 INTRODUCTION
What is Linux
Linux is a powerful and flexible family of operating systems that are free
to use and share. It was created by a person named Linus Torvalds in
1991. What’s cool is that anyone can see how the system works because
its source code is open for everyone to explore and modify. This
openness encourages people from all over the world to work together and
make Linux better and better. Since its beginning, Linux has grown into
a stable and safe system used in many different things, like computers,
smartphones, and big supercomputers. It’s known for being efficient,
meaning it can do a lot of tasks quickly, and it’s also cost-effective,
which means it doesn’t cost a lot to use. Lots of people love Linux, and
they’re part of a big community where they share ideas and help each
other out. As technology keeps moving forward, Linux will keep
evolving and staying important in the world of computers.
Linux Distribution
c. File Systems
1. mount [option(s)] [<device>] mountpoint
This command can be used to mount any data media, such as hard
disks, CD-ROM drives, and other drives, to a directory of the Linux
file system.
-r mount read-only
-t filesystem
Specifies the file system. The most common are ext2 for Linux hard
disks, msdos for MS-DOS media, vfat for the Windows file system,
and iso9660 for CDs.
2. umount [option(s)] mountpoint
This command unmounts a mounted drive from the file system. To
prevent data loss, run this command before taking a removable data
medium from its drive. Normally, only root is allowed to run the
commands mount and umount. To enable other users to run these
commands, edit the /etc/fstab file to specify the option user for the
respective drive.
d. System Commands
System Information
df [option(s)] [directory]
The df (disk free) command, when used without any options,
displays information about the total disk space, the disk space
currently in use, and the free space on all the mounted drives. If a
directory is specified, the information is limited to the drive on which
that directory is located.
-H : shows the number of occupied blocks in gigabytes, megabytes,
or kilobytes — in human-readable format
-t : Type of file system (ext2, nfs, etc.)
du [option(s)] [path]
This command, when executed without any parameters, shows the
total disk space occupied by files and subdirectories in the current
directory.
-a : Displays the size of each individual file
-h : Output in human-readable form
-s : Displays only the calculated total size
free [option(s)]
The command free displays information about RAM and swap space
usage, showing the total and the used amount in both categories.
-b : Output in bytes
-k : Output in kilobytes
-m : Output in megabytes
date [option(s)]
This simple program displays the current system time. If run as root,
it can also be used to change the system time. Details about the
program are available in date.
Processes
top [options(s)]
top provides a quick overview of the currently running processes.
Press H to access a page that briefly explains the main options to
customize the program.
ps [option(s)] [process ID]
If run without any options, this command displays a table of all your
own programs or processes — those you started. The options for this
command are not preceded by hyphen.
aux
Displays a detailed list of all processes, independent of the owner.
kill [option(s)] process ID
Unfortunately, sometimes a program cannot be terminated in the
normal way. However, in most cases, you should still be able to stop
such a runaway program by executing the kill command, specifying
the respective process ID (see top and ps).
kill sends a TERM signal that instructs the program to shut itself
down. If this does not help, the following parameter can be used:
-9
Sends a KILL signal instead of a TERM signal, with which the
process really is annihilated by the operating system. This brings the
specific processes to an end in almost all cases.
killall [option(s)] processname
This command is similar to kill, but uses the process name (instead
of the process ID) as an argument, causing all processes with that
name to be killed.
Network
ping [option(s)] host name|IP address
The ping command is the standard tool for testing the basic
functionality of TCP/IP networks. It sends a small data packet to the
destination host, requesting an immediate reply. If this works, ping
displays a message to that effect, which indicates that the network
link is basically functioning.
-c : number Determines the total number of packages to send and
ends after they have been dispatched. By default, there is no
limitation set.
-f flood ping: sends as many data packages as possible. A popular
means, reserved to root, to test networks.
-i :value Specifies the interval between two data packages in seconds.
Default: one second
nslookup
The Domain Name System resolves domain names to IP addresses.
With this tool, send queries to information servers (DNS servers).
telnet [option(s)] host name or IP address
Telnet is actually an Internet protocol that enables you to work on
remote hosts across a network. telnet is also the name of a Linux
program that uses this protocol to enable operations on remote
computers. Warning
Miscellaneous
passwd [option(s)] [username]
Users may change their own passwords at any time using this
command. Furthermore, the administrator root can use the command
to change the password of any user on the system.
su [option(s)] [username]
The su command makes it possible to log in under a different user
name from a running session. When using the command without
specifying a user name, you will be prompted for the root password.
Specify a user name and the corresponding password to use the
environment of the respective user. The password is not required
from root, as root is authorized to assume the identity of any user.
halt [option(s)]
To avoid loss of data, you should always use this program to shut
down your system.
reboot [option(s)]
Does the same as halt with the difference that the system performs
an immediate reboot.
clear
This command cleans up the visible area of the console. It has no
options.
Directory Structure
The directories help us to store the files and locate them when we need
them. Also, directories are called folders as they can be assumed of as
folders where files reside in the form of a physical desktop analogy.
Directories can be organized in a tree-like hierarchy in Linux and several
other operating systems.
The directory structure of Linux is well-documented and defined in the
Linux FHS (Filesystem Hierarchy Standard). Referencing those directories
if accessing them is achieved via the sequentially deeper names of the
directory linked by '/' forward slash like /var/spool/mail and /var/log.
These are known as paths.
The below table gives a very short standard, defined, and well-known top-
level Linux directory list and their purposes:
o / (root filesystem): It is the top-level filesystem directory. It must
include every file needed to boot the Linux system before another
filesystem is mounted. Every other filesystem is mounted on a well-
defined and standard mount point because of the root filesystem
directories after the system is started.
o /boot: It includes the static kernel and bootloader configuration and
executable files needed to start a Linux computer.
o /bin: This directory includes user executable files.
o /dev: It includes the device file for all hardware devices connected to
the system. These aren't device drivers; instead, they are files that
indicate all devices on the system and provide access to these
devices.
o /etc: It includes the local system configuration files for the host
system.
o /lib: It includes shared library files that are needed to start the
system.
o /home: The home directory storage is available for user files. All
users have a subdirectory inside /home.
o /mnt: It is a temporary mount point for basic filesystems that can be
used at the time when the administrator is working or repairing a
filesystem.
o /media: A place for mounting external removable media devices like
USB thumb drives that might be linked to the host.
o /opt: It contains optional files like vendor supplied application
programs that must be placed here.
o /root: It's the home directory for a root user. Keep in mind that it's
not the '/' (root) file system.
o /tmp: It is a temporary directory used by the OS and several
programs for storing temporary files. Also, users may temporarily
store files here. Remember that files may be removed without prior
notice at any time in this directory.
o /sbin: These are system binary files. They are executables utilized
for system administration.
o /usr: They are read-only and shareable files, including executable
libraries and binaries, man files, and several documentation types.
o /var: Here, variable data files are saved. It can contain things such as
MySQL, log files, other database files, email inboxes, web server
data files, and much more.
Each file system (used in the first sense) contains a control block,
which holds information about that file system. The other blocks in the
file ystem are inodes, which contain information about individual files,
and data blocks, which contain the information stored in the individual
files.
o the Linux kernel, however, the file system is flat. That is, it does not (1)
have a hierarchical structure, (2) differentiate between directories, files or
programs or (3) identify files by names. Instead, the kernel uses inodes to
represent each file.
2.4.Vi Editor
Vi is a command line text editor. As you would be quite aware now, the
command line is quite a different environment to your GUI. It's a single
window with text input and output only. Vi has been designed to work
within these limitations and many would argue, is actually quite powerful
as a result. Vi is intended as a plain text editor (similar to Notepad on
Windows, or Textedit on Mac) as opposed to a word processing suite such
as Word or Pages. It does, however have a lot more power compared to
Notepad or Textedit.
There are two modes in Vi. Insert (or Input) mode and Edit mode. In
input mode you may input or enter content into the file. In edit mode you
can move around the file, perform actions such as deleting, copying,
search and replace, saving etc.
When we run vi we normally issue it with a single command line argument
which is the file you would like to edit.
vi <file>
When you run this command it opens up the file.
You always start off in edit mode so the first thing we are going to do
is switch to insert mode by pressing i. You can tell when you are in
insert mode as the bottom left corner will tell you.
ZZ (Note: capitals) - Save and exit
:q! - discard all changes, since the last save, and exit
:w - save file but don't exit
:wq - again, save and exit
Below are some of the many commands you may enter to move around the
file. Have a play with them and see how they work.
Arrow keys - move the cursor around
j, k, h, l - move the cursor down, up, left and right (similar to the
arrow keys)
^ (caret) - move cursor to beginning of current line
$ - move cursor to end of the current line
nG - move to the nth line (eg 5G moves to 5th line)
G - move to the last line
w - move to the beginning of the next word
nw - move forward n word (eg 2w moves two words forwards)
b - move to the beginning of the previous word
nb - move back n word
{ - move backward one paragraph
} - move forward one paragraph
Below are some of the many ways in which we may delete content within
vi. Have a play with them now. (also check out the section below on
undoing so that you can undo your deletes.)
x - delete a single character
nx - delete n characters (eg 5x deletes five characters)
dd - delete the current line
dn - d followed by a movement command. Delete to where the
movement command would have taken you. (eg d5w means delete 5
words)
Undoing changes in vi is fairly easy. It is the character u.
u - Undo the last action (you may keep pressing u to keep undoing)
U (Note: capital) - Undo all changes to the current line
I need to do this for each of my partitions, but not for /dev/sda4 (my
extended partition). Linux supports types of file systems other than ext2.
You can find out what kinds your kernel supports by looking in:
/usr/src/linux/include/linux/fs.h
The most common file systems can be made with programs in /sbin that
start with "mk" like mkfs.msdos and mke2fs.
-t ext2
File system type. Other types you are likely to use are:
msdos (DOS)
hfs (mac)
iso9660 (CDROM)
/dev/sda1
/dev/cdrom (CDROM)
/opt
mount point. This is where you want to "see" your partition. When
you type ls /opt, you can see what is in /dev/sda1. If there are already
some directories and/or files under /opt, they will be invisible after
this mount command.
there are about 12000 files on / and about 44000 files on /var. At a block
size of 1 KB, about 6+22 = 28 MB of disk space are lost in the tail blocks
of files. Had I chosen a block size of 4 KB, I had lost 4 times this space.
Data transfer is faster for large contiguous chunks of data, though. That's
why ext2 tries to preallocate space in units of 8 contigous blocks for
growing files. Unused preallocation is released when the file is closed, so
no space is wasted.
Noncontiguous placement of blocks in a file is bad for performance, since
files are often accessed in a sequential manner. It forces the operating
system to split a disk access and the disk to move the head. This is called
"external fragmentation" or simply "fragmentation" and is a common
problem with MS-DOS file systems. In conjunction with the abysmal
buffer cache used by MS-DOS, the effects of file fragmentation on
performance are very noticeable. DOS users are accustomed to defragging
their disks every few weeks and some have even developed some
ritualistic beliefs regarding defragmentation.
None of these habits should be carried over to Linux and ext2. Linux
native file systems do not need defragmentation under normal use and this
includes any condition with at least 5% of free space on a disk. There is a
defragmentation tool for ext2 called defrag, but users are cautioned against
casual use. A power outage during such an operation can trash your file
system. Since you need to back up your data anyway, simply writing back
from your copy will do the job.
The MS-DOS file system is also known to lose large amounts of disk
space due to internal fragmentation. For partitions larger than 256 MB,
DOS block sizes grow so large that they are no longer useful (This has
been corrected to some extent with FAT32). Ext2 does not force you to
choose large blocks for large file systems, except for very large file
systems in the 0.5 TB range (that's terabytes with 1 TB equaling 1024 GB)
and above, where small block sizes become inefficient. So unlike DOS
there is no need to split up large disks into multiple partitions to keep
block size down.
Use a 1Kb block size if you have many small files. For large partitions,
4Kb blocks are fine.
2. 6.Shell Introduction
String operators in shell use unique among programming languages curly-
bracket syntax. In shell any variable can be displayed as
${name_of_the_variable} instead of $name_of_the_variable. This
notation was introduced to protect a variable name from merging with
string that comes after it, but was extended to allow string operations on
variables. Here is example in which it is used for separation of a variable
$var and a string "_string" using curly brackets:
$ export var='test'
$ echo ${var}_string # var is a variable that uses syntax ${var} with the
value test
$ echo $var_string # var_string is a variable that doesn't exist, echo doesn't
print anything
2.6.1.1Length Operator
There are several ways to get length of the string.
The simplest one is ${#varname}, which returns the length of the
value of the variable as a character string. For example, if filename
has the value ntp.conf, then ${#filename} would have the value 8.
The second is to use built-in function expr, for example
expr length $string
or
expr "$string" : '.*'
echo ${#stringZ} # 15
echo `expr length $stringZ` # 15
echo `expr "$stringZ" : '.*'` # 15
For checking if length of the string is zero you can use -n STRING
operator.
More complex example. Here's the function for validating that that string
is within a given max length. It requires two parameters, the actual string
and the maximum length the string should be.
check_length()
# check_length
# to call: check_length string max_length_of_string
{
# check we have the right params
if (( $# != 2 )) ; then
echo "check_length need two parameters: a string and max_length"
return 1
fi
if (( ${#1} > $2 )) ; then
return 1
fi
return 0
}
echo $NAME
This is pretty rarely used capability of expr built-in but still sometimes it
can be useful:
expr match "$string" '$substring'
where:
String is any variable of literal string.
$substring is a regular expression.
my_regex=abcABC123ABCabc
# |------|
2.7.String processing
2.7.1Manipulating Strings
Bash supports a surprising number of string manipulation operations.
Unfortunately, these tools lack a unified focus. Some are a subset of
parameter substitution, and others fall under the functionality of the UNIX
expr command. This results in inconsistent command syntax and overlap
of functionality, not to mention confusion.
String Length
${#string}
stringZ=abcABC123ABCabc
echo ${#stringZ} # 15
echo `expr length $stringZ` # 15
echo `expr "$stringZ" : '.*'` # 15
while read line # For as many lines as the input file has ...
do
echo "$line" # Output the line itself.
len=${#line}
if [[ "$len" -lt "$MINLEN" && "$line" =~ [*{\.}]$ ]]
# if [[ "$len" -lt "$MINLEN" && "$line" =~ \[*\.\] ]]
# An update to Bash broke the previous version of this script. Ouch!
# Thank you, Halim Srama, for pointing this out and suggesting a fix.
then echo # Add a blank line immediately
fi #+ after a short line terminated by a period.
done
exit
# Exercises:
# ---------
# 1) The script usually inserts a blank line at the end
#+ of the target file. Fix this.
# 2) Line 17 only considers periods as sentence terminators.
# Modify this to include other common end-of-sentence characters,
#+ such as ?, !, and ".
stringZ=abcABC123ABCabc
# |------|
# 12345678
Index
expr index $string $substring
stringZ=abcABC123ABCabc
# 123456 ...
echo `expr index "$stringZ" C12` #6
# C position.
Substring Extraction
${string:position}
Extracts substring from $string at $position.
If the $string parameter is "*" or "@", then this extracts the positional
parameters, [1] starting at $position.
${string:position:length}
stringZ=abcABC123ABCabc
# 0123456789.....
# 0-based indexing.
randstring="${str1:$POS:$LEN}"
# Can parameterize ^^^^ ^^^^
echo "$randstring"
exit $?
stringZ=abcABC123ABCabc
# 123456789......
# 1-based indexing.
stringZ=abcABC123ABCabc
# =======
Substring Removal
${string#substring}
${string##substring}
stringZ=abcABC123ABCabc
# |----| shortest
# |----------| longest
X='a*C'
${string%substring}
SUFF=TXT
suff=txt
${string%%substring}
stringZ=abcABC123ABCabc
# || shortest
# |------------| longest
echo ${stringZ%%b*c} # a
# Strip out longest match between 'b' and 'c', from back of
$stringZ.
#!/bin/bash
# cvt.sh:
# Converts all the MacPaint image files in a directory to "pbm"
format.
OPERATION=macptopbm
SUFFIX=pbm # New filename suffix.
if [ -n "$1" ]
then
directory=$1 # If directory name given as a script argument...
else
directory=$PWD # Otherwise use current working directory.
fi
# Assumes all files in the target directory are MacPaint image files,
#+ with a ".mac" filename suffix.
exit 0
# Exercise:
# --------
# As it stands, this script converts *all* the files in the current
#+ working directory.
# Modify it to work *only* on files with a ".mac" suffix.
#!/bin/bash
# Batch convert into different graphic formats.
# Assumes imagemagick installed (standard in most Linux distros).
exit $?
#!/bin/bash
# ra2ogg.sh: Convert streaming audio files (*.ra) to ogg.
########################################################
##################
mplayer "$1" -ao pcm:file=$OUTFILE
oggenc "$OUTFILE" # Correct file extension automatically added
by oggenc.
########################################################
##################
exit $?
# Note:
# ----
# On a Website, simply clicking on a *.ram streaming audio file
#+ usually only downloads the URL of the actual *.ra audio file.
# You can then use "wget" or something similar
#+ to download the *.ra file itself.
# Exercises:
# ---------
# As is, this script converts only *.ra filenames.
# Add flexibility by permitting use of *.ram and other filenames.
#
# If you're really ambitious, expand the script
#+ to do automatic downloads and conversions of streaming audio
files.
# Given a URL, batch download streaming audio files (using "wget")
#+ and convert them on the fly.
#!/bin/bash
# getopt-simple.sh
# Author: Chris Morgan
# Used in the ABS Guide with permission.
getopt_simple()
{
echo "getopt_simple()"
echo "Parameters are '$*'"
until [ -z "$1" ]
do
echo "Processing parameter of: '$1'"
if [ ${1:0:1} = '/' ]
then
tmp=${1:1} # Strip off leading '/' . . .
parameter=${tmp%%=*} # Extract name.
value=${tmp##*=} # Extract value.
echo "Parameter: '$parameter', value: '$value'"
eval $parameter=$value
fi
shift
done
}
Substring Replacement
${string/substring/replacement}
${string//substring/replacement}
stringZ=abcABC123ABCabc
echo ---------------
echo "$stringZ" # abcABC123ABCabc
echo ---------------
# The string itself is not altered!
echo
${string/#substring/replacement}
${string/%substring/replacement}
String=23skidoo1
# 012345678 Bash
# 123456789 awk
# Note different string indexing system:
# Bash numbers first character of string as 0.
# Awk numbers first character of string as 1.
echo "----"
# And likewise:
exit 0
2.8.Installing Application:
Online installation:
Offline installation:
say, you downloaded your desired softwares from some website. in this
case if you don't trust the origin of the software then don't install it or
install it at your own risk.
Your downloaded softwares may come as a .zip, tar.gz, tar.bz2, .deb,
.rpm, .tgz, tar.xz or any other types of archives.
if you are in linux mint or in ubuntu or in a debian based OS try to
download .deb packages because it's easier to install in debian based OS.
#5. installing .deb packages:
through terminal:
cd path_to_the_directory_that_contains_the_.deb_file
sudo dpkg -i filename.deb
through gdebi package manager:
if gdebi is not installed then you have to install it through one of the
processes #1,#2,#3 (requires internet connection)
1. then double click on the .deb file or open the file with gdebi package
manager and click install.
2. it will be istalled soon.
#6. installing .rpm packages:
rpm has to be installed in the system, otherwise follow one of the
processes #1,#2,#3 to install rpm (requires internet connection)
code:
cd path_to_the_directory_that_contains_the_.rpm_file
sudo rpm -i filename.rpm
#7. installing from archives(.zip tar.gz.......etc):
these archives generally conatains the source of the package. each of them
generally has a different approach to install. I will be discussing a common
method which will supposedly work for all of them.
general requirements:
1. flex
2. bison or bison++
3. python
As these archives contains the source, your system needs the required
programming languages to compile and build the source. so the general
requirement packages stated above may not be sufficient for you. in that
case you have to install the required packages through one of the processes
#1,#2,#3 (requires internet connection). you can know about the
dependencies about your software in a readme file included in the
archives.
steps:
1. open the archives with archive manager by double clicking it, then
extract it.
2. code:
cd path-to-the-extracted-folder
3. inside the extracted folder look carefully....
a. if you find a file named configure then :
code:
./configure
make
sudo make install
if the first code fails to execute then run this code before above codes:
chmod +x configure
b. if you find a file named install.sh then
code :
chmod +x install.sh
./install.sh or sudo ./install.sh (if it needs root permission)
or you can double click it and select run in terminal or simply run
N.B : sometimes there is a file, something like your_software_name.sh is
found instead of install.sh. for this case you have to replace install.sh with
the correct name in the previous codes.
c. if you find a file named install then
code:
chmod +x install
./install or sudo ./install (if it needs root permission)
or you can double click it and select run in terminal or simply run
d. if you find a file named make (if there is no configure file) then
code:
make
sudo make install
e. If you still can't find the required files
then it may be in a special folder (generally in a folder named 'bin'). move
to this folder with cd command with appropriate path and then look
again and follow the same process.
#8. pre installed archives:
some packages are archived as pre installed packages i.e you don't have to
install them , you just need to extract them in a secure place and theres an
executable file (name is generally same as the software name) in the
extracted folder or in child folders like bin,build etc. you have to find it
and make it executable.
Example: eclipse, adt bundle (android developing tool)
code to make executable:
chmod +x filename_with_exact_path
then you can run it with :
code:
filename
or double clicking it and selecting run in terminal or run, whatever
supports your software.
#9. installing .sh files:
some softwares come with a .sh file to install it
chmod +x filename.sh
./filename.sh or sudo ./filename.sh (if it needs root permission)
or double click it and select run in terminal or run, whatever supports
your software.
#10. installing .run files:
some softwares come with a .run file to install it
chmod +x filename.run
./filename.run or sudo ./filename.run (if it needs root permission)
or double click it and select run in terminal or run, whatever supports
your software.
Additional info about offline installation:Often, softwares generally have a
lot of dependencies. You need to download all of them, which sometimes
can be very difficult and tiring. for this you can follow the instructions
here which will save your time and effort.