Linux and Shell

Scripting
Dr. Shak Hossain
Jan 16, 2025
How about you?

● Who has experience programming?

● Using Linux?
● Using compute clusters?
● Using the Shared Computing Cluster (SCC)?
Linux
What, Who, When, Where & Why
What is Linux?
● Unix-like computer operating system assembled under the model of free and
open-source software development and distribution.
● These operating systems share the Linux kernel.
○ Typically have the GNU utilities
● Comes in several “distributions” to serve different purposes.
What is Linux? gcc

wc bash emacs
● Bird’s eye view

multitasking

tcsh
cat device file grep
access
Hardware system
sh

Kernel awk
sort

Shell

Utilities
Who is Linux?

Linus Torvalds Richard Stallman

Linux is an O/S core originally written by A set of programs written by Richard Stallman
Linus Torvalds. Now almost 10,000 and others. They are the GNU utilities.
developers including major technology
companies like Intel and IBM.
When is Linux?

~1991
Where is Linux?
● World Wide Web
○ 67% of the world’s web-servers run Linux (2016)
● Research/High-Performance Compute
○ Google, Amazon, NSA, 100% of TOP 500 Supercomputers.
● Modern Smartphones and devices
○ The Android phone
○ Amazon Kindle
○ Smart TVs/Devices
Why Linux?
● Free and open-source.
● Powerful for research data centers
● Personal for desktops and phones
● Universal
● Community (and business) driven.

The most common OS used by Engineers and Researchers when working

on a server or computer cluster.
Connecting
Let’s use Linux
Local System

Remote Server
Connecting from
Different Platforms
SSH X-Win SFTP
Microsoft MobaXterm
Windows https://mobaxterm.mobatek.net

Apple XQuartz Cyberduck

Terminal (Built in)
MacOS https://www.xquartz.org https://cyberduck.io

Linux Terminal (Built in) X11(Built in) Various (Built in)

Apple MacOS
● SSH: Terminal
○ Built in to macOS
Applications > Utilities > Terminal

● X-Windows: XQuartz
○ Download: https://www.xquartz.org/ (Note: This install requires a logout.)

● SFTP: Your choice

○ Filezilla: https://filezilla-project.org/
○ Cyberduck: https://cyberduck.io
○ Many others
Linux
Interaction
Shell, Prompt, Commands and System Use
Linux: The Shell
● Program that interprets commands and sends them to the OS

● Provides:
○ Built-in commands
○ Programming control structures
○ Environment variables

● Linux supports multiple shells.

○ The default on SCC is Bash.

Bash” = “Bourne-again Shell”

(GNU version of ~1977 shell written by Stephen Bourne)
Linux: The “Prompt”
Your Username Current Directory

username@hostname ~]$

The System Name Input

( In Linux “ ~ ” is a shorthand for your home directory. )

Linux: Command Basics
[username@scc1 ~]$ command --option argument

● Command: Command/program that does one thing

● Options: Change the way a command does that one thing
Short form: Single-dash and one letter e.g. ls -a
Long form: Double-dash and a word e.g. ls --all

● Argument: Provides the input/output that the command interacts with.

For more information about any command, use man or info (e.g. “man ls”)
Commands: Hands-On
● After you connect, type
○ whoami # my login
○ hostname # name of this computer
○ echo “Hello, world” # print characters to screen
○ echo $HOME # print environment variable
○ echo my login is $(whoami) # replace $(xx) with program output
○ date # print current time/date
○ cal # print this month’s calendar
○ shazam # bad command
Commands: Hands-On Options (Lab)
● After you connect, type
○ whoami # my login
○ hostname # name of this computer
○ echo “Hello, world” # print characters to screen
○ echo $HOME # print environment variable
○ echo my login is $(whoami) # replace $(xx) with program output
○ date # print current time/date
○ cal # print this month’s calendar
○ shazam # bad command
Commands: Hands on Options (Lab)
● Read Chapter 2
○ 2.1 Login
○ 2.2 Changing Password (passwd)
○ 2.3 Picking Secure Password
○ 2.4 Who are You? (whoami, who am i, id, users, who, )
○ 2.5 What is everyone doing on the computer (w)
○ 2.6 Checking current date and time (time, date)
● Read Chapter 10
○ 10.1 Editing the Unix way (vi, :q, vi filename, :q!, )
Commands: Hands-On Options
● Commands have three parts; command, options and arguments/parameters.

Example: cal –j 3 1999. “cal” is the command, “-j” is an option (or switch),
“3” and “1999” are arguments/parameters.

[username@scc1 ~]$ cal -j 3 1999

● What is the nature of the prompt?

● What was the system’s response to the command?
Commands

“Small programs that do one thing well”

● The Unix Programming Environment, Kernighan and Pike

… at its heart is the idea that the power of a system comes more from the
relationships among programs than from the programs themselves. Many
UNIX programs do quite trivial things in isolation, but, combined with other
programs, become general and useful tools.
Commands: Selected text processing utilities
● awk Pattern scanning and processing language
● cat Display file(s)
● cut Extract selected fields of each line of a file
Compare two files Just a few of the
● diff
Search text for a pattern commands for
● grep
Display the first part of files text processing
● head
● less Display files on a page-by-page basis
● sed Stream editor (esp. search and replace)
● sort Sort text files
● split Split files
● tail Display the last part of a file
● tr Translate/delete characters
● uniq Filter out repeated lines in a file
● wc Line, word and character count
Variables and Environment Variables

● Variables are named storage locations.

○ USER=augustin
○ foo=“this is foo’s value”

● “Environment variables” are variables used and shared by the shell

○ For example, $PATH tells the system where to find commands.

● Environment variables are shared with programs that the shell runs.
Bash variables
● To create a new variable, use the assignment operator ‘=‘

[username@scc1 ~]$ foo=“this is foo’s value”

● The foo variable can be printed with

echo
[username@scc1 ~]$ echo
$foo this is foo’s value

● To make $foo visible to programs run by the shell (i.e., make it an

“environment variable”), use export:

[username@scc1 ~]$ export foo

Environment Variables
● To see all currently defined environment variable, use printenv:
[username@scc1 ~]$ printenv
HOSTNAME=scc1
TERM=xterm-256color
SHELL=/bin/bash
HISTSIZE=1000
TMPDIR=/scratch
SSH_CLIENT=168.122.9.131 37606 22
SSH_TTY=/dev/pts/191
USER=cjahnke
MAIL=/var/spool/mail/cjahnke
PATH=/usr3/bustaff/cjahnke/apps/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin
PWD=/usr3/bustaff/cjahnke/linux-materials
LANG=C
MODULEPATH=/share/module/bioinformatics:/share/module/chemistry
SGE_ROOT=/usr/local/ogs-ge2011.11.p1/sge_root
Command History and Command Line Editing
● Try the history command
● Choose from the command history using the up ↑ and down ↓ arrows
● To redo your last command, try !!
● To go further back in the command history try !, then the number as shown
by history (e.g., !132). Or, !ls, for example, to match the most recent ‘ls’
command.

● What do the left ← and right → arrow do on the command line?

● Try the <Del> and <Backspace> keys
Help with Commands
● Type
○ date –-help
○ man date
○ info date

● BASH built-ins
○ A little different from other commands
○ Just type the command ‘help’
○ Or ‘man bash’

Yes, you can always Google it.

On using ‘man’ with ‘less’
● The man command outputs to a pager called less, which
supports many ways of scrolling through text:
○ Space, f # page forward
○ b # page backward
○ < # go to first line of file
○ > # go to last line of file
○ / # search forward (n to repeat)
○ ? # search backward (N to repeat)
○ h # display help
○ q # quit help
I/O Redirection
I/O redirection with pipes
● Many Linux commands print to “standard output”, which defaults to
the terminal screen. The ‘|’ (pipe) character can be used to divert or
“redirect” output to another program or filter.
○ w # show who’s logged on
○ w | less # pipe into the ‘less’ pager
○ w | grep ‘tuta’ # pipe into grep, print lines containing ‘tuta’
○ w | grep –v ‘tuta’ # print only lines not containing ‘tuta’
○ w | grep ‘tuta’ | sed s/tuta/scholar/g
# replace all ‘tuta’ with ‘scholar’
More examples of I/O redirection
● Try the following (use up arrow to avoid retyping each line):
○ w | wc # count lines
○ w | cut –d ‘ ’ –f1 | sort # sort users
○ w | cut –d ‘ ’ –f1 | sort | uniq # eliminate duplicates
● We can also redirect output into a file:
○ w | cut –d ‘ ’ –f1 | sort | uniq > users
● Note that ‘awk’ can be used instead of ‘cut’:
○ w | awk ‘{print $1;}’ | sort | uniq > users
● Quiz:
○ How might we count the number of distinct users currently logged in?
For extra credit, how can we avoid over-counting by 2? (Hint: use ‘tail’.)
The Filesystem
The Linux File System
● The structure resembles an upside-down tree
● Directories (a.k.a. folders) are collections of files and other
directories.
● Every directory has a parent except for the root directory.
● Many directories have subdirectories.
Navigating the File System

● Essential navigation commands:

○ pw print current directory
d list files
○ ls change directory
○ cd
Navigating the File System

We use pathnames to refer to files and directories in the Linux file system.

● There are two types of pathnames:

○ Absolute – The full path to a directory or file; begins with /
○ Relative – A partial path that is relative to the current working directory;
does not begin with /
Navigating the File System

● Special characters interpreted by the shell for filename expansion:

○ ~ your home directory (e.g., /usr1/tutorial/tuta1)
○ . current directory
○ .. parent directory
○ * wildcard matching any filename
○ ? wildcard matching any character
○ TAB try to complete (partially typed) filename
Navigating the File System

● Examples:
○ cd /usr/local Change directory to /usr/local/lib
○ cd ~ Change to home directory (could just type ‘cd’)
○ pw Print working (current) directory
d
○ cd .. Change directory to the “parent” directory
○ cd / Change directory to the “root”
○ ls –d pro* Listing of only the directories starting with “pro”
The ls Command
● Useful options for the “ls” command:
○ ls -a List all files, including hidden files beginning with a “.”
○ ls -ld * List details about a directory and not its contents
○ ls -F Put an indicator character at the end of each name
○ ls –l Simple long listing
○ ls –lR Recursive long listing
○ ls –lh Give human readable file sizes
○ ls –lS Sort files by file size
○ ls –lt Sort files by modification time (very useful!)
Some Useful File Commands
● cp [file1] [file2] copy file
● mkdir [name] make directory
● rmdir [name] remove (empty) directory
● mv [file] [destination] move/rename file
● rm [file] remove (-r for recursive)
● file [file] identify file type
● less [file] page through file
● head -n N [file] display first N lines
● tail -n N [file] display last N lines
● ln –s [file] [new] create symbolic link
● cat [file] [file2…] display file(s)
● tac [file] [file2…] display file in reverse order
● touch [file] update modification time
● od [file] display file contents, esp. binary
Manipulating files and directories
● Examples:
○ cd # The same as cd ~
○ mkdir test
○ cd test
○ echo ‘Hello everyone’ > myfile.txt
○ echo ‘Goodbye all’ >> myfile.txt
○ less myfile.txt
○ mkdir subdir1/subdir2 # Fails. Why?
○ mkdir -p subdir1/subdir2 # Succeeds
○ mv myfile.txt subdir1/subdir2
○ cd ..
○ rmdir test # Fails. Why?
○ rm –rf test # Succeeds
Symbolic links
● Sometimes it is helpful to be able to access a file from multiple locations
within the hierarchy. On a Windows system, we might create a “shortcut.” On
a Linux system, we can create a symbolic link:

○ mkdir foo # make foo directory

○ touch foo/bar # create empty file
○ ln –s foo/bar . # create link in current dir.
Finding a needle in a haystack
● The find command has a rather unfriendly syntax, but can be exceedingly
helpful for locating files in heavily nested directories.

● Examples:
○ find ~ -name bu –type d # search for “bu” directories in ~
○ find . –name my-file.txt # search for my-file.txt in .
○ find ~ -name ‘*.txt’ # search for “*.txt” in ~

● Quiz:
○ Can you use find to locate a file called “needle” in your haystack
directory?
○ Extra credit: what are the contents of the “needle” file?
Processes & Job Control
Processes and Job Control
● As we interact with Linux, we create numbered instances of running programs
called “processes.” You can use the ‘ps’ command to see a listing of your
processes (and others!). To see a long listing, for example, of all processes
on the system try:

[username@scc1 ~]$ ps -ef

● To see all the processes owned by you and other members of the class, try:

[username@scc1 ~]$ ps -ef | grep tuta

Processes and job control
● Use “top” to see active processes.
Tasks: 408 total, 1 running, 407 sleeping, 0 stopped, 0 zombie
Cpu(s): 0.3%us, 0.1%sy, 0.0%ni, 99.6%id, 0.0%wa, 0.0%hi, 0.0%st
Mem: 99022756k
0.0%si, total, 69709936k used, 29312820k free, 525544k buffers
Swap: 8388604k total, 0k used, 8388604k free, 65896792k cached

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND

7019 root 20 0 329m 137m 4852 S 4.0 0.1 217:01.56 sge_qmaster
38246 isw 20 0 88724 2764 1656 S 0.7 0.0 0:01.28 sshd
41113 cjahnke 20 0 13672 1512 948 R 0.7 0.0 0:00.03 top
2324 root 20 0 0 0 0 S 0.3 0.0 0:21.82 kondemand/2
7107 nobody 20 0 89572 10m 2400 S 0.3 0.0 2:18.05 gmond
27409 theavey 20 0 26652 1380 880 S 0.3 0.0 0:34.84 tmux
1 root 20 0 25680 1604 1280 S 0.0 0.0 0:05.74 init
2 root 20 0 0 0 0 S 0.0 0.0 0:00.07 kthreadd
3 root RT 0 0 0 0 S 0.0 0.0 0:00.89 migration/0
4 root 20 0 0 0 0 S 0.0 0.0 0:01.72 ksoftirqd/0
5 root RT 0 0 0 0 S 0.0 0.0 0:00.00 stopper/0

(refreshes every 2 seconds)

Foreground/background
● Thus far, we have run commands at the prompt and waited for them
to complete. We call this running in the “foreground.”

● Use the “&” operator, to run programs in the “background”,

○ Prompt returns immediately without waiting for the command to complete:

[username@scc1 ~]$
mycommand & ← process id
[1] 54356
[username@scc1 ~]$
Process Control Practice
● Let’s look at the “countdown” script, in your scripts folder for practice

[username@scc1 ~]$ cd ~/scripts

[username@scc1 ~]$ cat countdown

● Make the script executable with chmod:

[username@scc1 ~]$ chmod +x countdown

● First, run it for a few seconds, then kill with Control-C.

[username@scc1 ~]$ ./countdown 100
100
99
98
^ ← Ctrl-C = (^C)
C
Process control
● Now, let’s try running it in the background with &:
[username@scc1 ~]$ ./countdown 60 &
[1] 54355
[username@scc1 ~]$
60
59

● The program’s output is distracting, so redirect it to a file:

[username@scc1 ~]$ countdown 60 > c.txt &
[1] 54356
[username@scc1 ~]$
Process control
● Type ‘ps’ to see your countdown process.

● Also, try running ‘jobs’ to see any jobs running in the background from this
bash shell.

● To kill the job, use the ‘kill’ command, either with the five-digit process id:
○ kill 54356

● Or, you can use the job number (use ‘jobs’ to see list) with ‘%’:
○ kill %1
Backgrounding a running job with C-z and
‘bg’
● Sometimes you start a program, then decide to run it in the background.

[username@scc scripts]$ ./countdown 200 > c.out

1 ← Ctrl-Z = (^Z)
^Z
[1]+ Stopped ./countdown 200 > c.out
[username@scc1 scripts]$ bg
[1]+ ./countdown 200 > c.out &

[username@scc scripts]$ jobs

1
[1]+ Running ./countdown 200 > c.out &
[username@scc1 scripts]$
Editors
Regular expressions
● Many Linux tools, such as grep and sed, use strings that describe sequences
of characters. These strings are called regular expressions.
Here are some examples:

○ ^foo # line begins with “foo”

○ bar$ # line ends with “bar”
○ [0-9]\{3\} # 3-digit number
○ .*a.*e.*i.*o.*u.* # words with vowels in order*
File Editors
● gedit
○ Notepad-like editor with some programming features (e.g., syntax highlighting). Requires
X-Windows.
● nano
○ Lightweight editor. Non-Xwindows.
● emacs
○ Swiss-army knife, has modes for all major languages, and can be customized. Formerly steep
learning curve has been reduced with introduction of menu and tool bars. Can be used under
Xwindows or not.
● vim
○ A better version of ‘vi’ (an early full-screen editor). Very fast, efficient. Steep learning curve.
Popular among systems programmers. Terminal or X-Windows.
Creating and Running Code
“Hello, world” in C
● cd to “~/c”, and read hello.c into your editor of choice.
● Modify the text on the printf line between “[“ and “]” and save the file.
● Produce an executable file called “hello” by compiling the program with gcc:

[username@scc1 ~]$ gcc –o hello hello.c

● Run the program at the command line:

[username@scc1 ~]$ ./hello

● Optional: modify countdown script to run hello program

Obtaining the Supplementary Course Material

● In browser, navigate to http://rcs.bu.edu/tutorials

○ Scroll to Introduction to Linux and select “Cheat Sheets”.

● See also other Linux tutorials:

○ http://www.tutorialspoint.com/unix/
○ Edx Linux intro [Google “edx linux”]
○ http://www.cse.sc.edu/~okeefe/tutorials/unixtut/
Questions?
Linux Distributions
Linux has various distributions (Distros):
▪ Why many distributions?
• Server vs. Desktop
• Commercial user support
• Special hardware support

▪ Debian/RedHat/SUSE/…
• The major difference is the software package management on these distros.
 Structure of Linux OS

Linux
Shell
• Here is where you • User software
start interacting with and applications
the OS Supplement • High-level programs
• Multiuser environments
al
Software

Core
Linux
Tools
• OS core commands
• System-level programs
• GNU tools • Process/Thread
• Compilers scheduling
Linux
• Memory Management
Kernel • File and I/O management
• Device drivers
• System calls
Linux Shell
What is the shell?
▪ The shell is a program that takes commands from user’s keyboard and passes
them to the operating system to execute.
• Many shell programs are available for Linux:
• bash, sh, csh, tcsh, ksh, zsh, …
• For this tutorial, we will use bash, but other shells are conceptually similar

quanah:$ command

What is Terminal?
▪ Terminal is a program that opens in a window and lets users interact with the
shell.
 Working with
Files and
Directories
Understanding Linux Directory Structure
/
boot /
etc /boot
tmp /etc
hom
/tmp
/home
e
mydi /home/mahmadia
mahmadi /home/mahmadia/mydir
ra
/home/eraider
vareRaider
/var
lustr
/lustre
e
/lustre/work
work
/lustre/scratch
.. scratc
Basic Directory Operations
Where am I?
▪ pwd command to print working
directory
quanah:$ pwd
/home/username

▪ ls command to list contents of the current

directory
quanah:$
ls
test1.txt
Mydir1
mydir2
Basic Directory Operations
Make/Remove/Go into a directory?
▪ mkdir command to make a new
directory
quanah:$ mkdir my_new_dir
quanah:$
ls
▪ cd command to change into a
my_new_dir

directory
quanah:$ cd my_new_dir

▪ rmdir command to remove an empty

directory
quanah:$ rmdir my_new_dir
Basic Directory Operations
More about ls. (Commands with options)
▪ ls -a (--all) list all files including hidden
files
quanah:$ ls -a

▪ ls -l show file
details
quanah:$ ls -l

▪ Combine multiple options for a

command
quanah:$ ls -a
-l quanah:$ ls
-al
Special Directory Names (Characters)
▪ Return to home directory (~)
quanah:$ cd
quanah:$ cd ~
quanah:$ cd
~/
▪ Current directory (.)
quanah:$ ls .
quanah:$ ls
./
▪ Parent directory (..)
quanah:$ cd ..
quanah:$ cd ../
quanah:$ ls
../../
quanah:$ cd
Absolute VS. Relative path

Absolute Path of a file/directory:

▪ Always starts with / and goes all the way to the file or directory
• For instance:
quanah:$ cat /home/mahmadia/mydir/file1.txt

Relative Path of a file/directory:

▪ It defines from the current or parent directory
• For instance:
quanah:$ cat ../mydir/file1.txt
File and Directory Names
Naming: What to use: •
•
A-Z
A-z
▪ Letters and numbers [A-Z a-z 0-9]: • 0-9
• dot (.)
• File and directory names are case sensitive (e.g. ‘M’ vs ‘m’)
• dash (-)
▪ (.) can be used anywhere in the file/directory • underscore (_)
name
• Using (.) at the beginning of the file/directory name makes it hidden
• Using (.) alone will be considered as current working directory
• () parenthesis
What to Avoid: • " double
quotes
▪ Avoid spaces in the file • ' single quote
• ? Question
• “my file name.txt ” 🡪 “my_file_name.txt ”
names mark
▪ Avoid these characters: • $ dollar sign
• * asterisk
• () " ' ? $ *\ / :
• \ back slash
• / slash
A few Basic File Operations
Let’s learn a few file operations before Exercise #1
▪ touch creates an empty file (Not the primary job!)
quanah:$ touch
my_file quanah:$ ls
my_file

▪ echo prints its arguments to the standard output

▪ >redirects the standard output to a file (Creates/Rewrite the file)
▪ >> redirects the standard output to a file (Append to the file)
• More details on output redirects in next few slides
quanah:$ echo "hello world!" >
hello.txt quanah:$ echo "second line"
A few Basic File Operations (Cont.)
Let’s learn some file operations before Exercise #1
▪ cat shows the content of a file(s)
quanah:$ cat
hello.txt Hello
world!
second line
▪ wc counts number of lines, words, and characters in a text
file
quanah:$ wc hello.txt
2 4 25 hello.txt
quanah:$ wc -l
hello.txt
2 hello.txt
quanah:$ wc -w hello.txt
Exercise #1
1. Return to your home directory
2. Create a new directory and name it “exercise1”
3. Go into the “exercise1” directory
4. Create a new text file and name it “test1.txt”. Then add the following lines into
the file:
Exercise #1:
This is a test file
here is the last line

5. Now create a new directory under the “exercise1” directory and name it
“subdir1”.
6. Go into the “subdir1” and try to print the content of “test1.txt” on your terminal.
7. Return to the “exercise1” directory and count the number of lines in “test1.txt”.
File and Directory Operations
Copy Files and Directories:
▪ cp command to make a copy of a file(s) or directory(s)
• -r: Recursively copy the directories and subdirectories as well
File and Directory Operations
Move/Rename Files and Directories:
▪ mv command to make a copy of the source file(s) and
directory(s) into the destination and remove the source at the end.
• Can be used for renaming a file/directory as well
File and Directory Operations
Deleting Files and Directories:
▪ rm command to remove file(s) and directory(s) permanently
• Removing files and directories by this command may not be recovered
easily (This is an exception for /home directory on HPCC clusters)

• -r: Recursively copy the directories and subdirectories as well

Output Redirection
▪ So far, all the commands that we used sent their output to
the screen. We can control this in Linux:
• < redirects to the standard input
• > redirects the standard output
• >> appends the standard output
• 1> redirects the standard output (equals to >)
• 2> redirects the standard error
• &> redirects the standard output and standard error
• | (pipe) pass the output of one command to another
Exercise #2
1. Return to your home directory
2. Create a new directory and name it “exercise2”
3. Go to the “exercise2” directory
4. Copy the “test1.txt” file from the “exercise1” directory to
the “exercise2” directory
5. Make a new directory under “exercise2” and name it “subdir2”
6. Rename the file “test1.txt” to “test2.txt”
7. List the contents of “exercise2” and redirect the output to “ls.out”
file
8. Remove everything under the “exercise2” directory
Linux Essential Commands 2
File/Directory Ownership and Permissions:
▪ Every file/directory belongs to a specific user or a group of users

▪ Every user/group many have permissions to read, write, and/or

execute
User Group Other
rwx rwx rwx

▪ If you set write permission for a directory, you can (create/delete) new
entries

▪ If you set read permission for a directory, you can list (ls) the contents

▪ If you set execute permission for a directory, you can (cd) into the directory
Linux Essential Commands 2
File/Directory Ownership and Permission Examples:
▪ chmod command changes the rwx mode bits of a file or directory
• +/-: adds or removes the mode bits
• u: Sets the permissions for the owner of the file/directory
• g: Sets the permissions for the group that the owner belongs to
• o: Sets the permissions for the other users
• a: Sets the same permissions for user/group/other
Linux Essential Commands 2
File/Directory Ownership and Permission:
▪ With stat or ls –l commands you can check the ownership
and permissions of a file or directory

▪ whoami command Displays the username of the current user

▪ groups command prints the groups a user belongs to

Linux Essential
Commands
(Part 1)
Linux Essential Commands 1
Check the manual page of a command:
▪ man command to search and open the manual page of a
Linux command
man cp

• /word search the man page for the text ‘word’

• n search for the next found match
• N search for the previous found match
• g Go to the beginning of the manual page
• G Go to the end of the manual page
• q Exit (quit)
Linux Essential Commands 1
Other useful commands to find out more:
▪ Several built-in references are available within the operating
system.
• info more detailed tutorials and examples.
• apropos
quanah:$ info cpSearch the manual database for a “string”.
quanah:$ apropos
▪mkdir
Many reference guides are available.
• Search the web for “reference” – for example “bash reference”.
• Specifically for bash, try finding the “bash advanced scripting guide” (but only when
you are ready for the advanced stuff!
• Guides are available for other shells and commands also.
Linux Essential Commands 1

Paging through long text files/outputs:

▪ more command for paging through text one screenful at a time

• More only allows to move the text forward
quanah:$ more ./README
quanah:$ cat ./README |
more
▪ Less command is an improved version of more command
which allows backward movement in the file as well as
forward movement
quanah:$ less ./README
quanah:$ cat ./README |
less
Linux Essential Commands 1
Display a partial content of a file/output:
▪ head command to see first lines of a file (10 lined by
default)
• -n head
quanah:$ First
-n 15
<NUM>: NUM number
./README of a file/output
quanah:$ cat ./README | head
-3
▪ tail command to see last lines of a file (10 lined by
default)
• -n <NUM>: Last NUM number of a file/output
quanah:$ tail -n 20 ./README
quanah:$ cat ./README | tail
-5
Linux Essential Commands 1
Searching for pattern(s) in files/outputs:
▪ grep command to print lines in a file matching a pattern
• -i: Ignore case distinctions in both the PATTERN and the input
files
• -v: Invert the sense of matching, to select non-matching lines
quanah:$ grep "test"
../exercise1/test1.txt This is a test
file
quanah:$ grep "is"
../exercise1/test1.txt Exerc ise:
This is a test file
here is the last
Linux Essential Commands 1
History of your commands:
▪ history command to see the commands you have typed so
far.
• Your command are saved to the .bash_history file under your
home directory.
quanah:$ history | less
quanah:$ history | tail -15
quanah:$ history | grep
“cp”
Exercise #3
1. Search in your history and try to list your last 10 “ls”
commands that you have used

1. Check the manual of the “man” command and see what type
of manuals are available on Linux
Coming in Part 2 of this Training …
Part 2: Introduction to more advanced topics in Linux
❖ Linux Essential Commands (Part 2)

❖ Text Editors in Linux

❖ Linux Environment Variables

❖ Basic Bash Scripting in Linux

Course Agenda
Part 2: Introduction to more advanced topics in Linux
❖ Linux Essential Commands (Part 2)

❖ Text Editors in Linux

❖ Linux Environment Variables

❖ Basic Bash Scripting in Linux

Review
Command Description
pwd Prints Current Working Directory
ls Lists the contents of a directory
cd Change the current path to the destination directory
mkdir Makes a new directory
rmdir Removes an empty directory
cp Copy file or directory
mv Move/Rename file or directory
rm Delete file or directory
cat Concatenates and prints the contents a file
Review
Command Description
echo Write arguments to the standard output
wc word, line, character, and byte count
man Search and open the manual page of a Linux command
more Paging through text one screenful at a time
less Improved version of more allows backward/forward movement
head Display first lines of a file
tail Display last lines of a file
grep Print lines in a file matching a pattern
history See the commands you have typed so far
Linux Essential
Commands
(Part 2)
Linux Essential Commands 2
Define the file type:
▪ Unlike Windows, files extensions rarely define the type of a file in Linux.

• For example: “file.txt” many not be a TEXT file.

▪ file command displays the file type along with a brief summary of the
file contents.
quanah:$ file myfile
myfile: ASCII text
quanah:$ file /home/mahmadia/program.py
/home/mahmadia/program.py: Python script, ASCII text
executable quanah:$ file test.tar.gz
test.tar.gz : gzip compressed data, from Unix, last modified:
Wed Sep 4 14:04:10 2019
Linux Essential Commands 2
CRLF Line Terminator:
▪ Windows text editors such as notepad add a Carriage Return Line Feed
(CRLF) character at the end of each line of the text which causes
problems with many Linux applications.
▪ dos2unix command fixes the CRLF issue in text files from Windows.

quanah:$ file windows.txt

windows.txt: ASCII text, with CRLF line
terminators quanah:$ dos2unix windows.txt
dos2unix: converting file windows.txt to Unix format
... quanah:$ file windows.txt
windows.txt : ASCII text
Linux Essential Commands 2
Compression and File Packaging:
▪ zip command packages and compresses files and directories
• zip [OPTIONS] zip_file file_dir1
file_dir2 …

• -r : Add the directories and subdirectories contents into the zip file
quanah:$ ls
mydir
test1.txt
quanah:$ zip -r archive.zip ./*
adding: ./test1.txt (deflated
62%) adding: ./mydir/ (deflated
0%) quanah:$ ls
archive.zip mydir test1.txt
Linux Essential Commands 2
Compression and File Packaging:
▪ unzip command lists and extracts the contents of a zipped
file
• -l : Lists the contents of a zipped file
quanah:$
ls
archive.zi
p
quanah:$ unzip
archive.zip quanah:$ ls
archive.zip mydir test1.txt
Linux Essential Commands 2
Compression and File Packaging:
▪ Other compression commands available in Linux

Command Description Decompression File Ext.

zip Packages and compresses files and unzip .zip
directories
gzip A GNU tool for compressing or gunzip .gz
expanding files/directories gzip -d
bzip2 Compresses files using the Burrows- bunzip2 .bz,
Wheeler block sorting text bzip2 -d .bz2
compression algorithm.
xz Similar to gzip and bzip2 unxz .xz
Linux Essential Commands 2
Archiving:
▪ tar command saves many files and directories into a single “archive”
file
• tar OPTIONS dest_file src1 src2 …

• -f define the archive file path/name

• -c Create a new archive
• -a Append to the existing archive file
• -x Extract the contents of an archive file
• -z Compress archive file with gzip
• -j Compress archive file with bzip2
• -v verbosely list files processed
Linux Essential Commands 2
Archiving:
▪ tar command examples:
quanah:$ ls
mydir
test1.txt
quanah:$ tar -cf myarchive.tar
./* quanah:$ ls
myarchive.tar mydir test1.txt
quanah:$ tar -xvf
myarchive.tar test1.txt
mydir/
quanah:$ tar -czf docs.tar.gz
/home/mahmadia/docs quanah:$ ls
Docs.tar.gz myarchive.tar mydir
Linux Essential Commands 2
Download files from internet:
▪ wget command downloads files from internet

• -O : (capital O) defined the name of the destination file on your

system
quanah:$ wget
“https://repo.anaconda.com/miniconda/Miniconda3-
latest-Linux-x86_64.sh” -O miniconda3.sh
Exercise #4
1. Go to your home directory
2. Create a new directory and name it “exercise4”
3. Go to the “exercise4” directory
4. Choose a small directory from your home directory
5. Try to archive and compress the directory by tar and save it under
the “exercise4” directory.
6. Now try to decompress the tar file that you just created
7. Check the type of the file
8. Now try to untar the file
Linux Essential Commands 2
Access Control List (ACL):
▪ Second level of discretionary permissions that override the standard
ugo/rwx
▪ better granularity in setting access to a file or a directory

▪ Recommended on HPCC cluster environments

▪ getfacl shows the file/directory Access Control List

• getfacl <File | Directory>

▪ setfacl modify/remove the ACL permissions

• setfacl -m [u|g]:<username>:rwx <File|Directory>

• setfacl -x [u|g]:<username> <File|Directory>

Text Editors in
Linux
Text Editors in Linux
How to edit text files in Linux?
▪ There are many text editors available on Linux

• nano is a small, simple and friendly editor

• vi/vim is a powerful text editor which can be used to edit all kinds of
text
• emacs is part of the GNU project written by Richard Stallman

▪ In this training course we will cover nano and vi/vim

▪ Let’s look into nano (Demo)

Text Editors in Linux
How does vi/vim work?
▪ vi/vim is a very popular text editor among programmers and system
administrators
▪ It supports many programming and scripting languages

▪ Suitable for more advanced file editing

▪ vi/vim has two modes:

1. Text mode: which can be enabled by typing i (insert) or a (append)

2. Command mode: which will be enabled by pressing the Esc key on

keyboard.
Text Editors in Linux
Some useful vi/vim commands:
command description command description
! Forces the action i insert
:q quit a append
:q! Force quit x Delete a character
:w write y[count]y Yank (copy) [count] lines
:wq Write and quit d[count]d Cut (Delete) [count] lines
:x Write and quit p Paste after the current line

▪ Let’s look into vim

(Demo)
Exercise #5
1. Go to your home directory
2. Create a new directory and name it “exercise5”
3. Go to the “exercise5” directory
4. Open a new text file with the editor of your choice and type the lines below:

Exercise #5
This is a Linux text editor exercise
We finally made it!
5. Save the file and exit
6. Try to display the contents of the file on your screen
7. Change the permission of the file as below:
• Set read/write permission for the owner
• Set read-only permission to your group
• Give a read-only permission to user mahmadia and then remove it.
Environment
Variables in
Linux
Linux Environment Variables
What is the environment variable?
▪ Environment Variables stores any user-defined or system-defined information
that can be accessed within the shell.
▪ Environment Variables are useful for passing data to programs or being used in
shell scripts.
▪ Defining a variable is very simple (do not put spaces around = symbol)
quanah:$ VAR_NAME=“This is a variable”

▪ When referencing a variable place a ($) before the variable

name
quanah:$ echo
$VAR_NAME This is a
variable
Linux Environment Variables
Common Linux Environment
Variables
HOME Pathname of the current user’s home directory
PATH Colon separated list of directories where commands can be found
SHELL Name of the current Shell program
PWD Print current working directory
USER Print current username
TERM The type of the terminal
HOSTNAME Displays computer’s hostname
Linux Environment Variables
PATH Environment Variable
▪ Shell uses the PATH environment variable to locate commands
▪ The PATH variable is colon (:) separated, and can be displayed with
echo
quanah:$ echo $PATH
/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

▪ You can add a directory into the PATH variable of your own
environment
quanah:$ export PATH="/home/username/bin:$PATH"
/home/username/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

▪ export command promotes a shell variable to an environment

variable
Linux Environment Variables
Set Environment Variables at login time
▪ ~/.bashrc : Commands for environment variables that you want to set
up at login time should be included in your ~/.bashrc file.
• For HPCC users we highly recommend using modules instead of
modifying the ~/.bashrc file. (Modules are covered in HPCC New
User Training)
quanah:$ vim ~/.bashrc

export WORK=/lustre/work/mahmadia
export
SCRATCH=/lustre/scratch/mahmadia
export PATH=“$PATH:$WORK/bin”
W9
Scripting
Mar 11, 2025
Bash Scripting

● Read Chapter 10 and 11

○ Vi Options
○ Advanced Vi Commands

● Bash script is an executable file contains Bash

shell commands which could be used to automate
and simplify things.
○ Shell script is a text file starts with
(#!) followed by the path to the shell
interpreter (i.e. /bin/bash)
Writing and Executing Shell Script
Writing and Executing a Shell Script
● A shell script is a text file containing a sequence of commands.
● The script should start with a shebang (#!), which specifies the
interpreter:

#!/bin/bash
echo "Hello, World!"

Making the script executable:

$chmod +x script.sh
$./script.sh
Variables in Shell Script
Variables store data for reuse:

name="Linux"
echo "Welcome to $name!"

Use read to take user input:

echo "Enter your name:"

read username
echo "Hello, $username!"
Scripting: Input Output
## Problem: Write a shell script that takes a user's name as input and greets
them.

## Solution:

#!/bin/bash

# Ask the user for their name

echo "What's your name?"

read name

# Greet the user

echo "Hello, $name! Nice to meet you."

Scripting: Control Flow
## beginning of if-else statement

if [ $var -eq 1 ]; then

...
elif [ $var -ne 1 ]; then
...
elif [ $var -ge 1 ]; then
…
elif [ $var -le 1 ]; then
...
fi

## end of if-else statement

Conditional Statements
If-else statement:

echo "Enter a number:"

read num
if [ $num -gt 10 ]; then
echo "Number is greater than 10"
else
echo "Number is 10 or less"
fi
Scripting: If-else
## Problem: Create a shell script that checks if a file exists in the current
directory.

## Solution:

#!/bin/bash

file="example.txt"

# Check if the file exists

if [ -e "$file" ]; then

echo "File exists: $file"

else

echo "File not found: $file"

fi
Comparison Operators
Comparison operators:
● -eq (equal), -ne (not equal)
● -gt (greater than), -lt (less than)
● -ge (greater than or equal), -le (less than or equal)
Scripting: For Loop Structure

for VARIABLE in 1 2 3 … N; do
…
done

for VARIABLE in file1 file2 file3; do

…
done

for VARIABLE in `Linux command`; do

…
done
Loops in Shell Scripts
For Loop:
for i in 1 2 3 4 5; do
echo "Iteration $i"
done

While Loop:
count=1
while [ $count -le 5 ]; do
echo "Count: $count"
((count++))
done
Scripting: For Loop Example
## Problem: Write a shell script that calculates the sum of integers from 1 to N
using a loop.

## Solution:

#!/bin/bash

echo "Enter a number (N):"

read N

sum=0

for (( i=1; i<=$N; i++ )); do

sum=$((sum + i))

done

echo "Sum of integers from 1 to $N is: $sum"

Scripting: while loop
## Problem: How do you read lines from a file within a shell script?

## Solution

#!/bin/bash

file="inputfile.txt"

# Check if the file exists

if [ -e "$file" ]; then

while IFS= read -r line; do

echo "Line read: $line"

# Add your processing logic here

done < "$file"

else

echo "File not found: $file"

fi
Scripting: Switch Case
## Problem: Write script to build a simple calculator
#!/bin/bash
echo "Enter two numbers: "
read a
read b
#input type of operation
echo "Enter your choice: "
echo "1. Addition"
echo "2. Subtraction"
echo "3. Multiplication"
echo "4. Division"
read ch
#Acting on the user input
case $ch in
1) res=`echo $a + $b | bc`
;;
2) res=`echo $a - $b | bc`
;;
3) res=`echo $a \* $b | bc`
;;
4) res=`echo "scale=2; $a / $b" | bc`
;;
esac
#Printing result
echo "Result: $res"
Functions in Shell Scripting
Defining and calling a function:

function greet {
echo "Hello, $1!"
}
greet "User"
Defining a Function in Bash
function function_name { function_name() {
# Commands to execute echo "Hello from function"
echo "Hello from function" }
}
Calling a Function
function_name #!/bin/bash

# Define the function

greet() {
echo "Hello, Welcome to Bash Scripting!"
}

# Call the function

greet
Functions with Arguments
#!/bin/bash

# Function to add two numbers

add_numbers() {
sum=$(( $1 + $2 ))
echo "The sum of $1 and $2 is: $sum"
}

# Call the function with arguments

add_numbers 5 10
Functions with Return Value
#!/bin/bash

# Function to calculate square of a number

square() {
result=$(( $1 * $1 ))
return $result # Functions can only return numbers
}

# Call the function

square 6
echo "Square of 6 is: $?"
Using Function in a Loop
#!/bin/bash

# Function to check even or odd

check_even_odd() {
if (( $1 % 2 == 0 )); then
echo "$1 is even"
else
echo "$1 is odd"
fi
}

# Loop through numbers 1 to 5 and call function

for num in {1..5}; do
check_even_odd $num
done
Function with Switch Case
#!/bin/bash # Call the function
calculate 10 + 5
# Function to perform arithmetic operations calculate 20 - 8
calculate() { calculate 6 "*" 3
case $2 in calculate 50 / 2
"+") echo "$1 + $3 = $(( $1 + $3 ))" ;;
"-") echo "$1 - $3 = $(( $1 - $3 ))" ;;
"*") echo "$1 * $3 = $(( $1 * $3 ))" ;;
"/") echo "$1 / $3 = $(( $1 / $3 ))" ;;
*) echo "Invalid operation" ;;
esac
}
Function Calling Another Function
#!/bin/bash # Call the function
welcome "Alice"
# Function 1
greet() {
echo "Hello, $1!"
}

# Function 2 calls function 1

welcome() {
greet $1
echo "Welcome to Bash scripting!"
}
Example 1 - Check Disk Space
#!/bin/bash
Objective:
check_disk_usage() { Create a Bash script that monitors disk
usage=$(df -h / | awk 'NR==2 {print $5}' usage and alerts if usage exceeds a
| sed 's/%//') threshold.

threshold=80
Steps:

if (( usage > threshold )); then 1. Write a function check_disk_usage

echo "Warning: Disk usage is above that:
○ Uses df -h to check disk
$threshold%! Current usage: $usage%"
usage.
else ○ Extracts the usage percentage
echo "Disk usage is under control: of / (root partition).
Compares it to a threshold
$usage%" ○
(e.g., 80%).
fi ○ Prints a warning if the
} threshold is exceeded.
2. Call the function in the script and
schedule it using cron.
check_disk_usage # Call the function
Example 2 - Processing Files
#!/bin/bash
Objective:
rename_files() {
Write a script to rename all .txt
for file in *.txt; do
files in a directory and compress
mv "$file" "${file%.txt}_backup.txt"
them.
done
echo "Files renamed." Steps:
}
1. Write a function rename_files
compress_files() { that:
tar -czf backup.tar.gz *_backup.txt ○ Renames all .txt files
echo "Files compressed." by appending _backup.
} 2. Write another function
compress_files that:
○ Compresses all .txt
rename_files
files into a .tar.gz
compress_files archive.
3. Call both functions in the
script.
Function 3 - Network Tasks
#!/bin/bash
Objective:
ping_servers() {
Write a script to ping multiple
while read -r server; do
servers and log the results.
if ping -c 1 "$server" &>/dev/null;
then
Steps:
echo "$server is UP" >>
ping_results.log 1. Write a function ping_servers
else that:
echo "$server is DOWN" >> ○ Reads a list of servers
ping_results.log from servers.txt.
fi ○ Pings each server.
done < servers.txt ○ Logs success or failure.
2. Call the function inside the
}
script.

ping_servers
Function 4 - User Interaction
#!/bin/bash
Objective:
while true; do
Write a script with a menu-driven
echo "1) Check disk usage"
interface using case.
echo "2) List active users"
echo "3) Show system uptime"
Steps:
echo "4) Exit"
read -p "Enter your choice: " choice 1. Display a menu with the
following options:
case $choice in ○ Check disk usage
1) df -h ;; ○ List active users
2) who ;; ○ Show system uptime
3) uptime ;; ○ Exit
2. Use a while loop to keep the
4) echo "Exiting..."; break ;;
menu running until the user
*) echo "Invalid option! Try again."
exits.
;;
esac
done
File Handling
Reading from a file:

while read line; do

echo "$line"
done < myfile.txt

Writing to a file:
echo "This is a test" > output.txt
Scripting: Read input File
## Problem: Create a script that searches for a specific word in a file and
counts its occurrences.

## Solution:

#!/bin/bash

echo "Enter the word to search for:"

read target_word

echo "Enter the filename:"

read filename

count=$(grep -o -w "$target_word" "$filename" | wc -l)

echo "The word '$target_word' appears $count times in '$filename'."

Scripting: Use of Pipe
## Problem: Create a script that checks for and removes duplicate lines in a
text file.

## Solution

#!/bin/bash

input_file="input.txt"

output_file="output.txt"

sort "$input_file" | uniq > "$output_file"

echo "Duplicate lines removed successfully."

In-class Participation

Rewrite the Calculator script to handle all numerical situations such

as division or multiplication by Zero.
Lab Exercise
1. Go to your home directory
2. Create a new directory and name it “Lab6”
3. Go to the “Lab6” directory
4. Create a script file and name it “show_dirs.sh”
5. Program the “show_dirs.sh” to go over all the directories
under your home directory and print the following message
for each directory:
/home/username contains --> the_directory_name
then print the following line for “Lab6” directory
This is the last directory --> Lab6

6. Make your script file executable and run it.