Oslabs
Oslabs
Lab Manual
ver 2.0
Version History
1.0 Spring 2009: Installation, exercises spread over 11 lab sessions 1.1 Summer 2009: New set of exercises only spread over 11 lab sessions 2.0 Spring 2010: Compilation of all previous labs in book format with minor updates & corrections
This manual is customized for course cs206 Operating Systems taught at the National University of Computer & Emerging Sciences, Peshawar, Pakistan. The lab portion of this course provides cover for 1 credit hour out of total 4. Evaluation for this lab is at discretion of your instructor and/or teacher.
The manual can also be used as a learning tool for other operating system courses both at undergraduate or graduate level. Each chapter in this manual is intended to serve as a separate lab, although instructors should note that some chapter's require more than one lab sessions to complete. The manual is based on the Gentoo Linux Operating Systems version 2007.
Contents
I Installation & Introduction
1 Gentoo Installation
1.1 Operating System . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.1.1 1.1.2 1.2 1.2.1 1.2.2 1.2.3 1.2.4 1.2.5 1.2.6 1.2.7 1.2.8 1.2.9 Linux OS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Gentoo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Preparing Virtual Machine . . . . . . . . . . . . . . . . . . . . . . . . . . Disk Partitioning 1.2.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A Note on Linux Directory Structure & Partitions
7
9
9 9 10 10 10 11 11 13 13 14 14 14 15 15 15 17
Installation Steps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Getting the Linux Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . Conguring the Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . Compiling the Kernel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . System Conguration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
19
19 19 20 20 20 20 20 20 20 21 21 21 22 22 22 23 23 23 23 23
Practicing Commands
Text Editor
2.10 Searching
CONTENTS
2.11 GNU Compiler Collection 2.11.1 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 24 24 25 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
II System Calls
3 Processes
3.1 Understanding Processes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.1.1 3.2 Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
27
29
29 30 30 30 31 31 32 32 32 33 33 33 33 34 34 34 34
3.2.2 3.2.3
Sleep() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2.4
Termination States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2.4.1 3.2.4.2 3.2.4.3 3.2.4.4 Exit() Call Atexit() Call Abort Call . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
Kill Call . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3
Death of Parent or Child . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3.1 3.3.2 Parent Dies Before Child . . . . . . . . . . . . . . . . . . . . . . . . . . . Child Dies Before Parent . . . . . . . . . . . . . . . . . . . . . . . . . . .
4 Input/Output
4.1 4.2 4.3 4.4 Open a File Close a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
37
37 38 38 39
Writing to a File
5 IPC: Signals
5.1 Signal Delivery Using Kill 5.1.1 Kill Command 5.1.1.1 5.1.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
41
41 41 41 41 42 42 42
Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
5.2
6 IPC: Pipes
6.1 6.2 Pipe On the Shell . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Pipe System Call 6.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
43
43 44 45
Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
CONTENTS
III Threads
7 POSIX Threads
7.1 7.2 7.3 7.4 7.5 Thread Basics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Thread Creation 7.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Passing Multiple Arguments to Thread . . . . . . . . . . . . . . . . . . .
47
49
49 49 50 50 51 51 52
53
55
55 55 55 55 57
V Memory Management
9 Memory
9.1 PROC File System 9.1.1 9.1.2 9.1.3 9.1.4 9.2 9.2.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Exploring PROC Processes in PROC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Process Memory in PROC . . . . . . . . . . . . . . . . . . . . . . . . . . Overall Memory Usage in PROC Logical and Physical Addresses 9.2.1.1 9.2.1.2 9.3 9.3.1 9.3.2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
59
61
61 61 61 62 62 62 62 63 63 63 63 64 64 64 64 65 65
Exploring Memory
Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Memory Allocation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9.3.2.1 9.3.2.2 9.3.2.3 9.3.2.4 Malloc() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Free() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Realloc()
Calloc() . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
9.3.3
Exercise
CONTENTS
Processes Scheduling Inter-process Communication Synchronization Memory Management (physical memory allocation, virtual memory etc.) Resource Management Directory and File Management Communication
Application Software
lem. cic.
Application software
is a generic term used for such types of software which are used to solve a probThe problem to solve is user speWord processors, computer games,
1.1.1 Linux OS
Linux is an operating system which is a avor of Unix. It is a multi-user and multi-tasking operating system. It was developed by Linus Torvalds at the University of Helsinki in Finland. The interface between user interLinux and it's users is known
System Software
grams which are designed to operate, control, and extend the processing capabilities of the computer operating system itself, e.g. compilers, linkers etc.
1.1
Operating System
it.
acts with the Linux operating system through Operating System is a special type of System Software and is a collection (set) of programs, which performs two specic functions. First, it provides a user interface so that human user can interact with the machine. Second, the operating system manages computer resources. Computer resources are physical devices, which an operating system accesses and manages. Printers, memory, CPU, input/output devices, les, etc., are examples of computer resources. Some of important tasks, which a typical modern operating system has to perform, are given below: This is equivalent to what you have read about the command interpreter in your lectures. There may be two tasks to be performed by a shell. First, accepts commands from a user and second, interprets those commands. The core of the Linux OS is the kernel. As long as the system is operational, the kernel is running. The kernel is the part of the Linux Operating system which consist of routines, which interact with underlying hardware, and routines which include system call handling for process management, tem, etc. memory management, process scheduling, communication, le sys-
10
1.1.2 Gentoo
Linux operating systems are presented in dierent avours called distributions. One avour of Linux is the Gentoo Operating System. Other popular distributions are Red Hat, Suse, Fedora, Knoppix, Ubuntu, etc. Each distribution has it's own set of specialities and functions. The standing point of Gentoo is that it is a source based distribution. A source based distribution means that the complete OS is built & compiled from scratch from the provided source code. This provides remarkable customization power to the end-user. Gentoo users will use the following core steps to install their operating system via a LiveCD (Bootable CDROM). 1. Partition hard disk 2. Format partitions 3. Copy source code les to HD 4. Copy portage package management tree to HD 5. Congure Gentoo Kernel 6. Compile Gentoo Kernel 7. Compile Important System Software such as Bootloader 8. Congure Bootloader 9. Reboot. Note that installation of a graphical interface is not part of the core steps of Gentoo OS installation.
as a substi-
tute for installation portion in this manual. All software and les are shared on //192.168.1.10 or will be provided by your instructor.
Install VMWare Workstation on your machine. Once installed, create a new virtual machine with the following specications:
Virtual Machine Conguration: Custom Virtual Machine Format: New Workstation 5 Guest Operating System: Linux
Linux (Location: E:\Gentoo) Number of Processors: One Memory: Use default selected (256
Mb would be more then sucient) Network Connection: NAT I/O Adapter Types: SCSI, BusLogic Disk: Create a New Virtual Disk Virtual Disk Type: SCSI Disk Capacity: 6~8 GB (Click on
Go to Edit Menu and select Preferences. Then click on the Hot Keys tab, where CTRL+ALT will be selected. Instead of this, select CTRL+SHIFT+ALT.
In the devices section of your Gentoo VM, double click on CD-ROM. Select Use ISO Image and give it the location of the
2007.livecd.ISO.
Start your VM by clicking on the Play button from the menu. This will start your VM machine. Click inside the window and your mouse will be locked inside the virtual machine. By pressing CTRL+SHIFT+ALT, you can release the mouse back to the physical machine.
1.2
Installation Steps
The following set of instructions/commands are customized according to facilities in Khyber lab. The installation will be carried out For installation at in a virtual environment.
1 http://www.gentoo.org/doc/en/handbook/handbookx86.xml
You will be confronted by your VM boot up screen. When you receive the prompt boot:, type gentoo nox and press enter. Your livecd should now boot.
11
We are going to require 3 disk partitions for Gentoo. Disk partitioning can be carried out using the
type (L) to view the list of partition codes. Type (82) for choosing swap type
fdisk
tool
fdisk /dev/sda
The following options are useful for carrying out partitions: pprint partition structure ncreate new partition ddelete existing partition atoggle partition bootable tchange partition type lto view partition type hex codes wwrite partition table changes to disk
press n select p for partition type (primary) give partition number (3) select partition start sector (press enter for default) select partition size (press enter for default)
Check against Figure-1.1 and make sure that your partition structure resembles it.
press n select p for partition type (primary) give partition number (1) select partition start sector (press enter for default) select partition size (type +32m for 32MB) press a select partition (1). it as bootable This will mark
Write down the partition table structure on paper and keep it as a reference. Specically note down what is the device name of each partition (1st column), it's type (last column) and it's size (which you have specied). mapping: We are eventually going to use these partitions for the following
press n select p for partition type (primary) give partition number (2) select partition start sector (press enter for default) select partition size (give +512m for 512MB) press t select partition number (2)
Type w and press enter. This will commit the partition table changes to disc.
1.2.2.1
partition
mary,
extended
primary
12
/ (The top most directory is called the root. Its name is simply /)
Home
/var for mail server related les /opt for game servers /bin and /sbin for system les /dev contains information about congured i/o devices /lib contains library and API's /tmp is a location where les are located temporarily. These are deleted if not needed /etc for conguration les, all system administrator commands /root is the working directory for the root user (administrator is called root in linux) /usr contains most of the large applications, including portage /boot for storing the boot information for linux (and other os). Will store lilo and grub records as well.
is four. An
has to be a
primary partition (but a special one). It is special because it serves as a container for other partitions. A resides inside an extended partition. Its declarations are not stored in the MBR but rather in the Extended Partition. A le is simply a storage mechanism for a type of information. It can be in text format or binary format. A directory is also a le but a special kind of it. It may contain information about les, as well as other directories. Along with the information stored inside a le, there is also some information about the le. For example, the date the le was created, the size of this le, its permissions, etc. These are called the attributes of the le. Some examples are given below: 1. File name 2. File type (text, binary, image, executable, etc.) 3. Creator/Owner 4. Date created, accessed, modied 5. Size of le 6. Permissions of le Unlike DOS or Windows, which permit you to organize your disks anyway you please, the Linux le system is organized into a standard directory structure. Anything which is not in the right place may result in a broken system. The directory tree is given in Figure-1.2. description of some of the folders is as such: A
When working through the directory structure, you will frequently come across the term
name.
path-
/home/omar/code/source
Here, the rst slash on the left indicates the root directory. Within the root directory, we can nd the home directory, then moving down to sub-directory omar, then code, and nally to the le called source. So the portion
/home/omar/code is the path, and the le is the source. Accessing a le this way is referred to as Absolute Path.
Each le has to be accessed in accordance with it's corresponding path location. If
./runme
on
13
The stage archive usually does not appear in any Gentoo LiveCD's (unless you are using a liveDVD). It's size is roughly 130 MB so you cannot download it through FAST proxy servers. virtual environment. Press CTRL+SHIFT+ALT to exit from your Obtain the stage3x86-2007.0.tar.bz2 from //192.168.1.10 or your instructor and copy it onto your USB ash. ash. Once copied take out your USB
../runme
on the shell.
Here, the double dots indicate a parent direcAccessing a le or location this way is referred to as the
Relative Path.
We are going to format the /boot partition to ext2 lesystem and the root (/) partition to ext3 lesystem.
Enter your Virtual machine again by clicking inside it. Once inside, insert your USB ash again. rst. In order to access the contents of USB ash, we need to mount it
/mnt/usb.
Gentoo
Base
Now that it is mounted, we can extract the contents of stage archive to location
nt/gentoo.
/m-
(on a single line) with a space separating each parameter The process of
We are going to mount the partitions to it's relevant location. mounting simply takes a physical device and links it somewhere on the le system. In the following commands, we are using the
mount command to mount the sda3 device on location /mnt/gentoo, and the sda1 device on /mnt/gentoo/boot. But since /boot has not been created yet, we create it using the mkdir command.
mount /dev/sda3 /mnt/gentoo mkdir /mnt/gentoo/boot mount /dev/sda1 /mnt/gentoo/boot
Coming to the portage tree, that is available by default in the LiveCD image. We can extract it from there. Again note that following should be on a single line.
Now, all the required Gentoo base system les are in their relevant partitions, we can switch from the LiveCD environment to the actual Gentoo environment on disk.
Gentoo base system requires copying over of two main les. The rst is the archive and the second is the
stage portage
tree. The stage archive is simply a compressed le containing the basic skeleton of directories and les in the linux le system as shown in Figure-1.2. The portage tree is a feature of Gentoo through which you can install extra applications and tools (just like the equivalent of Add/Remove programs in Windows).
mount -t proc none /mnt/gentoo/proc mount -o bind /dev /mnt/gentoo/dev chroot /mnt/gentoo /bin/bash env-update source /etc/profile
14
The kernel that we are going to install is called the gentoo-sources. options. To obtain the gentoo-sources source code, you have two
make.conf
nano /etc/make.conf
Option 1: Since you have already set-up your network settings, you can download it straight from the internet. Run the
emerge
portage tree.
emerge gentoo-sources
Option 2: If you are already using your USB in virtual machine, unmount it using:
umount /mnt/usb
Press CTRL+ALT+SHIFT to exit from your virtual machine. Plug in your USB and copy over the gentoo-sources.tar.bz2 le from //192.168.1.10 or from your instructor into the USB.
In this installation you may be required to download some packages from the internet. For this, you have to congure your network settings.
Now, enter your virtual machine again by clicking inside it. your USB again. Once inside, Plug in Mount it as you have
ifconfig eth0
We are going to copy over our gentoosources le from USB to distles directory. But we have to create this rst.
from
the
output
something
inet
addr:192.168.X.XX
be some IP address, then your device is recognised. If not, you have to specify it manually using the following command (all one one line).
Once we have the gentoo-sources le, we can simply install it using the mand.
emerge com-
emerge gentoo-sources
menucong
we are going
utility at
/usr/s-
This will load a blue screen where you can navigate through many options in the list. Each entry that you see in the list will have any of the three identiers:
15
Press CTRL+X to exit and save changes. Next step is to choose your root password. In Linux, the administrator account is called
root.
/.
into the
passwd
Finally we are going to install some important system applications. Syslog is used for logging purpose, vixie-cron is the equivalent of windows-scheduler, and slocate is used for le searching. The command rc-update adds the program to the start-up list of processes to run when the OS boots.
[M ] Feature
into the kernel (supported as module) We will browse through the list one by
Warning: Don't touch the features that are already congured. Instead, make the following additions). To go through the
to the following specication. ( list, please refer to Figure-1.3. Once nished with the settings, go back to the root menu and select
<exit>.
emerge syslog-ng rc-update add syslog-ng default emerge vixie-cron rc-update add vixie-cron default emerge slocate
The purpose of the bootloader is to load the operating system image from disk. It resides in the master boot record (MBR) of the hard disk. The default bootloader program for linux is
Once the kernel is compiled, we need to copy the kernel boot image to the directory.
/boot
grub.
emerge grub
After it is installed, we need to manually congure the list that will appear from which we can start our operating system. This list is congured in the
cp arch/i386/boot/bzImage /boot/bzImage
The rst thing to do is to tell the OS what directory each partition is going to represent. This is done in the using
grub.conf
le.
nano /boot/grub/grub.conf
Make the le conform to the following. Once done, press CTRL+X to exit and save changes.
nano.
fstab
le. Edit it
nano /etc/fstab
default 0 timeout 30
1 0 0 0 2 title Gentoo Linux 0 1root (hd0,0) 0kernel /boot/bzImage root=/dev/sda3
16
The list is prepared. We need to copy it over to the MBR. This is done by running the grub shell. Invoke it using the following:
17
grub --no-floppy
In the grub shell (once it starts) Use the following commands in sequence:
1.2.11 Finalizing
Exit everything to go back to the LiveCD environment. From there, we can give the reboot command to reboot the system.
exit reboot
When the Virtual Machine starts again, go into it's BIOS and change the boot device priority. The Hard Disk should be the rst device in the list.
rity
Secu-
it full priviledges.
18
2.1
User Accounts
1
. So we are going to follow
Another way to access your bash shell is through a remote login client. To login remotely, you need to install SSH Server on your operating system. This is as simple as giving the following commands:
Your mother has probably told you never to work as root user our mother's advice and create a new username and password for ourselves.
<username>@localhost ~ $ as the prompt. The tilda ~ is the shortcut to your home directory (/home/<username>).
2.2 Your Shell
ssh 192.168.1.999
From windows, we can connect to a linux bash shell using
putty.
from //192.168.1.10.
the IP address of the remote shell and other details like port number (22), username, and password. Whichever method you choose, Linux will automatically rectory (By place default you in your home diare home directories
The default shell that users get to use when they install linux is the and remotely.
bash
19
20
2.3
A command is a request from a programmer, an operator, or a user to Linux operating system asking that a specic function be performed. For e.g., a request to list all les in your current directory will be the command
ls ab*
For exam-
ls.
2.4.3 Auto-Completion
Auto-Completion is a short-cut feature for quickly entering commands that are long or you have forgotten their spelling of. To practice, just type in a keyletter f and press the TAB key. You will see the list of all commands starting with an f. Type d and press TAB, you will see a list of all commands starting with fd. Type i and press TAB, you will see a list of all commands starting with fdi. So on and so forth. You can also use the auto-completion to detect directories. For example, you want to access the home directory of a user who for some strange reason is called
A command tells the operating system what to do (what action to be performed, copy a le, display a date etc.)
Option(s) tells the way of action to be performed. For example, ls command displays directory contents, and r option tells the way in which the directory should be displayed. Here r displays directory contents in reverse (alphabetically) order.
Argument tells that on what objects (le, directory, devices, etc.) the command and its arguments are applied. For example if we need to display all les starting with alphabet a, you will give ls a* and press enter.
vwxyz.
abcdefghijklmnopqrstuwill
bcdefghijklmnopqrstuvwxyz
be given automatically and you will be spared the time and eor of writing such a large name.
Note
always a space between the command, the options, and the arguments.
2.4.4 Redirection
You can use the > and < symbols to redirect your output. such: The types of redirection are as
2.4
redirection to a le as > output redirection to a le redirection from le to termi-
<Output
nal
delete all les in a current directory. With Linux, you can use rm * to do the same thing.
21
2.5
Practicing Commands
another# ls -a another# ls -l another# ls -lh another# cp newest newestest another# cat newestest another# another# cd .. temporary# mv newester another/newester temporary# ls temporary# ls another/n* temporary# cd .. # rm temporary # rm temporary/* # rm temporary # rm temporary -r -f
Easy? Okay try and attempt the following ex-
Some of the most commonly used commands are given below. Try and practice each one of them and see what they do.
ercise.
ls
2.5.1 Exercise 1
Implement the following directory tree.
Print the list of directory contents of the current directory (or any other directory if full path is specied)
cd
Change directory to another directory Create a new directory Remove a directory (if it is empty)
View contents of a le, or write contents to a le. Copy a le from one location to another Move a le from one location to another Remove le(s) and/or directory(ies) The boxes in blue are directories. The boxes in gray are les. Each le should contain a random mark of your liking. Once done, delete all the les/directories that you have created.
To see them working, practice the following set of commands. The prompt.
Count the total number of commands you entered to do this job. I managed using 6 commands.
# mkdir temporary # cd temporary temporary# ls temporary# cat > newfile Type any text and press CTRL+D temporary# cat newfile temporary# mkdir another temporary# cp newfile another/newest temporary# cp newfile newester temporary# cd another another# ls
2.5.2 Exercise 2
Consider the following directory tree:
22
Change your directory so that your current directory is Physics. From here, change your directory in only one command such that your current directory becomes subject.
nano <myFile>
Near the end of your screen you will see a list of shortcuts. The ones which you should get yourself familiar with are as such:
From
subject,
issue
only
one
com-
for exit for saving for searching for cutting for pasting for displaying cursor position
2.6
Text Editor
A text editor is a software where you can enter text in its native format and save it to le. A word processor is a software where you can take text and process its appearance, format, spell-check, paragraph settings, etc. Linux has a number of text editors (both graphical and command-line based). The one which you will use most commonly is the nano text editor.
2.7
Links
nano
At the moment you will have access editor. Nano is an advanced text
Links are the equivalent of Shortcuts in Windows. The syntax of creating a link to a le or directory is as such:
nano
ln existingfile linkname
To practice, try out the following commands:
nano
You may also start your nano by explicitly mentioning the le you want to work on. This le may be already existing or you may be creating a new one.
# # # # #
2.10. SEARCHING
# # # # # # # # # ln -s blankdir dirpointer ln blankfile filepointer cat < blankfile cat < filepointer echo "Hello dear" > filepointer cat < blankfile cat < filepointer ls ls -lh
23
q.
Look carefully at the contents of both lepointer and dirpointer. Changing one will automatically change the other. Also look at how the directory pointers appear using ls -lh.
down
2.8 Manuals
The most important thing to get yourself familiar with in linux is the
up
and down arrow keys. An alternative is using output redirection that you have covered in section-2.4.4. Usage would be as such:
manuals.
man
Usually a user has to specify a manual page. A manual page is usually listed by it's program name, command name, or system call name. For example:
2.10
Searching
ls cp man fork
or
arrow keys.
nd
command.
An-
other powerful search program that we installed when installing gentoo was
slocate.
To search, type / (slash), followed by your search string, and enter To exit, type q
2.9
More | Less
Sometimes a user may give a command which generates so much output that it scrolls very fast. As a result, the top information simply For example, use ows out of the screen whereas only the lowend information is visible. the following command
ls /bin -lh
We can view the lost information using the more or less commands. Try both of them rst.
24
2.11
Write 1 1 0
21
Execute 1 0 1
20
Prorams written in C on linux are compiled using the gcc compiler. Programs written in C++ are compiled using the g++ compiler. Both these compilers are provided by GNU under the label GCC. Any C program written for linux should have the extension of .c (e.g., hello.c), whereas a C++ program should have an extension of .cpp (e.g., hello.cpp). When compiling, the general syntax of command is as follows:
2.11.1 Exercise
myFirst.c.
Your program
should prompt the user for his name, and then should display the name on the shell. Tip: Use the printf() and the scanf() calls for this purpose & remember to check their respective man pages as well (man printf & man scanf )
gcc|g++
To run a linux command from your program, we can use the system() function. Use the following inside your myFirst.c program and see it's output.
rst.c|rst.cpp []
tension of your source code. Anything inside this square brackets is an optional arguments. mand. Do not write the square brackets themselves in your com-
system("ls");
-o
is the output lename. the name you specify. cutable le named
ment for making an executable le with source code will be compiled into an exe-
2.12
Create a directory with the name test in your current directory. Then display the contents of that directory Finally erase the test directory
a.out
rst
pass to -o argument. So considering that we have a source code with the name rst.c, and compile it with the above command, we can execute it using:
File Permissions
All les and directories in Linux have an associated set of owner permissions that are used by the operating system to determine access. These permissions are grouped into
./first
Where ./ species the current directory, and rst is the name of executable le which you passed as an argument to the compiler. Try it out using the following code:
three
group, everyone else} whereas the bits represent {read, write, execute}. So overall, we have 9 permissions (See Table 2.1). If we look at the rst row for Owner, we see three 1's. Which specify that the Owner Since all of the bits are in has read, write, and execute permissions on a le or directory. allow mode, we can add them up together to
25
22 + 21 + 20 = 4 + 2 + 1 = 7.
Similarly,
the second row for Group, i.e., users within the same group as that of the le owner, have read and write, but no execute permissions. This 2 1 will be translated only as 2 + 2 = 4 + 2 = 6. And lastly, every other user can only execute les and have no permission to read or write 0 to them. This will be translated as 2 = 1. So the le permissions would be command:
761.
To give the
ls -lh
2.13
mand:
File Ownership
<username>
<lename>
26
27
Chapter 3 Processes
A system call is an interface between your program and the kernel. The linux kernel's job is to provide a variety of services to application programs and this is done using the provision of system calls.
STAT:
ruptible sleep (R) Running or ready (S) Interruptible sleep (T) Stopped (Z) Zombie (<) High Priority (N) Low Priority (s) Session leader, i.e., has child processes (l) multi-threaded (+) foreground
3.1
Understanding cesses
Pro
The fundamental building block of each program is the process. A process is the name
given to a program when it is loaded for the purpose of execution on the operating system. For a full description of what is comprised inside a process, please refer to your class-notes or slides. To view a list of current processes in the system for a user, you may use the ps command. The basic attribute of a process is it's ID (PID), and it's Parent ID (PPID). The system calls that can nd the process ID, and the Parent Process ID are the getpid() and getppid() calls respectively. To use both of them, you will be required to include the and
COMMAND:
The program/command
ps
To view the complete list, you can adapt it to:
unistd.h
sys/types.h
C libraries.
ps au
You will see plenty of columns as output. The columns you should be familiar with at this moment are underlined below:
The mere presence of the PPID means that there is a hierarchy of processes inside our operating system. Each process has a track of who are it's children, and each child process has a record of who is it's parent. The original process that is the grand-father of all processes The owner of that process The integer identier Percent utilization of CPU Percent utilization of Memory This command will show a list of all processes currently in the system in the form of a tree. is the
Init
pstree
MEM:
space
29
30
CHAPTER 3. PROCESSES
Q2
Increase the value in for loop from i<1 to i<2 (i.e., 2 iterations in the loop). Compile and run your program. How many processes does it show this time? Draw a tree hierarchy of processes that you just created as given in Figure-3.1.
3.1.1 Exercise
With respect to the myrst.c le that you created in last labs, write a code that is able to nd out the following:
The PID value for myrst.c The PPID value for myrst.c The process name from the PPID value. (Note: You can use the system() call for this purpose. To nd a process name from PPID, you would normally enter the following command on the shell, where 12345 is the ID of any process)
Q3
Increase the value again to i<3 (i.e., 3 iterations). Compile and run your program. How many processes does it show? Draw a tree again. Why is it that we have called fork() 3 times in our code, yet we are seen ing 2 1 processes listed on screen?
Q4
For fun, increase the value yet again to 100. Compile and run. What is going to happen? Does your OS Crash? Does your Program Crash? Can you modify your code to count the total number of fork()s made?
3.2
Process Lifecycle
Each process has to go through the following states during it's existence: 1. Creation 2. Running 3. Non-Running (a) Ready (b) Waiting 4. Termination
01 02 03 04 05 06 07 08 09
#include <stdio.h> #include <sys/types.h> #include <unistd.h> int main(void) { fork(); printf("He\n"); fork(); printf("Ha\n"); fork(); printf("Ho\n"); }
Note that we are calling each He, Ha, and Ho only once. Yet that does not appear when the program is run.
Q5
Explanation
The Fork() system call simply creates a new process which is exact replica of parent process. It requires the as such:
#include <stdio.h> #include <sys/types.h> #include <unistd.h> int main(void) { int i; printf("Process PID %6d \t PPID %6d \n", getpid(), getppid()); for (i = 0; i<1; ++i) { if (fork()==0) printf("Process PID %6d \t PPID %6d \n", getpid(), getppid()); } return 0; }
unistd.h
Q1
// Fork code 1 #include <unistd.h> #include <???> // See Question 2 int main() { int p; p = fork(); printf("Job Done\n"); }
31
Question1
answer.
man fork
Why for
Question2 Question3
screen.
library is used for this call? Run your program. Why is it that
if (p == 0) { printf("Child PID = %d, PPID = %d\n", getpid(), getppid() ); } else { printf("Parent PID = %d, Child ID = %d\n", getpid(), p); }
printf() is used only once, yet we see the output Job Done displaying twice on our What would happen if we don't use the If/Else conditions Add the following statement to and immediately write the two printf() statements? Make sure you understand the structure of the Code, as well as what are the ways of knowing the following:
Question4
the end of your code and run it again. What output would you see?
1. The PID
3.2.1.1
Exercise 1
call code 2
above and add a
system call for sleep() after both the printf() statements. Give a time of 120 seconds to the sleep call. While both the parent and child are sleeping, open a new terminal and check out the outcome of the
ps
pstree
command as well as
command.
parent and child process in both commands, especially noting the STAT column.
3.2.1.2
Exercise 2
// Fork code 2 #include <unistd.h> #include <???> // For printf() int main() { int p; printf("Original Process, pid = %d\n", getpid() ); p = fork();
Model a fork() call in C/C++ so that your program can create a total of EXACTLY 6 processes (including the parent). (Note: You may check the number of processes created using the method from Exercise 1 (Section-3.2.1.1 by using a sleep of 60 seconds or more and entering eithe ps, or pstree in another terminal). Your process hierarchy should be as follows:
32
CHAPTER 3. PROCESSES
printf("Child Process\n"); } else { printf("Parent Process\n"); }
}
Figure 3.1: Exercise 2 Model
We have referred to
Exec() system call but in code we see Execv(). Read man pages for this and nd out. To help
execv() call? What is it's contents? What is the 2nd argument to it?
What is it's contents? What is arg? Look at the code of the child proHow many times does the
man 3 exec
const char
int execlp(const char *le, const char *arg, ...); int execle(const char *path, const char *arg, char *const envp[]); int execv(const char *path, char *const argv[]); int execve(const char *path, const char *argv[], char *const envp[]); int execvp(const char *le, char *const argv[]);
cess (p==0).
exec() system calls would be represented in Figure 3.2, where the dotted line mark the execution and transfer of control whereas the straight lines mark the waiting time.
#include <unistd.h> 3.2.3 Waiting States #include <stdio.h> int main() 3.2.3.1 Sleep() { int p; The sleep() call can be used to cause delay in char *arg[] = {"/usr/bin/ls", 0}; execution of a program. The delay can be prop = fork(); vided as an integer number (representing number of seconds). Usage is simply sleep(int), if (p == 0) { which will delay the process execution for int printf("Child Process\n"); seconds. To use sleep(), you will require the execv(arg[0], arg); unistd.h C library.
33
{ }
3.2.4.2
The
Atexit() Call
below provides two functions;
code
atexit() and exit(). Note the structure of code and the behaviour of execution and then attempt the questions in the end.
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19
#include<stdio.h> #include<stdlib.h> int main(void) { void f1(void), f2(void), f3(void); atexit(f1); atexit(f2); atexit(f3); printf("Getting ready to exit\n"); exit(0); } void f1(void) { printf("In f1\n"); } void f2(void) { printf("In f2\n"); } void f3(void) { printf(In f3\n); }
Q1
What is the dierence between exit() and atexit()? What do they do? (Check
atexit
and
man 3 exit ).
man
3.2.4.1
Exit() Call
Q2
What does the 0 provided in the exit() call mean? What will happen if we change it to 1? (Check manual page for exit)
The exit() system call causes a normal program to terminate and return status to the parent process. Study the behaviour of the following program. You will see that there may be a Can number of exit points from a program. time a program exits??
Q3
If we add an exit call to function f1, f2, or f3. What will happen to execution of our program?
Q4
Why do you think we are getting reverse order of execution of atexit calls?
#include <stdio.h> 3.2.4.3 Abort Call int main() { Type, run and execute the code below. Then int num; answer the questions in the end. void anotherExit(); // Function Prototype printf("Enter a Number: "); 01 #include <stdio.h> scanf("%i", num); 02 #include <stdlib.h> if (num>25) 03 int main() { 04 abort(); { 05 exit(0); printf("exit 1\n"); 06 } exit(1); } Q1 Check the man pages for abort. How else does the abort call terminate the proanotherExit(); gram? What is the name of the particular } void anotherExit() signal?
34
CHAPTER 3. PROCESSES
Execute your program. What is the output of our program?
Q2 Q3
Include the abort call in function f3 in our code provided for Atexit() call. How does our program terminate using this?
3.2.4.4
Kill Call
But as a demonstration, you
Full description of this call will appear in Section-5.1.2. call works. may run the following code to see how the kill()
#include <stdio.h> #include <sys/types.h> #include <signal.h> int main() { printf("Hello"); kill(getpid(), 9); printf("Goodbye"); }
You are already familiar with getpid() system call. To nd out what 9 is, rst look at the output of the command:
#include <unistd.h> #include <stdio.h> int main() { int i, pid; pid = fork(); if (pid > 0) // Parent { sleep(2); exit(0); } else if (pid == 0) // Child { for (i=0; i < 5; i++) { printf("My parent is %d\n", getppid()); sleep(1); } } }
Run the code. 1. What are the PPID values you are receiving from the for loop? 2. What has happened when the numbers of the PPID change? 3. What is now PID of the init process?
kill -l
Find out the word mentioned next to 9. Search on the internet for this.
3.3
Death Child
of
We covered the process termination using the exit() system call and that a program may have more than one exit points. In a relationwhip where a process has spawned children, what would be the eect if either the parent dies or the child dies on the other processes linked to it? This section will look and study such a development.
#include <unistd.h> #include <stdio.h> int main() { int i, pid; pid = fork(); if (pid > 0) { sleep(120); } else if (pid == 0) { exit(0); } }
// Parent
// Child
Run the code. In another window (terminal), check the status of your parent and child processes using the ps command. You should be able to see a Z, or <defunct> next to the child process which has been created. Such a process is usually termed a Zombie process.
35
36
CHAPTER 3. PROCESSES
Chapter 4 Input/Output
Probably the rst thing you covered in I/O when you were doing your C/C++ programming courses was
echo "Hello"
Of these,
cout displays something on the standard output (display screen), whereas cin is used for obtaining input from keyboard input device. In linux, there are three types of les that are open all the time for input and output purposes for each process. These are: 1. Standard Input Stream 2. Standard Output Stream 3. Standard Error Stream Each of these streams are represented by a unique integer number called a
Hello
to a le
hello.txt
using the output redirection method that has been covered in Section-2.4.4. You can view the contents of this le to ensure that the string has indeed been written to it.
In previous bullet,
hello.txt
was a le. We
are going to use the same mechanism but a little dierently. Replace X with the PID value that you noted down earlier. in later sections. You will see what /proc directory is used for We are specifying that we want to write to le descriptor = 1 of process ID marked by X.
tor.
File Descrip-
output is 1, and standard error is 2. Other les that are in use by a process will be assigned le descriptor numbers of 3, 4,
. . ..
scriptors refer to each and every instance of an open le for a process. So, if we want to open a le, close a le, read from a le, or write to a le, it has to be done through it's corresponding le descriptor. To visualise how this works, we are going to perform a simple exercise. For this, we would require two shells.
Now go back to the previous terminal by pressing CTRL+ALT+F1. Surprise! The string Hello has been written there.
4.1
Open a File
It's
Type the following command and note down the current bash shell PID. Let this be
We use the open() call to open a le. Open() can also be used for creating a new le. syntax is as such:
X.
ps
Press CTRL+ALT+F2 to open a new terminal window and login with your details. The echo command is used to display a line of text.
sys/stat.h
and
fcntl.h
sys/types.h,
C libraries. As parame-
37
38
CHAPTER 4. INPUT/OUTPUT
4.2 Close a File
So we saw that the open() call returns an integer number (the le descriptor) which has been stored in
1. Path name of the le to open 2. Flag specifying how to open it (i.e., for read only, write only, etc.) 3. Access permissions for a le (Provided the le is newly created) Some of the ags are listed below:
fd
what we are doing with the le, the le should be closed. by default closes all le descriptors so you may not close a le by yourself if you don't want to. But if you do, then read on. Linux uses a total of 1,024 le descriptors per process by default. So it's a good idea that you close your le descriptor's if they are not used. To close a le descriptor, just use the following in your code:
O_RDONLY for marking a le as read only O_WRONLY for marking a le as write only O_RDWR for marking a le as read and write O_CREAT for creating the new le O_EXCL for giving an error when creating a new le and that le already exists
close(fd);
Look at the following code and see what it is the output?
int main(int argc, char* argv[]) { char *path = argv[1]; int fd = open(path, O_WRONLY | O_EXCL | O_CREAT); if (fd == -1) { printf("Error: File not Created\n"); return 1; } return 0; }
Compile this, but when running specify the command as such:
int main(int argc, char* argv[]) { if (argc != 2) { printf("Error: Run like this: "); printf("%6s name-of-new-file\n", argv[0]); return 1; } char *path = argv[1]; int i = 0; while(i<2) { int fd = open(path, O_WRONLY | O_CREAT); printf("Created! Descriptor is %d\n", fd); close(fd); i++; } return 0; }
Comment out the line pile and run again. ferent values?
ls
Give the same command again and check out the output.
4.3
Writing to a File
Question
Writing to a le is done using the write call. What is the size of the le? Why To write, we should obviously open a le rst. The syntax of the write() call is as such:
is it this size?
39
4.4
unistd.h
C library. The
Read() system call is going to be used for this purpose. It's syntax is as such:
The le descriptor (must be open ... otherwise how are you going to write to a notopen le?)
unistd.h
C library. the
parameters for read() are somewhat the same as that for write(). I.e.,
So let's see the write call in action. Type, compile and run the following code. Then answer the questions at the end.
The le descriptor (must be open ... otherwise how can you read from a not-open le?)
So let's see the read in action. Type, compile, and run the following code:
char* get_timeStamp() { time_t now = time(NULL); return asctime(localtime(&now)); } int main(int argc, char* argv[]) { char *filename = argv[1]; char *timeStamp = get_timeStamp(); int fd = open(filename, O_WRONLY | O_APPEND | O_CREAT, 0666); size_t length = strlen(timeStamp); write(fd, timeStamp, length); close(fd); return 0; }
Q1 Q2
What is 0666 that is specied in the open() call? What does it mean? What is O_APPEND doing in the same call? Run the program again and check it's output.
Q3
Modify the following line in the code and then compile and run the program and check it's output. What has happened?
int main(int argc, char* argv[]) { if (argc != 2) { printf("Error: Run like this: "); printf("%6s name-of-existing-file\n", argv[0]); return 1; } char *path = argv[1]; int fd = open(path, O_RDONLY); if (fd == -1) { printf("File does not exist\n"); return 1; } char buffer[200]; read(fd, buffer, sizeof(buffer)-1); printf("Contents of File are:\n"); printf("%s\n", buffer); close(fd); return 0; }
40
CHAPTER 4. INPUT/OUTPUT
kill -l
Hence, supposing that we want to terminate a process, we may enter
5.1
Signal Kill
Delivery
Using
kill -9 12345
Where 12345 will be a process id of any active process in the system.
Kill is the delivery mechanism for sending a signal to a process. Unlike the name, a kill() call is used only to send a signal to a process. It does not necessarily mean that a process is going to be killed (although it can do exactly that as well). As mentioned earlier, the signal facility is just an event notication facility provided by the operating system. For e.g., a shutdown signal (SIGHUP) can be sent to all processes currently active in the system in order to notify them about a shutdown process event. Upon receipt of this signal, all processes will prepare to terminate. Once the processes terminate, the system can shutdown. The Kill facility can be used in two ways; as a command from your command prompt, or via the kill() call from your program.
5.1.1.1
Exercise
Let us kill our bash shell using the TERM signal. Find out:
The
integer
representation
for
the
SIGTERM signal The PID of your current active bash shell using the ps command Then, use the kill command to send over this signal.
kill(int, int);
For this to work, we will require the &
signal.h
sys/types.h
kill -s PID
Here, kill is the command itself, -s is an argument which species the type of signal to send, and PID is the integer identier of the process
is the integer number of signal type (again, can be checkable from eter is the PID of the process to which signal is
to be delivered. As an example, if we want to send the Terminate signal to the current process, we will use
41
42
kill(9, getpid())
the SIGINT signal and count the total number of times that it is received. See the code below for this purpose:
5.1.2.1
Exercise
Write code which does the following: Parent process creates a child process using
fork()
#include <signal.h> #include <stdio.h> #include <unistd.h> int sigCounter = 0; void sigHandler(int sigNum) { printf("Signal received is %d\n", sigNum); ++sigCounter; printf("Signals received %d\n", sigCounter); } int main() { signal(SIGINT, sigHandler); while(1) { printf("Hello Dears\n"); sleep(1); } return 0; }
Here, signal is the signal handler call and accepts as input the signal number, and the name of function which is going to behave as signal handler. (If we want it to behave the default way, we specify SIG_DFL, or if we want it to ignore a certain signal, we specify SIG_IGN). When we run the program, we are instructing our program that if in case the SIGINT signal is detected, we will perform the steps provided in the
ment blocks).
Parent sends the terminate signal via kill() call to child and then waits for 120 seconds
While parent is waiting, the user should check the outcome of ps command.
5.1.2.2
Exercise
and then
Modify the code in Exercise-5.1.2.1 such that signal is sent from child to parent. have your child wait for 120 seconds. Again, while child is waiting, the user should check the outcome of ps command.
5.2
Signal delivery is handled by the kill command or the kill() call. The process receiving the signal can behave in a number of ways, which is dened by the signal() call. The syntax of the call is as such:
signal(int, conditions)
The signal will require the to work.
signal.h
C library
int
sigHandler
is the
as there is no event, the program will keep on executing the while loop. delivered by pressing
integer identier of the respective signal which is sent, and the last parameter is either of the following:
CTRL+C.
Try pressing CTRL+C with, and without the signal() call and you will understand the dierence yourselves.
SIG_DFL which will perform the default mechanism provided by the operating system for that particular signal
One process will read from the buer while the other will write to it. One process cannot read from the buer unless and untill the other has written to it.
6.1
pstree
As you will notice, the output is too long to t in the screen. Now run the command with:
pstree | less
Using the up and down arrows you will notice that you can browse through the output which was otherwise not visible in the rst command which we gave. To exit, press The
have guessed, pstree and less are two processes. In this usage, the become the
Redirect the standard output of one process to become the standard input of another
The pipe will be a buer region in main memory which will be accessible by only two processes.
43
44
6.2
(b) In Parent
The above code creates a pipe using the pipe() system call. We have passed it the name of the integer array on.
pfd
Task:
Rewrite this code so that you can view Continuing with rest of code: You should see two numbers.
the contents of the array using printf arguments. What are these numbers?
01 #include <unistd.h> 02 int main() 03 { 04 int pid; 05 int pfd[2]; 06 char aString[20]; 07 pipe(pfd);
09 10 11 12 13 14 15 16 17 }
// For child // Write onto pipe // For parent // Read from pipe
// // // //
for storing fork() return for pipe file descriptors Temporary storage Just like we open a le, we read from a le, create our pipe
we write to a le, and we close a le, we will perform the same operations of open(), close(),
When line number 07 completes, we will have the following in our process:
Task:
the contents of aString in the parent before and after the read() call. the contents? You will notice that Hello has been mentioned in the chid process. Then how is it possible that we are able to see the term Hello in the parent process? Figure 6.1: Before Fork() The answer is through the pipe mechanism which we just used.
In the code we just saw, since the child is only going to write to a pipe and the parent will only read from the pipe, it makes sense to close the read capabilities for the child and write capabilities for the parent for that particular pipe. When line number 08 completes, we will have the following in our two processes: Diagramatically, we want to achieve something like the following:
08
pid = fork();
45
}
(a) In Child
(b) In Parent
close(pfd[0]);
and the following before line 15:
close(pfd[1]);
6.2.1 Example
Type, run and execute the code below. It should give output which is equivalent to the command ls | wc (Wc is used for printing three numbers; the number of newlines, the number of words, and the byte count for a le). Note the usage of pipes.
#include <unistd.h> #include <string.h> #include <stdio.h> int main() { int pfd[2]; pipe(pfd); if (fork() == 0) { close(pfd[1]); dup2(pfd[0], 0); close(pfd[0]); execlp("wc", "wc", (char *) 0); } else { close(pfd[0]); dup2(pfd[1], 1);
46
47
7.1
Thread Basics
A thread can be dened as a separate stream of instructions within a process. From developer point of view, a thread is simply a function or procedure, that has its own existence and runs independently from the program's main() procedure/function. To visualize, imagine a program C program with a number of functions. This program can be run by entering the executable a.out on the command interface. Then imagine each function being scheduled by the operating system. This would be a multi-threaded program. Unlike child processes, a thread doesn't know which thread is responsible for it's creation, neither does it maintain a list of current active threads inside a process. Within the same process, a thread may share the process instructions (text section), global data (data section) and open les (le descriptors). Each thread has a unique thread ID, set of registers and stack. These will be individual to a thread itself.
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21
#include <stdio.h> #include <stdlib.h> #include <XXXXX> // Check Man void *print_message(char *ptr) { char *message; message = *ptr; printf("%s \n", message); } int main() { pthread_t thread1, thread2; char *message1 = "Thread 1"; char *message2 = "Thread 2"; int return_value1, return_value2; return_value1 = pthread_create(&thread1, NULL, print_message, *message1); return_value2 = pthread_create(&thread2, NULL, print_message, *message2); pthread_join( thread1, NULL ); pthread_join( thread2, NULL ); exit(0); }
Compile the above code. Compilation of multithreaded programs are dierently from the normal method. For threads, the -lpthread argument is provided as an addition. When this program runs, there will be a total of 3 threads running in this process; main, thread1, and thread2. Three additional kernel level threads will be required for servicing these three user-level threads. uses 1:1 thread model). When the thread is created using pthread_create(), the main thread proceeds executing with the remainder of instructions. Meanwhile, the newly created thread will complete executing as well. If in case the (Remember, Linux
7.2
Thread Creation
main() thread nishes executing before any of the other threads, the process will exit. Any thread which had not nished will be interrupted before its termination. this, To avoid the pthread_join() call can be used.
A thread is created using the pthread_create() call. Just like process creation using fork(), a pthread_create() call will return certain inte-
49
50
With this,
successfull completion of any thread specied in the join call. Actually, pthread_join() is the opposite of pthread_create(). Pthread_create() will split our single threaded process into two-threaded process. Pthread_join() will join back the twothreaded process into a single threaded process. Read the manual pages and nd out the answers to the following:
void *print(void *threadArg) { struct thread_data *my_data; my_data = (struct thread_data *) threadArg; printf("X: %d, Y: %d, Z: &d", my_data->x, my_data->y, my_data->z); }
Is our thread function. Here, we have declared a pointer called my_data and assign it location of argument passed to thread as input. Since this is a pointer to a structure, we will be using the arrow operator (->) instead of the dot (.) operator to access it's members.
Q1 Q2 Q3
What is the pthread_create() and the pthread_join() calls doing? In the pthread_create() call, what are the 4 parameters? In pthread_create() call, the 4th parameter is used for passing a pointer to argument of a function. What will we need to do if we want to pass multiple arguments to that function?
int main() { thread_t tid; omar.x = 1; omar.y = 2; omar.z = omar.x + omar.y; pthread_create(&tid, NULL, print, (void *) &Omar); }
Here, we assign the members x, y, and z some values. Instead of sending these individual members over to the thread, we send the address of the instance Omar.
Q4 Q5
In the pthread_join() call, what are the 2 paramters? What is the purpose of the return_value1 and return_value2 variables? Find out the contents of both these variables using a printf function. What do they contain?
7.3
Thread Termination
Argu-
The thread returns from its starting routine Thread makes a call to pthread_exit() call The entire process is terminated due to call to exec() or exit()
Multiple arguments can be passed to a thread through declaring a structure. For example,
If the main() thread nishes executing before any other thread in the process, all threads will terminate (Bullet 4). However, if main() exits using a pthread_exit() call, all other threads in the process will continue to execute. Thus the pthread_exit() call will terminate the main thread but keep on clinging to resources such as process memory space and open le descriptors. The following code will show both thread creation and thread termination.
51
29 myGlobal = myGlobal + 1; 30 printf("o"); 31 fflush(stdout); 32 sleep(1); 33 } 34 pthread_join(myThread, NULL); 35 printf("\nMy Global Is: %d\n", myGlobal); 36 exit(0); 37 }
Fush() is used to force a write of all user-space buered data. Since we have specied stdout, therefore it will write it to standard output. We have two threads here again as well. The main thread has a for loop which is printing the character o. The threadFunction thread also has a for loop which is printing the character .. You will notice thread-scheduling in action when you see an output of:
7.4
Data
Sharing
between
Threads
Compile and run the following code:
about the myGlobal value ... Should it be 6? Or should it be 10 (2 For loops each running for 5 iterations)? Now comment the sleep line (both or just one) and check the output. It should be 10. How come a sleep(1) can make such a dierence? Here is a little explanation: We are using the sleep() call to impose a rudimentary form of synchronization between both threads. With sleep, CPU alternates between both threads a total number of 5 times. Each time the myGlobal variable is overwritten by the threadFunction thread. sleep, there is just one alternation. Without The my-
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26
int myGlobal = 0; void *threadFunction() { int i, j; for (i = 0; i<5; i++) { j = myGlobal; j = j+1; printf("."); fflush(stdout); sleep(1); myGlobal = j; } }
Global variable rst counts upto 5, and then it counts upto 5 again, totalling 10. But there is no synchronization between threads in this way. If we want to impose synchronization and at the same time preserve data integrity, we would be needing something more accurate
int main() than a simple sleep() call. { pthread_t myThread; int i; pthread_create(&myThread, NULL, threadFunction, NULL); Every process has a certain portion of code 27 for (i = 0; i < 5; i++) 28 { which is called the critical section of a pro-
through
52
cess. As an example, this is the area where a process may: 1. Change shared variables 2. Write to a File 3. Use a shared resource It would be desirable that if one process is working in this region, then another process should not be allowed to modify the contents of any shared variable within it. If, and only if, that process leaves the critical section, then another process may be allowed access to that shared region. In other words, such a region is mutually exclusive to one-and-only-one process at a time. Ideally, if a process enters this region, it should lock it. Any process trying to access Once the process leaves it in the meanwhile will not gain any access because it is locked. for anybody else. We will take this concept of critical-section and mutual exclusion and apply it to our problem in Section 7.4. Here, we apply our lock If a and unlock mechanism with the help of Mutexes (taken from Mutually Exclusives). thread is currently locked into it's critical section, another thread trying to access it will go into sleep mode. Compile and run the code given. Here, the lock is our entry section and the unlock is our exit section. this region, it will un-lock it, rendering it open
7.5
Exercise
A bonacci series is a set of numbers where th th every n number is the sum of the n 1 and th n 2 numbers. The only exception to this st nd rule is the 1 and 2 numbers which are 0 and 1 respectively. The following code can nd the nth number in the bonacci sequence:
01 02 03 04 05 06 07 08 09 10 11 12 13 14 15 16 17 18 19 20
int myGlobal = 0; pthread_mutex_t myMutex; void *threadFunction() { int i, j; for (i = 0; i<5; i++) { pthread_mutex_lock(&myMutex); j = myGlobal; j = j+1; printf("."); fflush(stdout); sleep(1); myGlobal = j;
int fib(int n) { if (n<=0) return 0; else if (n==1) return 1; else return fib(n-1)+fib(n-2); } int main() { int find = 40; printf("Element No. %d in series is: %d", find, fib(find) ); exit(0); }
Note that the call to b() function is recursive. Modify the above code so that each b() call is implemented in a separate thread.
53
The following sections will help you explain the dierence between normal waiting (using sleep()) and busy waiting. Run both the codes. Both codes are ALMOST the same. The only dierence is the contents of the sleepy() function. Make sure you are not doing ANYTHING ELSE besides staring at your screen while executing these programs.
#include <do yourself> int counter = 60; void *sleepy() { sleep(60); printf("Hello"); } int main() Compile the above code, and for running, enter { the following command: pthread_t tid; pthread_create(&tid, NULL, sleepy, NULL); while (counter > 0) { # uptime && ./programName && uptime counter--; sleep(1); The above command bundle is going to run the } uptime tool, followed by your compiled propthread_join(tid, NULL); } gram, and followed by the uptime tool again
in quick succession. Note down the load averCompile the above code, and for running, enter the following command: age for the rst 1 minute. What is the dierence?
Banking Example
Since it will be too much work (for me) to have a separate problem set for each synchronization algorithm, I will put forward a simple case of an account debit and account credit in a banking software.
55
56
balance.
At the
moment, this bank only has support for hanWhenever the customer visits the bank, the bank manager will note down his time of arrival. If the customer visit is in regard of adding money to his account, the bank manage will note down the amount deposited. If the customer wants to withdraw money from the bank, the bank manager will note down the amount deducted. Later on before closing hours, the bank manager will take his daily log book and start making entries for the numer of times the customer visited the bank and what kind of transaction was the customer interested in. If at any time the transaction was only credit, the bank manager will select option 1. If the transaction was only debit, the bank manager will select option 2. If the transaction was both debit and credit, then the manager will choose option 3. A sample code (not run, not tested, may have errors) for this bank software is given below:
#include <pthread.h> #include <stdio.h> #include <stdlib.h> double balance = 0; double temp1 = 0; double temp2 = 0; int times = 0; void *credit(void * arg) { int a = *(int *) arg; balance = balance + a; } void *debit(void * arg) { int a = *(int *) arg; balance = balance - a; } int main() { int choice; pthread_t credit_thread, debit_thread; while(1) { system("clear"); printf("Name: Alif Noon\nAcc No.: 420\n"); printf("Available Balance: Rs. %f/-\n", balance); printf("\nChoose Transaction Type:\n"); printf("------------------------\n"); printf("1. Account Credit\n2. Account Debit\n 3. One Credit + One Debit\n 4. Multiple Credits\n
5. Multiple Debits\n"); printf("6. Multiple Credit and Debit\n 7. Exit\n"); scanf("%d", &choice); if (choice == 1) { printf("Enter amount to credit\n"); scanf("%d", &temp1); printf("main. balance = %d\n", balance); pthread_create(&credit_thread, NULL, credit, &temp1); } else if (choice == 2) { printf("Enter amount to debit\n"); scanf("%d", &temp2); pthread_create(&debit_thread, NULL, debit, &temp2); } else if (choice == 3) { printf("Enter amount to credit\n"); scanf("%d", &temp1); printf("Enter amount to Debit\n"); scanf("%d", &temp2); pthread_create(&credit_thread, NULL, credit, &temp1); pthread_create(&debit_thread, NULL, debit, &temp2); } else if (choice == 4) { printf("\nHow many times to make credit transaction?"); scanf("%d", ×); printf("Enter amount to credit\n"); scanf("%d", &temp1); int loop; for (loop = 0; loop < times; loop++) { pthread_create(&credit_thread, NULL, credit, &temp1); } } else if (choice == 5) { printf("\nHow many times to make debit transaction?"); scanf("%d", ×); printf("Enter amount to debit\n"); scanf("%d", &temp2); int loop; for (loop = 0; loop < times; loop++) { pthread_create(&debit_thread, NULL, debit, &temp2); } } else if (choice == 6) { printf("Enter amount to credit\n"); scanf("%d", &temp1); printf("Enter amount to Debit\n"); scanf("%d", &temp2); printf("Enter number of times to Credit and Debit\n"); scanf("%d", ×); pthread_create(&credit_thread, NULL, credit, &temp1); int loop; for (loop = 0; loop < times; loop++) { pthread_create(&credit_thread, NULL, credit, &temp1); } for (loop = 0; loop < times; loop++) { pthread_create(&debit_thread, NULL, debit, &temp2); } } else if (choice == 7) { break; } else { printf("Wrong Choice");
57
continue; Now run your code. What is the behaviour } you are getting now? } pthread_join(credit_thread, NULL); Try adding a sleep(1) at the end of the funcpthread_join(debit_thread, NULL); tion and study the behaviour now. printf("Account Summary:\n----------------\n"); printf("Available Balance: Rs. %f/-\n", balance); Now move the sleep(1) call from the end exit(0);
of the function to the middle of the function (in between the instructions). haviour now. Presto!!! The software has gone nuts. Welcome race condition. The dierence we have seen is that initially we had multiple threads but they were executing sequentially. When we place sleep(1) in the middle, we are imposing some kind of rudimentary synchronization so that the threads execute parallely rather than sequentially. Parallel execution of threads mean that multiple threads are trying to enter to access/modify the shared resource (balance variable) at the same time. It is this simultaneous usage that leads to race conditions. Check the be-
The code is also given on http://lectures for you to copy paste. Run and compile the code and play around with it. Most of the code is self explanatory and you have done it already in previous labs. Check the program to see if it is giving any improper behaviour and note it down. In the following block:
void *credit(void * arg) { int a = *(int *) arg; balance = balance + a; } void *debit(void * arg) { int a = *(int *) arg; balance = balance - a; }
Add a sleep(1); at the end of each function and then run the program. Why is it that still we are given the same kind of behaviour. Whereas in previous lab you will recall that adding sleep(1) actually gave you dierent behaviour. Modify the code such as follows:
8.3
Exercise:
Introducing
Synchronization
Try introducing synchronization in the banking software according to the 3 methods that we studied during lectures. are: 1. Algorithm 1 (Usage of turn variable) 2. Algorithm 2 (Usage of boolean array of size 2) The techniques
Us-
local
local
age of both Algorithm 1 + Algorithm 2) You may use your lecture notes as a reference.
balance =
local
local
a;
balance =
local ;
Both the above modications are doing the same thing. There is nothing new being implemented. The only dierence is that instead of a short atomic instruction, you have divided the task over a series of 3 instructions.
58
59
Chapter 9 Memory
9.1 PROC File System
Q3
Now
check
the
current
time
on
your
watch? Give the following command without any arguments and press Enter: Do you notice that the modication time is the same as the time you entered the command. If not, wait for 1 minute and run the command again. You will see a list of all partitions that are currently mounted on your system. If you look closely, there will be an entry called PROC. PROC is a special le system and is an open door to the Linux kernel. You can peep in here to see what the kernel is doing. The proc le system can be accessed by changing into the /proc directory. You will notice that the modication time has changed yet you haven't even touched the le so far. This proves that the information appears on the y and that nothing exists in the directory as such. Now let's view the contents of this le. Type the following:
mount
cat /proc/version
You can see version numbers such as the kernel, and the compiler which compiled the kernel. Give the ls command again. Note the following again:
cd /proc
Here, you will apparently see plenty of les. But these are not les, rather they are parameters, data structures, and statistics collected from the kernel and which appear like les inside the /proc directory. The contents of this directory are created 'on-the-y' by the kernel whenever we read its contents. The existence of les in this directory only appear the instance you request it. Otherwise, the directory is EMPTY.
What is the le size? What is the modication time? How come so much information was just showed as contents of the le yet the le size is zero? Again, this is conrmation of previous point.
ls -l /proc/version
Note the following about above command:
Q1 Q2
cd /proc/<number>
61
62
CHAPTER 9. MEMORY
Size of shared libraries Memory used by the process for its stack Number of dirty pages (pages of memory that have been modied by the program) Check these information for your currently logged in shell.
View the contents. You should be able to see les like cmdline, cwd, environ, exe, fd, stat, statm, status, etc. process. Using the ps command, nd out the process id of your current shell. Of these, fd contains the number of open le descriptors for any given
ps
Now give the following command:
cat /proc/<your shell process id>/statm cat /proc/<your shell process id>/status
Q Q
What open le descriptors can you see from the output? What do the arrow operator (->) represent?
cat /proc/meminfo
Q Q Q
Find out the total memory of your machine How much of it is free How much is your swap space How much of that swap space is free from meminfo
9.2
Exploring Memory
dow was one instance of shell process (so 2 processes). We gave a command in one shell and the output appeared in the other shell. Isn't this communication between two processes? The IPC method of communication was through a le.
Total process size Size of process currently resident in main memory Size of memory shared with other processes Text section size of a process
#include <unistd.h> #include <stdio.h> #include <fcntl.h> int main() { int fd, bw, br; char *buffer=(char*)calloc(NULL, BUFSIZ); fd = open("/tmp/foo.txt",O_CREAT|O_RDWR); br = read(0,buffer, BUFSIZ); bw = write(fd,buffer,BUFSIZ); close(fd); return 0; }
You can see usage of I/O system calls such as open(), read(), write() and close(). You can also see a system call called calloc(). Calloc() is allocating un-used space for an array of calloc() and its sister system calls shortly.
63
Now exit and recompile your code with the static option.
gcc code.c -c
and then
What is the size of the executable le with and without the
-g
-g
argument?
What is the size of the executable le with and without the
-g
argument?
We are now going to use the objdump utility to disassemble (-d) the program and view the information about the object le. Run the following:
Q Q Q Q
What is the starting point of your rst instruction again? What is the address of the main function? Do you think this time it is a logical or a physical address? At what instruction addresses are the dierent calls located (calloc, open, read, write, close, etc)
In which section is the write() call referring to? Where is it referring to in memory address? Go to that address !!!
Q Q Q
The address on which your rst instruction of your program starts. Note it down. The address at which the main() function starts. Note this down as well. Identify the instructions for calloc(),
Do you think this is the actual implementation area of the write() call?
open(), read(), write() and close() calls and note down the addresses on which these instructions appear. This should be in the main function.
9.3
9.3.1 Introduction
For sake of denition, a process is a running program. It means an OS has loaded a program into memory, has arranged some environment for it, and has started running it. Memory is required for the following areas in a process:
Q Q
What do you think? Are the addresses to the left logical or physical addresses? Find out where and in what section the write() call is referring to? Go to that section. How many lines of code are in that section (making a rough estimate)?
Do you think these lines of code are the actual implementation of the write call? Write() call is from the unistd.h library. So do you think we are currently looking at the function from the unistd.h library?
Text
Section:
The
portion
of
process
where the actual executable code resides. Data Section: Containing global data that are to be generated during run-time when the program starts.
64
CHAPTER 9. MEMORY
Heap: The region from where memory can be dynamically allocated to a process.
Stack
Section:
Where
local
variables
(non-static) and function calls are implemented. To have a general overview of how much size is allocated to a process, do the following simple test. 1. Create a simle Hello World 2. Compile it into an executable called Hello 3. Then, type the command: ls -l Hello and note down the size of the executable 4. Then, type the command: size Hello and note down the total size of all sections
#include <stdio.h> #include <stdlib.h> int main() { struct coord { int x, y, z; }; struct coord *p; p = malloc(sizeof(struct coord)); if (p==NULL) { printf("Failed"); } else { printf("\n%d bytes allocated at address %d\n", sizeof(struct coord), p); } return 0; }
9.3.2.2
Free()
Once we are done with memory, we can return it back using the free() function. Continuing with previous code, we add the following:
free(p); p = NULL;
The rst line will free the memory address returned to pointer p. This means it is marked for use if addtional memory is required for the process. However, variables still pointing to it can still use it until the time it is overwritten. This is known as a dangling pointer. It is good idea to set the pointers to NULL along with usage of free. Try free two times and see the output.
9.3.2.3
Realloc()
Using realloc() call, we can dynamically change the size of the block of memory allocated earlier on. It's usage is simple:
9.3.2.1
Malloc()
p = realloc(p, sizeof(struct coord)*10);
This will take the current block returned to pointer p, and change the size to whatever we specify as the second parameter. There is, however, a problem with the above statement .. can you identify it?
Initially memory is allocated using malloc. The value passed is the total number of bytes requested. If memory is not allocated, NULL is returned. If memory is allocated, a pointer to 1st address is returned. Try multiplying sizeof by a large number to see if it can allocate that much memory.
65
9.3.3 Exercise
Study the code below. What is wrong with it?
#include <stdio.h> #include <stdlib.h> void f(void) { void* s; s = malloc(50); return; } int main(void) { while (1) f(); return 0; }
66
CHAPTER 9. MEMORY
Bibliography
[1] W.R.STEVENS, Unix Network Programming: Interprocess Communications, Vol 2, 2nd Edition, Prentice-Hall India [2] M.SINGHAL., Advanced Concepts in Operating Systems, Tata-McGraw-Hill [3] Gentoo x86 handbook, x86.xml available online:
http://www.gentoo.org/doc/en/handbook/handbook-
67
Index
/bin, 12 /boot, 12 /dev, 12, 13 /etc, 12 /home, 12 /lib, 12 /opt, 12 /root, 12 /sbin, 12 /tmp, 12 /usr, 12 /var, 12 absolute path, 12 bash, 13 boot, 11 boot image, 15 bootloader, 15 broadcast, 14 cag, 14 chroot, 13 copy, 14 cp, 14 cron, 15 cxxag, 14 directory, 12 directory tree, 12 emerge, 14, 15 env-update, 13 eth0, 14 ext, 13 ext2, 13 ext3, 13 extended partition, 11 fdisk, 11 le, 12 nland, 9 formatting, 13 fstab, 15 kernel, 9, 14 khyber lab, 10 lilo, 12 linux, 9 linux torvalds, 9 live cd, 10 liveCD, 13 liveDVD, 13 locate, 15 logical partition, 11 make, 15 make.conf, 14 makeopts, 14 master boot record, 12 mbr, 12 menucong, 14 mkdir, 13 mke2fs, 13 mkswap, 13 module, 15 mount, 13 mount, usb, 13, 14 nano, 15 netmask, 14 network, 14 operating system, 9 partition, 13 partition, extended, 11 partition, logical, 11 gentoo, 10, 13 gentoo-sources, 14 getpid(), 29 getppid(), 29 grub, 12, 15 grub.conf, 15 helsinki, 9 ifcong, 14 iso image, 10
68
INDEX
partition, primary, 11 partitioning, 11 password, 15 path name, 12 path, absolute, 12 path, relative, 13 pid, 29 portage, 14 ppid, 29 primary partition, 11 proc, 13 proxy, 14 rc-update, 15 reboot, 17 relative path, 13 root, 11 run-level, 15 shell, 9 software, 9 source, 13 source based distribution, 10
69
portage, 13
stage, 13
swap, 11, 13 swapon, 13 syslog, 15 system software, 9 system(), 24, 30 tar, 13 types.h, 29 umount, 14 unistd.h, 29 unix, 9 virtual machine, 10 vmware, 10