Operating system

An operating system is the program that controls all the other parts of a computer system - both the hardware and the software. An operating system provides orderly and controlled allocation and use (i.e., sharing) of the resources by the users (jobs) that compete for them. One major function of an operating system is to hide the complexity of the underlying hardware and give the user a better view (an abstraction) of the computer.

Operating system components

An operating system generally consists of the following components:
Process management (Disk) storage management Memory management I/O (device) management File systems Networking Protection User Interface

OS architecture

Operating System
Operating system interacts with user in two ways Operating system commands Enables user to interact directly with the operating system.
Operating system calls Provides an interface to a running program and the operating system. System calls in UNIX are written in C.

Bootstrapping
The process of initializing the computer and loading the operating system is known as bootstrapping. This usually occurs when the computer is powered up or reset. The initial loading is done by a small program that usually resides in non-volatile memory (e.g., EPROM). This in turn loads the OS from an external device. Once loaded, how does the operating system know what to do next? It waits for some event to occur: e.g., the user typing a command on the keyboard.

History of the UNIX operating system

The first version of UNIX was created in 1969 by Kenneth Thompson and Dennis Ritchie, system engineers at AT&T's Bell Labs. It gained popularity in 1977, when it was first made commercially available by Interactive Systems Corporation. A team from the University of California Berkeley was working to improve UNIX the same time. In 1977 it released the first Berkeley Software Distribution, which became known as BSD. By 1983 commercial interest was growing and Sun Microsystems produced a UNIX workstation called Solaris.

History of the UNIX operating system

System V appeared, directly descended from the original AT&T UNIX and the prototype of the more widely used variant today.

Modern variants of UNIX

There are two main versions of UNIX in use today: System V and BSD. System V is the more popular of the two.

Advantages of Linux OS
Linux source code is freely distributed Linux is Multi-user. Linux runs on a wide range of hardware Linux is exceptionally stable Linux has the tools and applications you need Linux is one of the most secure operating systems.

Features of UNIX
Multi-user, multitasking, timesharing
Portability

Modularity
File structure Security Strong networking support & advanced graphics

UNIX system/kernel structure

Accessing OS services
The mechanism used to provide access to OS services (i.e., enter the operating system and perform a privileged operation ) is commonly known as a system call. The (only) difference between a procedure call and a system call is that a system call changes the execution mode of the CPU (to supervisor mode) whereas a procedure call does not. System call interface: A set of functions that are called by (user) programs to perform specific tasks.

UNIX file system

"On a UNIX system, everything is a file; if something is not a file, it is a process." A file system is a logical method for organizing and storing large amounts of information in a way which makes it easy manage. The file is the smallest unit in which information is stored.

UNIX FILE SYSTEM HIERARCHY

The UNIX file system is laid out as a hierarchical tree structure which is anchored at a special top-level directory known as the root . A directory can have many child directories, but only one parent directory.

File System Structure

Boot Block Super Block Inode Block Data Block
Type of the file Link counter Uid, gid, size Date and time of Creation Date and time of access Date and time of modification : :

Address of datablock Address of datablock : :

Address of the addr block Address of the addr block Address of the addr block

UNIX file system

Sorts of files
REGULAR Files: Most files contain normal data, for example text files, executable files. Directories: files that are lists of other files. Special files: the mechanism used for input and output. Most special files are in /dev.
Links: a system to make a file or directory visible in multiple parts of the system's file tree. (Domain) sockets: a special file type, similar to TCP/IP sockets, providing interprocess networking Named pipes: act more or less like sockets and form a way for processes to communicate with each other.

 At the time a new file is created, it gets a free inode. The following information in that inode :
Owner and group owner of the file. File type (regular, directory, ...) Permissions on the file Date and time of creation, last read and change. Date and time this information has been changed in the inode. Number of links to this file File size An address defining the actual location of the file data.

UNIX file is represented by an inode file system In Unix file system, a

The Shell
Whenever you login to a Unix system you are placed in a program called the shell. On most Linux systems a program called bash acts as the shell program. bash stands for Bourne Again SHell, an enhanced version of the original Bourne shell program, sh, written by Steve Bourne There are several additional shell programs available on a typical Linux system. These include: ksh, tcsh and zsh.

PROCESS
A process is a running instance of a program. A process is said to be born when starts executing and remains alive as long as the program is active. As files, processes do have attributes,
PID: Process ID PPID: Parent process ID TTY: Terminal on which the process is connected

The Shell
The Shell is the program that directly executes your commands. The shell acts as a command interpreter. Activities The shell issues a prompt and waits for command. The shell scans for meta-character and expands them. It then passes command to kernel. Waits till the command is executed by the kernel and reappears again.

UNIX COMMAND SET

Where are commands located ? Unix commands are executable binary files located in directories with the name bin (for binary). Many of the commands that you use are located in the directory /usr/bin.

UNIX COMMAND SET

GENERAL FORMAT

$ command -options targets

GENERAL COMMANDS date cal echo Clear Who man exit

Simple Commands

pwd
(print [current] working directory)

date
Displays the current date and time Date has various options, which can be used as
+%d to display day of month (01..31)

+%D +%e +%H

to display date in the format (mm/dd/yy) to display day of month, blank padded to display hours component (00 .. 23) of current date

Simple Commands
who
Displays the names of all the users who have currently logged in

who am i
Displays the name of the current user.

Listing the Directory Contents

ls Syntax options:
:ls [options] [file.] -l list in long format -a list all files including those beginning with a dot -i list inode no of file in first column -s reports disk blocks occupied by file -R recursively list all sub directories -F mark type of each file -C display files in columns

Listing the Directory Contents

$ls -l

DIRECTORY AND FILE HANDLING COMMANDS

File types in a long list Symbol : d: l: c: s: p: b: Meaning Regular file Directory Link Special file Socket Named pipe Block device

Getting Help
The command man gives you access to an on-line manual containing a complete description of every command available on this system. man can also provide you with one line descriptions of commands specified by name; or for all commands whose description contains any of a set of keywords.

Meta Characters
Meta Characters * ? [] Purpose Example $ ls l *.c file* $ ls l file? $ ls l file[abc] $ cat file1; cat file2 $ cat abc | wc $ (echo ==== x.c ====; cat x.c) > out count=`expr $count + 1` assuming count has value3, this increments the value of count echo expr $count + 1 displays expr $count + 1 echo expr $count + 1 displays expr 3 + 1 assuming the variable count has value 3 Match with one or more characters or none Match with any single character Match with any single character within the brackets ; Command separator | Pipe two commands () Group commands Useful when the output of thecommand group has to be redirected `command` Execute the command enclosed within back quotes. Useful when the output of a command into a variable in a shell script

string
string

Quote all characters with no substitution (ex. no special meaning for $ ) Quote all characters with substitution. The characters $,\ (back slash) and back quote have special meaning.

Changing Permissions - chmod

chmod u+x file_name
Syntax: chmod <category> <operation> <permission> <filename(s)> or chmod <octal number> filename Octal Number 4 - for read 2 - for write 1 - for execution

$ chmod 744 xyz this sets read, write and execute permissions for owner, read permission for group and others

Directory Creation
Command Syntax mkdir [OPTION] DIRECTORY $ mkdir <path>/<directory> $ mkdir m <directory> $ mkdir p <directory1>/<directory2>/<directory3>

Example: $ mkdir project1 This creates a directory project1 under current directory Note: Write and execute permissions are needed for the directory in which user wants to create a directory

Directory Removal
rmdir command removes directory Syntax rmdir <directory name> Example Removes project1 directory in the current directory rmdir project1 Remove multiple directories rmdir pos1 pos2 Remove the directory recursively rmdir p dir1/dir2/dir3 rmdir removes a directory if it is empty and is not the current

Command - cp
Used to copy files across directories Syntax cp <source file> <new file name> Options to cp -p Copies the file and preserves the following attributes -r recursive copy; copy subdirectories under the directory if any -i interactive; prompts for confirmation before overwriting the target file, if it already exists

Command - mv
Used to move a file, or rename a file
Preserves the following details owner id group id permissions Last modification time

-f
-i

suppresses all prompting (forces overwriting of target)

prompts before overwriting destination file

Command - rm
Used to remove a file Syntax : rm file(s)
-f suppresses all prompting

-i

prompts before deleting destination file

-r will recursively remove the file from a directory (can be used to delete a directory along with the content )

Caution: Use i option along with r to get notified on deletion

cat
cat command takes the input from the keyboard, and sends the output to the monitor

We can redirect the input and output using the redirection operators
$ cat > file1 Type the content here press <ctrl d> $ cat file1 Displays the content of the file $cat >> file1 This will append standard input to the content of file1

touch
touch is used to change the time stamp of the file

Syntax: touch [options] file

Options: -a to change the access time -m to change the modification time -c no create if not exists

touch <file> will change the time of change of the file if the file exists
If the file does not exist, it will create a file of zero byte size.

wc & sort
wc A filter used to count the number of lines, words, and characters in a disk file or from the standard input. -l - displays the number of lines -w - displays the number of words -c - displays the number of characters sort

sort
Sorts the contents of the given file based on the first char of each line. -n numeric sort (comparison made according to strings numeric value) reverse sort specify delimiter for fields specify sorting field numbers

-r -t
+num

grep
grep -Global Regular Expression Printer is used for searching
regular expressions Syntax grep <options> <pattern> <filename(s)> -c displays count of the number of occurrences -n displays line numbers along with the lines -v displays all lines except lines matching pattern -i Ignores case for matching

Filter Command head & tail

head
Displays the first n lines of the file. By default head command displays first 10 lines $ head -3 file1

tail
Displays the last n lines of a file $ tail -3 file1 Can also specify the line number from which the data has to be displayed till the end of file $ tail +5 file1

Filter command - tr
tr - translate filter used to translate a given set of characters Example : tr [a-z] [A-Z] < filename This converts standard input read from lower case to upper case. -s char
Squeeze multiple contiguous occurrences of the character into single char

-d char
Remove the character

Command Piping
Allows the output (only the standard output) of a command to be sent as input to another command. Multiple pipes may appear in one command line. Example: $ cat * | wc $ cat fil1 | head | wc -l

Filter Command cut

Used to extract specified columns of a text Option remark -c used to extract characters -d Delimiter for fields -f Field no. Examples $ cut -c2-5 file1 $ cut -d | -f2,3 file1

find
Lets user to search set of files and directories based on various criteria Syntax: find [path...] [expression] [path] where to search [expression] What type of file to search (specified with type option) What action to be applied (exec, print, etc.) Name of the files (specified as part of name option, enclosed in ) Example find . name *.c -print

lists all files with .c extension from the current dir & its subdirectories

find
Finding files on the basis of file size

size [+ ]n[bc]

n represents size in bytes (c) or blocks (b) of 512 bytes find . size 1000c lists all files that are exactly 1000 bytes in size

find . size +1000c lists all files that are more than 1000 bytes in size find . size 1000c lists all files that are less than 1000 bytes in size

Finding files on the basis of access time (atime) or modified time (mtime)

atime [+-]n mtime [+-]n

n represents number of days ( actually 24 * n hours) lists files accessed exactly 2 days ago

find . atime 2

find . atime +2
find / mtime 2

lists files accessed more than 2 days ago

lists files modified less than 2 days ago

Compression Utilities
gzip, Usage is very similar to compress and pack utilities in Unix: gzip [-vc] filename where -v displays the compression ratio. -c sends the compressed output to standard output and leaves the original file intact. gunzip gunzip can uncompress files originally compressed with compress.

DIRECTORY AND FILE HANDLING COMMANDS

Linking files
Rather than having multiple copies of a file, Linux uses linking to one file to save disk space and administrative headaches trying to keep multiple copies up to date and synchronized. Linux supports two types of links,
hard links and symbolic links.

DIRECTORY AND FILE HANDLING COMMANDS Hard link

Associate two or more file names with the same inode. Hard links share the same data blocks on the hard disk, while they continue to behave as independent files. There is an immediate disadvantage: hard links can't span partitions, because inode numbers are only unique within a given partition. $ln targetfile linkname

DIRECTORY AND FILE HANDLING COMMANDS Soft link or symbolic link

A small file that is a pointer to another file. A symbolic link contains the path to the target file instead of a physical location on the hard disk. Since inodes are not used in this system, soft links can span across partitions. Note that removing the target file for a symbolic link makes the link useless. ln -s targetfile linkname

DIRECTORY AND FILE HANDLING COMMANDS SORTING FILES

sort filenames $ sort input1.txt input2.txt

uniq filename $ sort input.txt | uniq > output.txt

DIRECTORY AND FILE HANDLING COMMANDS FILE COMPRESSION compress, gzip $ compress filename or $ gzip filename $ uncompress filename or $ gunzip filename

Process Control Commands

There are several commands that can be used to control processes. They are:
ps - list the processes running on the system kill - send a signal to one or more processes (usually to "kill" a process) jobs - a way of listing your own processes bg - put a process in the background fg - put a process in the forground.

VI Editor
Introduction
The VI editor is a screen-based editor used by many Unix users. The VI editor has powerful features to aid programmers. The VI editor lets a user create new files or edit existing files. The command to start the VI editor is vi, followed by the filename.

VI Editor
The Two Modes of VI The VI editor has two modes:
command and insert.

The command mode allows the entry of commands to manipulate text. These commands are usually one or two characters long, and can be entered with few keystrokes. The insert mode puts anything typed on the keyboard into the current file.

VI Editor
VI starts out in command mode. The commonly used commands to get into insert mode are a and i. Once you are in insert mode, you get out of it by hitting the escape key. You can hit escape two times in a row and VI would definitely be in command mode. Hitting escape while you are already in command mode doesn't take the editor out of command mode. It may beep to tell you that you are already in that mode.

VI Editor
Some Simple VI Commands

a
enter insert mode, the characters typed in will be inserted after the current cursor position. If you specify a count, all the text that had been inserted will be repeated that many times.

h
move the cursor to the left one character position.

j
move the cursor down one line.

k
move the cursor up one line.

l
move the cursor to the right one character position.

VI Editor
i
enter insert mode, the characters typed in will be inserted before the current cursor position. If you specify a count, all the text that had been inserted will be repeated that many times.

r
replace one character under the cursor. Specify count to replace a number of characters

u
undo the last change to the file. Typing u again will redo the change.

x
delete character under the cursor. Count specifies how many characters to delete. The characters will be deleted after the cursor.

VI Editor
Deleting words and lines Press the ESC key to enter command mode before using these commands. To delete
current word previous word entire line to end of line to start of line next n lines dw db dd d$ d0 (zero) ndd

VI Editor
Copy and Paste
The command y yanks the text into the temporary buffer. The command p pastes the contents of the temporary buffer back into the file immediately after the cursor.

VI Editor
Saving files and exiting vi Press the ESC key to enter command mode before using these commands. To quit vi
and save the contents of the buffer to the file
without saving the contents of the buffer to the file:

:wq

:q! To quit the vi editor: :q If you have edited (changed) the file you will be prompted with the message: No write since last change You can either save the changes or quit without saving any of the changes you have made.

Shell Programming
Shell is a command line interpreter and also provides a variety of useful programming features to make your scripts truly powerful. A Shell script is collections of commands that are stored in a executable plain file. The shell can read this file and act on the commands as if they were typed at the keyboard. ADVANTAGE A wide range of tasks can be automated.

Shell Programming
EXAMPLE: $vi shell_ex echo "This is a very simple shell procedure " echo "created with the basic echo command " "and three other very basic commands echo ps echo who echo ls

echo

Shell Programming
$sh shell_ex This is a very simple shell procedure created with the very basic echo command and three other very basic commands PID TTY TIME COMMAND 10443 rt02120 0:01 sh 10427 rt02120 0:04 ksh sgavlick rt021e0 Sep 7 13:26 teacher rt021b0 Sep 7 14:39 memo class_notes

Shell Programming
VARIABLES

The Shell has no true numeric variables. It uses string variables to represent numbers, as well as text. Naming a variable has few rules.
It must start with a letter. It must not contain embedded spaces. Use underscores instead. Don't use a name that is already a word understood by bash. These are called reserved words and should not be used as variable names.

Shell Programming
There are three types of variables in the Shell. They are
user variables, Shell variables, and Read-only Shell variables.

You can declare, initialize, read, and modify user variables from a Shell script or from the command line. The Shell also initializes the read-only shell variables, and you can read but not modify them.

Shell Programming
User Variables It is legal to assign any sequence of non-blank characters as the name of a variable.

Sample Sesssion: $ person=Richard $echo person person $echo $person Richard $

Shell Programming
Shell Variables The Shell declares and initializes variables that determine such things as
home directory, what directories the shell will look in when you give commands, your prompt, and many other things. internal-field separator

You can assign new values to these variables from the command line.

Shell Programming
Read-Only User Variables The value of read-only variables can not be changed. The variable must be initialized to some value; and then, by entering the following command, it can be made read only. Command format: readonly variable_name variable_name name of the variable to be made read only

Shell Programming
Sample Session: $person=Kathleen $readonly person $echo $person Kathleen $person=Richard person: is read only $

Shell Programming
The readonly command given without any arguments will display a list of all the read-only variables. Sample Session:

$person=Kathleen $readonly person $example=Richard $readonly example $readonly readonly person readonly example $

Shell Programming
Read-Only Shell Variables
The read-only shell variables are similar to the read-only user variables; except the value of these variables is assigned by the shell, and the user CANNOT modify them. Name of the Calling Program The shell will store the name of the command you used to call a program in the variable named $0. It has the number zero because it appears before the first argument on the command line.

Shell Programming
Sample Session: $vi name_ex echo 'The name of the command used' echo 'to execute this script was' $0 $sh name_ex The name of the command used to execute this was name_ex $

script

Shell Programming
Command Line Arguments The Shell will store the first nine command line arguments in the variables named $1, $2, ..., $9. These variables appear in this section because you cannot change them using the equal sign. It is possible to modify them using the set command.

Shell Programming
Sample Session:
$vi arg_ex echo 'The first five command line' echo 'arguments are' $1 $2 $3 $4 $5 $sh arg_ex Richard Kathleen Douglas The first five command line arguments are Richard Kathleen Douglas $

Shell Programming
The script arg_ex will display the first five command-line arguments. The variables representing $4 and $5 have a null value. The Shell variable $* represents all of the command-line arguments as shown in the following example.

Sample Session:
$sh display_all echo $* $sh display_all Richard Kathleen Douglas Richard Kathleen Douglas $

Shell Programming
The Shell variable $# contains the number of arguments on the command line. This is a string variable that represents a decimal number. You can use the expr utility to perform calculations with that number and test to perform logical tests on it.

Sample Session: $vi num_args echo 'This script was called with' echo $# 'arguments' $sh num_args Richard Kathleen Douglas This script was called with 3 arguments $

Shell Programming
Set When set is called with arguments, it sets the value of the command-line arguments ($1-$n) to the arguments. The example sets the first three arguments. Sample Session: $vi set_ex set who really cares echo $#: $* $sh set_ex 3: who really cares $

Shell Programming
Shift
The shift command promotes each of the command-line arguments. The second argument, represented by $2, is now the first argument, represented by $1. The third becomes the second and so on until the last argument becomes the next to last. You can access only the first nine command-line arguments (as $1 through $9). The shift command gives you access to the tenth, and the first becomes unavailable. There is no "unshift" command that will return the arguments that are no longer available.

Shell Programming
Sample Session: $vi demo_shift echo 'arg1='$1 ' shift echo 'arg1='$1 ' shift echo 'arg1='$1 ' shift echo 'arg1='$1 '

arg2='$2 ' arg3='$3 arg2='$2 ' arg3='$3

arg2='$2 ' arg3='$3

arg2='$2 ' arg3='$3

$sh demo_shift Richard Kathleen Douglas arg1=Richard arg2=Kathleen arg3=Douglas arg1=Kathleen arg2=Douglas arg3= arg1=Douglas arg2= arg3=

Shell Programming
IFS This is the internal-field separator Shell variable. One can assign the IFS variable to another character and use this character as the field separator. Example:

. $num_args a:b:c:d . This example shows only one argument, namely a:b:c:d. . $IFS=: . $num_args a:b:c:d
This example now shows four different arguments; each being separated by the new IFS, (:).

Shell Programming
PS1

This is the Shell prompt which lets you know that the shell is waiting for you to give it a command. The shell stores the prompt as a string variable in PS1. When you change the value of this variable, the appearance of the prompt will change. When you are working on several different machines, it might be useful to have the prompt be the name of the machine you are working on. Example
$PS1='domax0: ' domax0: Notice that prompt is now domax0:

Shell Programming
SHELL METACHARACTERS
Character Meaning

`
" | & ? * ; ;; > file >> file

Command substitution
Quotes Pipe character Run program in background Match one character Match any number of characters Command separator End of Case statement Standard output Append to standard output

Shell Programming
SHELL METACHARACTERS
$# $* $? $0-$9 $! && || [ 1-9] [] \ Number of arguments to script Arguments to script Status of previous command Command line arguments PID of last background job Short-circuit AND Short-circuit OR Match range of characters Test Avoid Expansion of Metacharecter

Shell Programming
expr The expr command will perform arithmetic operations in the Shell. Command format: `expr expression`
`back quotes

The arguments are taken as an expression. After the evaluation has taken place, the result is written to standard output. The terms of the expression must be separated by blanks. Special characters to the shell must be escaped. Strings containing blanks or other special characters must be quoted.
83

Shell Programming
Sample Session: $expr 7 + 8 + 10 25 $expr 10 - 8 2 expr will also work with user defined variables

$x=`expr 135 / 5` $ echo $x 27 $

`back quotes

84

Shell Programming
POSITIONAL PARAMETERS

A Shell script can also read in command-line arguments. The first argument is referred to as $1, the second is $2, and so on. Command-line arguments are referred to as positional parameters. Sample Session: If we type the name of the Shell script $vi neat_shell with no arguments, we get the following results. echo $1 $2 $3 echo $0 is the name Sample Session: of the shell script $sh neat_shell echo "There were neat_shell is the name $# arguments." of the shell script echo $* 85 There were 0 arguments.

Shell Programming
Reading Input Into a Shell Variable

The Shell script can read user input from standard input. The read command will read one line from standard input and assign the line to one or more variables. The following example shows how this works.
Sample Session: $vi read_script echo "Please enter a string of your choice" read a echo $a $
86

Shell Programming
Command Substitution

You can execute a command by enclosing it within two grave accent marks [these are sometimes called backquotes (`)]. The Shell will replace the command and the grave marks with the output from the command.
Sample Session: $vi dir dir=`pwd` echo 'You are using the' $dir 'directory' $

87

Shell Programming
Comments in Shell Scripts

Comments can be inserted into the Shell script by beginning each comment line with the pound symbol (#) or a colon (:). All characters after the comment character will be ignored by the shell.
Sample Session: $vi list # This program displays all the files and directories in long # listing format present in current working directory. echo ls
88

Shell Programming
Shell Environment - Exporting Variables

Within a process, you can declare, initialize, read, and modify variables. The variable is local to that process. When a process forks a child process, the parent process does not automatically pass the value of the variable to the child process.
Here is an example of the variables not being exported.

89

Shell Programming
Sample Session: $vi no_export car=mercedes echo $0 $car $$

sh inner echo $0 $car $$ $vi inner echo $0 $car $$ $ $sh no_export no_export mercedes 4790 inner 4792 no_export mercedes 4790 $

# set the variable # $0 = name of file executed # $car =value of variable car # $$ = PID number (process id) # execute another BourneShell script # display same as above
# display variables for this process

90

Shell Programming
Sample Session:

OUTPUT
$vi export_it car=mercedes export car echo $0 $car $$ sh inner1.sh echo $0 $car $$ $vi inner1.sh echo $0 $car $$ car=chevy echo $0 $car $$ $sh export_it export_it mercedes 4798 inner1 mercedes 4800 inner1 chevy 4800 export_it mercedes 4798

91

Shell Programming
In the export_it Shell script, the variable car was initialized to mercedes; and then it was exported. This means that the value of car is now available to a child process. When inner1 prints out the value of car it has the value of mercedes. The next line of inner1 changes the value of car to chevy. The last line of the session shows the return to the parent process and the value is still mercedes. Exporting variables is only valid from the parent to the child process.
92

Shell Programming
CONTROL CONSTRUCTS

The Shell control constructs can alter the flow of control within the script.
The Shell provides simple
two-way branch if statements, multiple-branch case statements, for, while, and until statements.
93

Shell Programming
test utility:

The test utility evaluates expressions and returns a condition indicating whether or not the expression is true (equal to zero) or false (not equal to zero). There are no options with this utility. The format for this utility is as follows:
# test expression expression - composed of constants, variables, and operators
94

Shell Programming
Test on Numeric Values Test expressions can be in many different forms. The expressions can appear as a set of evaluation criteria. The general form for testing numeric values is: int1 op int2 This criterion is true if the integer int1 has the specified algebraic relationship to integer int2. The valid operators (op) are: -eq equal -ne not equal -gt greater than -lt less than -ge greater than or equal -le less than or equal

95

Shell Programming
Test on Character Strings

The evaluation criterion for character strings is similar to numeric comparisons. The general form is:
string1 op string2 The operators (op) are: string1 = string2 string1 != string2 string1 true if string1 and string 2 are equal true if string1 and string2 are not equal true if string1 is not the null 96 string

Shell Programming
if then The format for this construct is: if expression then commands fi The if statement evaluates the expression and then returns control based on this status. The fi statement marks the end of the if, notice that fi is if spelled backward.
97

Shell Programming
The if statement executes the statements immediately following it if the expression returns a true status. If the return status is false, control will transfer to the statement following the fi. Sample Session:

$cat check_args if (test $# = 0) then echo 'Please supply at least 1 argument' exit fi echo 'Program is running' $
98

Shell Programming
if then else The format for this construct is: Command Format: if expression then commands else commands fi The else part of this structure makes the single-branch if statement into a two-way branch. If the expression returns a true status, the commands between the then and the else statement will be executed. After these have been executed, control will start again at the statement after the fi.
99

Shell Programming
Sample Session: $cat test_string number=1 numero=0001 if test $number = $numero then echo "String values of $number and $numero are equal" else echo "String values of $number and $numero not equal" fi if test $number -eq $numero then echo "Numeric values of $number and $numero are equal" else echo "Numeric values of $number and $numero not equal" fi

100

Shell Programming
if then elif The format for this construct is: Command Format: if expression then commands elif expression then commands else commands fi The elif construct combines the else and if statements and allows you to construct a nested set of if then else structures.
101

Shell Programming
case The format for this construct is:

case test-string in pattern-1 ) commands-1 ;; pattern-2 ) commands-2 ;; pattern-3 ) commands-3 ;; . . *) commands ;; esac
The case structure allows a multiple-branch decision mechanism. The path that is taken depends on a match between the test-string and one of the patterns.
102

Shell Programming
while The format for this construct is:

while expression do commands done

As long as the expression returns a true exit status, the structure continues to execute the commands between the do and the done statement. Before each loop through the commands, the structure executes the expression. When the exit status of the expression is false (non-zero), control is passed to the statement following the done statement. The commands to be executed must change the expression test or an infinite loop can result. 103

Shell Programming
until The format for this construct is:

until expression do commands done

The until and while structures are very similar. The only difference is that the test is at the top of the loop. The until structure will continue to loop until the expression returns true or a nonerror condition. The while loop will continue as long as a true or nonerror condition is returned. The commands to be executed must change the expression test or an infinite loop can result.
104

Shell Programming
for The format for this construct is: for loop-index in argument-list do commands done This structure will assign the value of the first item in the argument list to the loop index and executes the commands between the do and done statements. The do and done statements indicate the beginning and end of the for loop.
105

Shell Programming
After the structure passes control to the done statement, it assigns the value of the second item in the argument list to the loop index and repeats the commands. The structure will repeat the commands between the do and done statements once for each argument in the argument list. When the argument list has been exhausted, control passes to the statement following the done.
106

UNIX system/kernel structure

System Calls
The mechanism used to provide access to OS services (i.e., enter the operating system and perform a privileged operation ) is commonly known as a system call. The (only) difference between a procedure call and a system call is that a system call changes the execution mode of the CPU (to supervisor mode) whereas a procedure call does not. System call interface: A set of functions that are called by (user) programs to perform specific tasks.

System Calls
The system call code is physically located in the kernel. The kernel itself is stored in a separate area of memory - which is normally not accessible to the process. Therefore, the first thing that is required to execute a system call is to change to Kernel Mode - so that the kernel memory can be accessed. A system call is implemented by a ``software interrupt'' that transfers control to kernel code.

System Calls
- creat / open - read, write - lseek - close, unlink - dup/dup2 - fcntl - stat - select - sync

Opening a file
int creat (char *file_name, mode_t mode)
int open (char *file_name, int flags); int open (char *file_name, int flags, mode_t mode); Ex: fd - open("temp", O_RDWR|O_CREAT, 0744);

read & write

int read (int fd, void *buf, int count); It reads count bytes from the file and store the data into the buf.
int write (int fd, const void *buf, int count); It writes count bytes to the file, from the buf. On success return with: 0 -1 - Number of bytes written or read - indicates nothing was written - on error.

Random Access
int lseek (int fd, long int offset, int whence); whence: SEEK_SET - from the beginning SEEK_CUR - from the current position SEEK_END - from the end of file

Close a file
int close (fd); int unlink("file_name"); equivalent to $rm file_name;

Closing & Remove a file

write modifications to the disk file
void sync (void); Close a fd int close (fd); Removing a file

int unlink("file_name");
equivalent to $rm file_name;

Process Management
Introduction
Process is a program in execution. Processes carry out tasks in a system A process includes program counter (PC), CPU registers and process stacks, which contains temporary data. Unix is a multiprocessing system The UNIX kernel is reentrant

115

Processes : Introduction
A process uses many resources like memory space, CPU, files, etc., during its lifetime. Kernel should keep track of the processes and the usage of system resources. Kernel should distribute resources among processes fairly. Most important resource is CPU. In a multiprocessing environment, to attain an ideal performance of a system, the CPU utilization should be maximum.
116

Mode and space

In order to run Unix, the computer hardware must provide two modes of execution - kernel mode - user mode Some computers have more than two execution modes - eg: Intel processor. It has four modes of execution. Each process has virtual address space, references to virtual memory are translated to physical memory locations using set of address translation maps. 117

Context Switch
Execution control is changing from one process to another. When a current process either completes its execution or is waiting for a certain event to occur, the kernel saves the process context and removes the process from the running state. Kernel loads next runnable process's registers with pointers for execution. Kernel space: a fixed part of virtual address space of each process. It maps the kernel text and data structures.
118

Process structure
Every process is represented by a task_struct data structure. This structure is quite large and complex. When ever a new process is created a new taskstruct structure is created by the kernel and the complete process information is maintained by the structure. When a process is terminated, the corresponding structure is removed. Uses doubly linked list data structure.
119

Identifiers
Every process in the system has a process identifier.
The process identifier is not an index into the task vector, it is simply a number. Each process also has User and Group identifiers, these are used to control the process access to the files and devices in the system eg: ppid, pid, uid, gid, euid, egid
120

Process System calls

fork()

#include <sys/types.h> #include <unistd.h>

pid_t fork(void); When a fork() system call is made, the operating system generates a copy of the parent process which becomes the child process. If the fork system call is successful, the process ID of the child process is returned in the parent process and a 0 is returned in the child process. If the fork system call fails, it will return a -1. 121

Process System calls

The fork system call does not take an argument. The operating system will pass to the child process most of the parent's process information. However, some information is unique to the child process:
The child has its own process ID (PID) The child will have a different PPID than its parent All recorded locks on files are reset The action to be taken when receiving signals is different
122

example of fork()
#include <stdio.h> #include <sys/types.h> #include <unistd.h> int main(void) { printf("Hello \n"); fork(); printf("bye\n"); return 0; }
123

example of fork()
Hello - is printed once by parent process bye - is printed twice, once by the parent and once by the child If the fork system call is successful a child process is produced that continues execution at the point where it was called by the parent process. After the fork system call, both the parent and child processes are running and continue their execution at the next statement in the parent process.

A summary of fork() return values follows: if (fork()==-1) then fork() failed and there is no child if (fork()>0) then this is the parent if (fork()==0) then this is the child

124

Process System calls

wait()

#include <sys/types.h> #include <sys/wait.h>

pid_t wait(int *status); A parent process usually needs to synchronize its actions by waiting until the child process has either stopped or terminated its actions. The wait() system call allows the parent process to suspend its activities until one of these actions has 125 occurred.

Process System calls

A few additional notes about fork():
an orphan is a child process that continues to execute after its parent has finished execution (or died) to avoid this problem, the parent should execute: wait(&return_code);

The wait() system call accepts a single argument, which is a pointer to an integer and returns a value defined as type pid_t. If the calling process does not have any child associated with it, wait will return immediately with a value of -1. If any child processes are still active, the calling process will suspend its activity until a child process terminates. 126

Example of wait()
int status; int pid; pid = fork(); if (pid == 0) /* child process */ { printf("\n I'm the child!"); exit(0); } else /* parent process */ { wait(&status); printf("\n I'm the parent!") printf("\n Child returned: %d\n", status) }

127

Process System calls

getpid() #include <sys/types.h> #include <unistd.h> pid_t getpid(void); pid_t getppid(void);
getpid() returns the process id of the current process. The process ID is a unique positive integer identification number given to the process when it begins executing. getppid() returns the process id of the parent of the current process. The parent process forked the current child process. 128

Process System calls

exec*() #include <unistd.h> extern char **environ; int execl(const char *path, const char *arg, ...); int execlp(const char *file, const char *arg, ...); int execle(const char *path, const char *arg , ..., char * const envp[]); int execv(const char *path, char *const argv[]); int execvp(const char *file, char *const argv[]);
129

Process System calls

The naming convention: exec*

'l' indicates a list arrangement (a series of null terminated arguments) 'v' indicate the array or vector arrangement (like the argv structure). 'e' indicates the programmer will construct (in the array/vector format) and pass their own environment variable list 'p' indicates the current PATH string should be used when the system searches for executable files. NOTE: In the four system calls where the PATH string is not used (execl, execv, execle, and execve) the path to the program to be executed must be fully specified.
130

Process System calls

"The exec family of functions replaces the current process image with a new process image." Commonly a process generates a child process because it would like to transform the child process by changing the program code the child process is executing. The text, data and stack segment of the process are replaced and only the u (user) area of the process remains the same. If successful, the exec system calls do not return to the invoking program as the calling image is lost.
131

Signals

132

Signals
Signals are various notifications sent to a process in order to notify it of various "important" events. Signals are one of the oldest inter-process communication methods used by Unix systems. By their nature, they interrupt whatever the process is doing at that moment, and forces to handle them immediately. Each signal has an integer number that represents it (1, 2 and so on), as well as a symbolic name that is usually defined in the file /usr/include/signal.h A signal could be generated by a keyboard interrupt or an error condition such as the process attempting to access a non-existent location in its virtual 133 memory.

Signals
There are a set of defined signals that the kernel can generate or that can be generated by other processes in the system

134

Signals
Each signal in linux has a default action. If a signal's handler is set to the default action then the kernel will handle it. The SIGSTOP signal's default handler will change the current process's state to Stopped and then run the scheduler to select a new process to run. Alternatively, the process can specify its own signal handler.

Processes can block all the signals, with the exception of SIGSTOP and SIGKILL. If a blocked signal is generated, it remains pending until it is unblocked.
135

Signals
Signals have no relative priorities. The number of supported signals is limited to the word size of the processor. Processes with a word size of 32 bits can have 32 signals whereas 64 bit processors like the Alpha AXP may have up to 64 signals. Not every process in the system can send signals to every other process, the kernel can and super users can. Signals are generated by setting the appropriate bit in the task_struct's signal field. 136

System Calls for Signals

Handling Signals: signal() To give a signal a new action, you can use the signal() system call: #include <signal.h> sighandler_t signal(int signum, sighandler_t handler); Data Type: sighandler_t This is the type of signal handler functions. Signal handlers take one integer argument specifying the signal number, and have return type void. void handler (int signum) { ... } 137

Pre-defined Signal Handlers

For convenience, there are two pre-defined signal handler functions that can be used instead of writing our own: SIG_IGN: Causes the process to ignore the specified signal. For example, in order to ignore Ctrl-C completely (useful for programs that must NOT be interrupted in the middle, or in critical sections): signal(SIGINT, SIG_IGN); SIG_DFL: Causes the system to set the default signal handler for the given signal (i.e. the same handler the system would have assigned for the signal when the process started running): signal(SIGTSTP, SIG_DFL);
138

System Calls for Signals

Sending Signals: kill() #include <sys/types.h> #include <signal.h> int kill(pid_t pid, int sig); The kill system call can be used to send signal to any other process specified by pid.
If pid is positive, then signal sig is sent to pid. If pid equals 0, then sig is sent to every process in the process group of the current process.

A process can send itself a signal with the raise function. This function is declared in signal.h.

int raise (int signum)

The raise function sends the signal signum to the calling process. It returns zero if successful and a nonzero value if it fails. About the only reason for failure would be if the value 139 of signum is invalid.

System Calls for Signals

alarm()
Every process has an alarm clock stored in its system-data segment. When the alarm goes off, signal SIGALRM is sent to the calling process. The prototype for alarm() is: unsigned int alarm(seconds) unsigned int seconds; where seconds defines the time after which the UNIX system sends the SIGALRM signal to the calling process.

140

Example
#include <stdio.h> #include <sys/signal.h> void main() { signal (SIGALRM, times_up); alarm (10); while(1); /* endless loop. */ } int times_up(int sig) { printf("Caught signal #< %d >n", sig); exit(sig); /* return the signal number */ }
141

The POSIX Signal Environment

The sigaction function has the same basic effect as signal: to specify how a signal should be handled by the process. The basic signal function is a feature of ISO C, while sigaction is part of the POSIX.1 standard. #include<signal.h> int sigaction (int signal_number, struct sigaction *new_handler, struct sigaction *old_handler); Where signal_number specifies the signal for which a new handler is getting registered.

142

The POSIX Signal Environment

Data Type: struct sigaction
This structure contains the following members: struct sigaction { void (*sa_handler)(int) int sa_flags; sigset_t sa_mask; void (*sa_sigaction)(int, siginfo_t *, void *); void (*sa_restorer)(void); }

void (*sa_handler)(int)
This is used in the same way as the action argument to the signal function. The value can be SIG_DFL, SIG_IGN, or a function pointer.

143

The POSIX Signal Environment

int sa_flags
This specifies various flags which can affect the behavior of the signal. SA_NOCLDSTOP
If signum is SIGCHLD, do not receive notification when child processes stop (i.e., when child processes receive one of SIGSTOP, SIGTSTP, SIGTTIN or SIGTTOU).

SA_ONESHOT or SA_RESETHAND
Restore the signal action to the default state once the signal handler has been called.

SA_ONSTACK
Call the signal handler on an alternate signal stack provided by sigaltstack. If an alternate stack is not available, the default stack will be used.

SA_SIGINFO
The signal handler takes 3 arguments, not one. In this case, sa_sigaction should be set instead of sa_handler.
144

The POSIX Signal Environment

Blocking Signals for a Handler
When a signal handler is invoked, you need to block those signals that might confuse it or corrupt its data. When a handler function is invoked on a signal, that signal is automatically blocked during the time the handler is running. However, by default, other kinds of signals are not blocked; they can arrive during handler execution. The reliable way to block other kinds of signals during the execution of the handler is to use the sa_mask member of the sigaction structure.
sigset_t sa_mask
145

The POSIX Signal Environment

Data Type: sigset_t
The sigset_t data type is used to represent a signal set. All of the signal blocking functions use a data structure called a signal set to specify what signals are affected. Thus, every activity involves two stages: creating the signal set, and then passing it as an argument to a library function. 146

The POSIX Signal Environment

To add/delete individual signal specified by the signo to the signal set pointed to by set. int sigaddset(sigset_t *set, int signal_number);
Add a signal to a signal mask.

int sigdelset(sigset_t *set, int signal_number);

Remove a signal from a signal mask.

int sigfillset(sigset_t *set);

Fill (add all possible signals to) a signal mask.

int sigemptyset(sigset_t *set);

Clear a signal mask.
147

The POSIX Signal Environment

void (*sa_sigaction)(int, siginfo_t *, void *) sa_sigaction also specifies the action to be associated with signum. This function receives the
signal number as its first argument, a pointer to a siginfo_t as its second argument and a void pionter as its third argument.
The siginfo_t parameter to sa_sigaction is a struct with the following elements siginfo_t { int si_signo; /* Signal number */ int si_errno; /* An errno value */ int si_code; /* Signal code */ pid_t si_pid; /* Sending process ID */ uid_t si_uid; /* Real user ID of sending process */ int si_status; /* Exit value or signal */ clock_t si_utime; /* User time consumed */ clock_t si_stime; /* System time consumed */ sigval_t si_value; /* Signal value */ int si_int; /* POSIX.1b signal */ void * si_ptr; /* POSIX.1b signal */ void * si_addr; /* Memory location which caused fault */ int si_band; /* Band event */ 148 int si_fd; /* File descriptor */ }

void (*sa_restorer)(void) The sa_restorer element is obsolete and should not be used. POSIX does not specify a sa_restorer element.

Process Signal Mask

The collection of signals that are currently blocked is called the signal mask. Each process has its own signal mask. When you create a new process it inherits its parent's mask. You can block or unblock signals by modifying the signal mask. #include <signal.h> int sigprocmask(int how, sigset_t *new_set, sigset_t *old_set); Sigprocmask() examines or manipulates the signal mask. This mask is the set of signals that are currently blocked. The how argument determines the action that must be performed. How can be one of:
SIG_BLOCK: Add the signals referenced by set to the mask. SIG_UNBLOCK: Remove the signals referenced by set from the mask. SIG_SETMASK: Set the signal mask to the set referenced by set.

Returns 0 on success and -1 on error.

149

Checking for Pending Signals

One can find out which signals are pending at any time by calling sigpending. #include <signal.h> int sigpending(sigset_t *set) Sigpending() returns the set of signals that are waiting to be delivered, currently blocked by the signal mask. One can test whether a particular signal is a member of this set using sigismember Testing whether a signal is pending is not often useful. Returns 0 on success and -1 on error.

int sigismember (const sigset_t *set, int signum)

The sigismember function tests whether the signal signum is a member of the signal set set. It returns 1 if the signal is in the set, 0 if not, and -1 if there150 is an error.

Waiting for a Signal

If the program is driven by external events, or uses signals for synchronization, then when it has nothing to do it should probably wait until a signal arrives.

#include <signal.h> int sigsuspend(const sigset_t *set) This function replaces the process's signal mask with set and then suspends the process until a signal is delivered whose action is either to terminate the process or invoke a signal handling function. In other words, the program is effectively suspended until one of the signals that is not a member of set arrives. If the process is woken up by delivery of a signal that invokes a handler function, and the handler function returns, then sigsuspend also returns. The mask remains set only as long as sigsuspend is waiting. The function sigsuspend always restores the previous signal 151 mask when it returns.

Inter Process Communications

Inter Process Communications

Introduction In a multiprocessing environment, often many processes are in need to communicate with each other and share some of the resources. IPC mechanisms have many distinct purposes: for example - Data transfer - Event notification - Synchronisation - Sharing data - Resource sharing

The UNIX operating system provides a rich set of features that allows processes to communicate with each other.

IPC Mechanisms
Primitive - Unnamed pipe - Named pipe (FIFO) System V IPC - Message queues - Shared memory - Semaphores POSIX 1.b - Message queues - Shared memory - Semaphores Socket Programming

Unnamed Pipes
There is no form of IPC that is simpler than pipes. In Unix, a pipe is a unidirectional, stream communication that allows related-processes to communicate. One process writes to the ``write end'' of the pipe, and a second process reads from the ``read end'' of the pipe. The order in which data is written to the pipe, is the same order as that in which data is read from the pipe. pipe is a data structure in the kernel.

Unnamed Pipes
A pipe is created by using the pipe system call int pipe(int* filedes); A pipe consists of
two descriptors, one for reading, one for writing. reading from the pipe advances the read pointer writing to the pipe advances the write pointer operating system buffers data in the pipe (Unix pipe 4096 bytes (4K)) operating system blocks reads of empty pipe operating system blocks writes to full pipe pipe data consists of unstructured character stream

Pipe Creation & Usage

First, a process creates a pipe, and then forks to create a copy of itself. Parent opens & writes file, child reads file
parent closes read end of pipe child closes write end of pipe

EXAMPLE
#define DATA "hello world" #define BUFFSIZE 1024 int rgfd[2]; /* file descriptors of streams */ main() { char sbBuf[BUFFSIZE]; pipe(rgfd);

if (fork()) { /* parent, read from pipe */ close(rgfd[1]); /* close write end */ read(rgfd[0], sbBuf, BUFFSIZE); printf("-->%s\n", sbBuf); close(rgfd[0]); } else { /* child, write data to pipe */ close(rgfd[0]); /* close read end */ write(rgfd[1], DATA, sizeof(DATA)); close(rgfd[1]); exit(0); } }

Pipes (Bi-directional)
If one wants to use pipes for bidirectional communication. The algorithm looks like
parent creates pipe 1, pipe 2 fork parent closes read end of pipe1 parent closes write end of pipe2 child closes write end of pipe1 child closes read end of pipe2

Named pipe (FIFO)

One limitation of anonymous pipes is that only processes 'related' to the process that created the pipe may communicate using them. If we want two un-related processes to communicate via pipes, we need to use named pipes. A named pipe (FIFO) is a pipe whose access point is a file kept on the file system. By opening this file for reading, a process gets access to the reading end of the pipe. By opening the file for writing, the process gets access to the writing end of the pipe. If a process opens the file for reading, it is blocked until another process opens the file for writing. The same goes the other way around.

Creating A Named Pipe With The mknod Command

A named pipe may be created either via the 'mknod' (or its newer replacement, 'mkfifo'), or via the mknod() system call (or by the POSIX-compliant mkfifo() function). To create a named pipe with the file named 'prog_pipe', we can use the following command: mknod prog_pipe p We could also provide a full path to where we want the named pipe created. If we then type 'ls -l prog_pipe', we will see something like this:

prw-rw-r-- 1 choo choo 0 Nov 7 01:59 prog_pipe The 'p' on the first column denotes this is a named pipe. Just like any file in the system, it has access permissions, that define which users may open the named pipe, and whether for reading, writing or both.

Opening/Reading/Writing From/To A Named Pipe

Opening a named pipe is done just like opening any other file in the system, using the open() system call, or using the fopen() standard C function. If the call succeeds, we get a file descriptor (in the case of open()), or a 'FILE' pointer (in the case of fopen()). Reading from a named pipe is very similar to reading from a file, and the same goes for writing to a named pipe. Yet there are several differences:
Either Read Or Write - a named pipe cannot be opened for both reading and writing. The process opening it must choose one mode, and stick to it until it closes the pipe. Read/Write Are Blocking - when a process reads from a named pipe that has no data in it, the reading process is blocked. It does not receive an end of file (EOF) value, like when reading from a file. When a process tries to write to a named pipe that has no reader (e.g. the reader process has just closed the named pipe), the writing process gets blocked, until a second process re-opens the named pipe.

EXAMPLE
#include<stdio.h> #include<fcntl.h> #include<sys/stat.h> int main() { int x,y,z; mknod("./fif",S_IFIFO|0666,0); x=open("fif",O_RDONLY); write(x,"hello",5); close(x); unlink("fif"); } #include<stdio.h> #include<fcntl.h> int main() { int x,y,z; char a[10]; x=open("fif",O_WRONLY); read(x,a,5); write(1,a,5); close(x); }

System V IPC
Linux supports three types of interprocess communication mechanisms which first appeared in Unix System V. These are message queues, semaphores and shared memory. These System V IPC mechanisms all share common authentication methods. Each IPC object has a unique IPC identifier associated with it. When we say ``IPC object'', we are speaking of a single message queue, semaphore set, or shared memory segment. This identifier is used within the kernel to uniquely identify an IPC object.

System resources only by passing a V IPC Processes may access these

unique reference identifier to the kernel via system calls. Access to these System V IPC objects is checked using access permissions, much like accesses to files are checked. The access rights to the System V IPC object is set by the creator of the object via system calls. All Linux data structures representing System V IPC objects in the system include an ipc_perm structure which contains the owner and creator processes user and group identifiers, the access mode for this object (owner, group and other) and the IPC object's key.

System V IPC
To obtain a unique ID, a key must be used. The key must be mutually agreed upon by both client and server processes. Two sets of key are supported: public and private. Private means that it may be accessed only by the process that created it, or by child processes of this process. Public means that it may be potentially accessed by any process in the system, except when access permission modes state otherwise.

The ipcs Command

The ipcs command can be used to obtain the status of all System V IPC objects.

Command
ipcs ipcs ipcs ipcs ipcs

Option
-q: -s: -m: --help:

Action

to check usage of all SysV IPCs Show only message queues Show only semaphores Show only shared memory Additional arguments

The 'ipcrm' command accepts a resource type ('shm', 'msg' or 'sem') and a resource ID, and removes the given resource from the system. ipcrm <msg | sem | shm> <IPC ID> We need to have the proper permissions in order to delete a resource.

Message Queues
One of the problems with pipes is that it is up to you, as a programmer, to establish the protocol. With a stream taken from a pipe, it means you have to somehow parse the bytes, and separate them to packets. Other problem is that data sent via pipes arrives in FIFO order. A message queue is a queue onto which messages can be placed. Message queues allow one or more processes to write messages which will be read by one or more reading processes. Each message queue (of course) is uniquely identified by an IPC identifier. Linux maintains a list of message queues, the msgque vector; each element of which points to msqid_ds data structure which fully describes the message queue.

 When message queues are created a new msqid_ds data structure is allocate from system memory and inserted into the vector. Each msqid_ds data structure contains an ipc_perm data structure and pointers to the messages entered onto this queue. A message is composed of a message type (which is a number), and message data. A message queue can be either private, or public. If it is private, it can be accessed only by its creating process or child processes of that creator. If it's public, it can be accessed by any process that knows the queue's key. Several processes may write messages onto a message queue, or read messages from the queue. Messages may be read by type, and thus not have to be read in a FIFO order as is the case with pipes.

Message Queues

Creating A Message Queue - msgget()

The msgget() system call is used to create message queue. int msgget ( key_t key, int msgflg ); RETURNS: message queue identifier on success 1 on error: This system call accepts two parameters - a queue key, and flags. The key may be one of:
IPC_PRIVATE - used to create a private message queue. a positive integer - used to create (or access) a publiclyaccessible message queue.

 The second parameter contains flags that control how the system call is to be processed. IPC_CREAT
Create the queue if it doesn't already exist in the kernel.

Creating A Message Queue - msgget()

IPC_EXCL
When used with IPC_CREAT, fail if queue already exists.

If IPC_CREAT is used alone, msgget() either returns the message queue identifier for a newly created message queue, or returns the identifier for a queue which exists with the same key value. If IPC_EXCL is used along with IPC_CREAT, then either a new queue is created, or if the queue exists, the call fails with 1. An optional octal mode may be OR'd into the mask, since each IPC object has permissions that are similar in functionality to file permissions on a UNIX file system!

Writing Messages Onto A Queue msgsnd()

Once we have the queue identifier, we can begin performing operations on it. To deliver a message to a queue, you use the msgsnd system call: int msgsnd ( int msqid, struct msgbuf *msgp, int msgsz, int msgflg ); The first argument to msgsnd is our queue identifier, returned by a previous call to msgget. The second argument, msgp, is a pointer to our redeclared and loaded message buffer.
struct msgbuf { long mtype; /* message type, positive number (cannot be 0 ). */ char mtext[1]; /* message body array. larger than one byte. */ };

 The msgsz argument contains the size of the message in bytes, excluding the length of the message type (4 byte long). The msgflg argument can be set to 0 (ignored), or: IPC_NOWAIT
If the message queue is full, then the message is not written to the queue, and control is returned to the calling process. If not specified, then the calling process will suspend (block) until the message can be written.

Writing Messages Onto A Queue msgsnd()

RETURNS: 0 on success -1 on error.

Reading A Message From The Queue msgrcv() int msgrcv ( int msqid, struct msgbuf *msgp, int
msgsz, long mtype, int msgflg ); This system call accepts the following list of parameters:
int msqid - id of the queue, as returned from msgget(). struct msgbuf* msg - a pointer to a pre-allocated msgbuf structure. It should generally be large enough to contain a message with some arbitrary data (see more below). int msgsz - size of largest message text we wish to receive. Must NOT be larger than the amount of space we allocated for the message text in 'msg'. int msgtyp - Type of message we wish to read. may be one of:
0 - The first message on the queue will be returned.

Reading A Message From The Queue msgrcv() a positive integer - the first message on the queue whose
type (mtype) equals this integer (unless a certain flag is set in msgflg, see below). a negative integer - the first message on the queue whose type is less than or equal to the absolute value of this integer.

int msgflg - a logical 'or' combination of any of the following flags:

IPC_NOWAIT - if there is no message on the queue matching what we want to read, return '-1', MSG_NOERROR - If a message with a text part larger than 'msgsz' matches what we want to read, then truncate the text when copying the message to our msgbuf structure. If this flag is not set and the message text is too large, the system call returns '-1.

msgctl()
To perform control operations on a message queue, you use the msgctl() system call.
int msgctl ( int msgqid, int cmd, struct msqid_ds *buf );

Parameters:
int msqid - id of the queue, as returned from msgget(). Int cmd - process the following command
IPC_STAT Retrieves the msqid_ds structure for a queue, and stores it in the address of the buf argument. IPC_RMID Removes the queue from the kernel.

Semaphores
One of the problems when writing multi-process application is the need to synchronize various operations between the processes. Communicating requests using pipes, sockets and message queues is one way to do it. However, sometimes we need to synchronize operations amongst more than two processes, or to synchronize access to data resources that might be accessed by several processes in parallel. Semaphores are a means supplied with SysV IPC that allow us to synchronize such operations. A semaphore is a resource that contains an integer value, and allows processes to synchronize by testing and setting this value in a single atomic operation. As message queues, the kernel maintains special internal data structure for each semaphore set which exists within its addressing space. This structure is of type semid_ds.

Semaphores
Two types of operations can be carried on a semaphore: wait and signal. A set operation first checks if the semaphore's value equals some number. If it does, it decreases its value and returns. If it does not, the operation blocks the calling process until the semaphore's value reaches the desired value. A signal operation increments the value of the semaphore, possibly awakening one or more processes that are waiting on the semaphore. A semaphore set is a structure that stores a group of semaphores together, and possibly allows the process to commit a transaction on part or all of the semaphores in the set together.

Creating A Semaphore Set semget() the semget() system call. Creation of a semaphore set is done using
#include <sys/sem.h> int semget(key_t key, int nsems, int semflg);

/* ID of the semaphore set. */ int sem_id_1; int sem_id_2; /* create a private semaphore set with one semaphore in it, with access only to the owner. */ sem_id_1 = semget (IPC_PRIVATE, 1, IPC_CREAT | 0600);

/* create a semaphore set with ID 250, three semaphores */ /* in the set, with access only to the owner. */ sem_id_2 = semget (250, 3, IPC_CREAT | 0600);

Setting And Getting Semaphore Values With semctl()

After the semaphore set is created, we initialize the value of semaphores in the set using the semctl() system call. Assume we want to set values of the three semaphores in our second set to values 3, 6 and 0, respectively. The ID of the first semaphore in the set is '0', the ID of the second semaphore is '1', and so on. /* use this to store return values of system calls. */ int rc; /* initialize the first semaphore in our set to '3'. */ rc = semctl(sem_id_2, 0, SETVAL, 3);

/* initialize the second semaphore in our set to '6'. */ rc = semctl(sem_id_2, 1, SETVAL, 6);
/* initialize the third semaphore in our set to '0'. */ rc = semctl(sem_id_2, 2, SETVAL, 0);

Using Semaphores Operations With semop()

The semop() function performs operations on semaphores in a semaphore set.

#include <sys/sem.h> int semop(int semid, struct sembuf *sops, size_t nsops);
Each semaphore operation specified by the sops array is performed on the semaphore set specified by semid. The entire array of operations is performed atomically; no other thread will operate on semaphore set until all of the operations are done or it is determined that they cant be done. The members of the sembuf structure are as follows:
unsigned short sem_num The semaphore number in semaphore set. short sem_op The operation to perform on the semaphore. short sem_flg The operation flags.

Using Semaphores Operations With semop()

The semop() function changes each semaphore specified by sem_num according to the value of sem_op as follows:
If sem_op is positive, semop() increments the value of the semaphore and wakes up any threads waiting for the semaphore to increase. If sem_op is negative, semop() attempts to decrement the value of the semaphore. If the result would be negative, it waits for the semaphore value to increase. If the result would be positive, it decrements the semaphore. If the result would be zero, it decrements the semaphore and wakes up any threads waiting for the semaphore to be zero. If sem_op is zero, the thread waits for the semaphore's value to be zero.

Using Semaphores Operations With semop()

If IPC_NOWAIT is set in sem_flg and the operation cannot be completed, semop() returns with a return value of -1 and errno set to EAGAIN instead of causing the thread to wait. If SEM_UNDO is set in sem_flg, semop() causes IPC to reverse the effect of this semaphore operation when the thread ends, effectively releasing the resources or request for resources controlled by the semaphore. This value is known as the semaphore adjustment value.

nsops
(Input) Number of sembuf structures in sops array.

Return Value
0 semop() was successful. -1 semop() was not successful.

 The semctl() function allows the caller to control the semaphore set specified by the semid parameter.

Perform Semaphore Control Operation semctl()

#include <sys/sem.h> int semctl(int semid, int semnum, int cmd, ...);
A semaphore set is controlled by setting the cmd parameter to one of the following values: SETVAL
Set the value of semaphore semnum to the integer value of type int specified in the fourth parameter and clear the associated per-thread semaphore adjustment value.

SETALL
Set the values of each semaphore in the semaphore set to the values contained in the array pointed to by the fourth parameter, which is a pointer to an array of type unsigned short.

Perform Semaphore Control Operation semctl()

According to the manual, the last parameter for this system call should be a union of type union semun. However, since SETVAL (set value) operation only uses the int val part of union, we simply passed an integer to the function. The proper way to use this system call was to define a variable of this union type, and set its value appropriately, like this: /* use this variable to pass the value to the semctl() call */ union semun { int val; struct semid_ds *buf; unsigned short *array; } arg; /* initialize the first semaphore in our set to '3'. */ union semun sem_val; sem_val.val = 0; rc = semctl(sem_set_id_2, 2, SETVAL, sem_val);

Perform Semaphore Control Operation semctl()

IPC_RMID
Remove the semaphore set identifier semid from the system and destroy the set of semaphores. Any threads that are waiting in semop() are woken up and semop() returns with a return value of -1.

IPC_STAT
Store the current value of each member of the semid_ds data structure into the structure pointed to by the fourth parameter. The IPC_STAT command requires read permission to the semaphore set.

GETNCNT
Return the number of threads waiting for the value of semaphore semnum to increase. This value is the semncnt value in the semaphore_t data structure associated with the specified semaphore.

GETVAL
Return the current value of semaphore semnum.

GETALL
Return the values of each semaphore in the semaphore set into the array pointed to by the fourth parameter, which is a pointer to an array of type unsigned short.

GETZCNT
Return the number of threads waiting for the value of semaphore semnum to reach zero.

EXAMPLE
#include<stdio.h> #include<sys/sem.h> #include<sys/ipc.h> #include<stdlib.h> union semun { int x; struct semid_ds *y; unsigned short *array; }; int id; struct sembuf op; union semun s; unsigned short a[ ]={1,1}; int main() { s.array=a; id=semget(0x11,2,IPC_CREAT); semctl(id,0,SETALL,s); op.sem_num=0; op.sem_op=-1; op.sem_flg=0; semop(id,&op,1); sleep(10); printf("HELLO\n"); op.sem_num=0; op.sem_op=1; op.sem_flg=0; semop(id,&op,1); }

 #include<stdio.h> #include<sys/sem.h> #include<sys/ipc.h> #include<stdlib.h>

EXAMPLE (Cont..)

int main() { int id; struct sembuf op; id=semget(0x11,1,IPC_CREAT); op.sem_num=0; op.sem_op=-1; op.sem_flg=0; semop(id,&op,1); printf("WORLD\n"); op.sem_num=0; op.sem_op=1; op.sem_flg=0; semop(id,&op,1); }

Shareddescribed as the mapping of an memory Shared memory can best be

area (segment) of memory that will be mapped and shared by more than one process. This is by far the fastest form of IPC, because there is no intermediation (i.e. a pipe, a message queue, etc). Instead, information is mapped directly from a memory segment, and into the addressing space of the calling process. A segment can be created by one process, and subsequently written to and read from by any number of processes. As with message queues and semaphore sets, the kernel maintains a special internal data structure for each shared memory segment which exists within its addressing space. This structure is of type shmid_ds, and is defined in linux/shm.h

 In order to create a new memory segment, or access an existing segment, the shmget() system call is used. PROTOTYPE: int shmget ( key_t key, int size, int shmflg); The first argument to shmget() is the key value. This key value is then compared to existing key values that exist within the kernel for other shared memory segments. At that point, the open or access operation is dependent upon the contents of the shmflg argument. IPC_CREAT
Create the segment if it doesn't already exist in the kernel.

SYSTEM CALL: shmget()

IPC_EXCL
When used with IPC_CREAT, fail if segment already exists.

RETURNS:
shared memory segment identifier on success -1 on error: errno = EINVAL (Invalid segment size specified)

 Once a process has a valid IPC identifier for a given segment, the next step is for the process to attach or map the segment into its own addressing space. int shmat ( int shmid, char *shmaddr, int shmflg); RETURNS:
address at which segment was attached to the process or -1 on error.

SYSTEM CALL: shmat()

If the addr argument is zero (0), the kernel tries to find an unmapped region. This is the recommended method. An address can be specified to resolve conflicts with other apps. The SHM_RND flag can be OR'd into the flag argument to force a passed address to be page aligned (rounds down to the nearest page size).

SYSTEM CALL: shmat()

In addition, if the SHM_RDONLY flag is OR'd in with the flag argument, then the shared memory segment will be mapped in, but marked as readonly. Once a segment has been properly attached, and a process has a pointer to the start of that segment, reading and writing to the segment become as easy as simply referencing or dereferencing the pointer. Be careful not to lose the value of the original pointer, If this happens, you will have no way of accessing the base (start) of the segment.

SYSTEM CALL: shmctl()

int shmctl ( int shmqid, int cmd, struct shmid_ds *buf ); RETURNS:
0 on success -1 on error

This particular call is modeled directly after the msgctl call for message queues. In light of this fact, it won't be discussed in too much detail. Valid command values are: IPC_STAT
Retrieves the shmid_ds structure for a segment, and stores it in the address of the buf argument

IPC_RMID
Marks a segment for removal.

The IPC_RMID command doesn't actually remove a segment from the kernel. Rather, it marks the segment for removal. The actual removal itself occurs when the last process currently attached to the segment has properly detached it. Of course, if no processes are currently attached to the segment, the removal seems immediate.

SYSTEM CALL: shmdt()

To properly detach a shared memory segment, a process calls the shmdt system call. int shmdt ( char *shmaddr ); RETURNS:
0 on success -1 on error

EXAMPLE
#include<stdio.h> #include<sys/shm.h> #include<string.h> #include<stdio.h> #include<sys/shm.h> #include<string.h> int main() int main() { { int x,y,z; int x,y,z; char *a; x=shmget(0x30,1024, char *a; IPC_CREAT|0666); x=shmget(0x30,1024,IPC_CREAT|0 666); a=shmat(x,NULL,0); strcpy(a,"hello"); a=shmat(x,NULL,0); sleep(5); sleep(5); write(1,8,a); printf("\n%s\n",a); strcpy(a+6,"bye"); shmdt(a); shmctl(x,IPC_RMID); shmdt(a); } }

POSIX Clocks &Timers

Realtime applications must be able to operate on data within strict timing constraints in order to schedule application or system events. Timing requirements can be in response to the need for either high system throughput or fast response time. Applications requiring high throughput may process large amounts of data and use a continuous stream of data points equally spaced in time. The correctness of realtime applications often depends on satisfying timing constraints. A systemwide clock is the primary source for synchronization and high-resolution timers to support realtime requirements for scheduling events.

Clock Functions
The supported time-of-day clock is the CLOCK_REALTIME clock, defined in the time.h header file. The CLOCK_REALTIME clock is a systemwide clock, visible to all processes running on the system. The CLOCK_REALTIME clock measures the amount of time that has elapsed since 00:00:00 January 1, 1970 Greenwich Mean Time (GMT). The CLOCK_REALTIME clock measures time in nanoseconds; clock resolution does not reflect fractions of nanoseconds.

Function
clock_settime clock_gettime clock_getres

Description
Set clock to a specified value Get value of clock Get resolution of clock

Clock Functions
#include <time.h> int clock_getres(clockid_t clk_id, struct timespec *res); int clock_gettime(clockid_t clk_id, struct timespec *tp); int clock_settime(clockid_t clk_id, const struct timespec *tp); DESCRIPTION The function clock_getres() finds the resolution (precision) of the specified clock clk_id, The functions clock_gettime() and clock_settime() retrieve and set the time of the specified clock clk_id. struct timespec { time_t tv_sec; /* seconds */ long tv_nsec; /* nanoseconds */ }; All implementations support the system-wide realtime clock, which is identified by CLOCK_REALTIME.

EXAMPLE
#include<stdio.h> #include<time.h> int main() { struct timespec s1,s2; int x,y; clock_gettime(CLOCK_REALTIME,&s1); printf("%d\n",s1.tv_sec); printf("%d\n",s1.tv_nsec); clock_getres(CLOCK_REALTIME,&s2); printf("%d\n",s2.tv_sec); printf("%d\n",s2.tv_nsec); }

Timers
Two types of timers are provided to support realtime timing facilities: one-shot timers and periodic timers. Timers can be set up to expire only once (one-shot) or on a repetitive (periodic) schedule. A one-shot timer is armed with an initial expiration time, expires only once, and then is disarmed. A timer becomes a periodic timer with the addition of a repetition value. The timer expires, then loads the repetition interval, rearming the timer to expire after the repetition interval has elapsed. The initial expiration value can be relative to the current time or an absolute time value. A relative timer has an initial expiration time based on the amount of time elapsed, such as 30 seconds from the start of the application or 0.5 seconds from the last timer expiration. An absolute timer expires at a calendar date and time.

Timers
Function timer_create timer_delete timer_settime timer_gettime timer_getoverrun Description Create a timer Delete a timer Set and arm or disarm a timer Get remaining interval for an active timer Get current overrun count for a timer

One create a timer with the timer_create function, which is associated with a sigevent structure. To use signals with timers, include the following steps in your application: Create and declare a signal handler. Set the sigevent structure to specify the signal you want sent on timer expiration. Establish a signal handler with the sigaction function. Create the timer. Set the timer.

Timers
The timespec and itimerspec data structures are used in many realtime clock and timer functions. The timespec data structure contains members for both second and nanosecond values. The itimerspec data structure contains two timespec data structures. This data structure sets up an initial timer and repetition value . struct itimerspec { struct timespec it_interval; /* Timer interval */ struct timespec it_value; /* Initial expiration */ };

Timers
#include <time.h> #include <signal.h> int timer_create(clockid_t clockid, struct sigevent *evp, timer_t *timerid); The timer_create() function creates a per-process timer using the specified clock, clock_id, as the timing base.

int timer_settime(timer_t timerid, int flags, const struct itimerspec *value, struct itimerspec *ovalue); The timer_settime() function sets the time until the next expiration of the timer specified by timerid from the it_value member of the value argument and arm the timer if the it_value member of value is non-zero. If the specified timer was already armed when timer_settime() is called, this call resets the time until next expiration to the value specified. If the it_value member of value is zero, the timer is disarmed.

Timers
If the flag TIMER_ABSTIME is not set in the argument flags, timer_settime() behaves as if the time until next expiration is set to be equal to the interval specified by the it_value member of value. That is, the timer expires in it_value nanoseconds from when the call is made. If the flag TIMER_ABSTIME is set in the argument flags, timer_settime() behaves as if the time until next expiration is set to be equal to the difference between the absolute time specified by the it_value member of value and the current value of the clock associated with timerid. That is, the timer expires when the clock reaches the value specified by the it_value member of value. If the argument ovalue is not NULL, the function timer_settime() stores, in the location referenced by ovalue, a value representing the previous amount of time before the timer would have expired or zero if the timer was disarmed.

Timers
int timer_gettime(timer_t timerid, struct itimerspec *value); int timer_getoverrun(timer_t timerid); The timer_gettime() function stores the amount of time until the specified timer, timerid, expires and the reload value of the timer into the space pointed to by the value argument. The it_value member of this structure contains the amount of time before the timer expires, or zero if the timer is disarmed. The timer_getoverrun() function returns the timer expiration overrun count for the specified timer.

Timers
int timer_delete(timer_t timerid); The timer_delete() function deletes the specified timer, timerid, previously created by the timer_create() function. If the timer is armed when timer_delete() is called, the behavior shall be as if the timer is automatically disarmed before removal.

int nanosleep(const struct timespec *rqtp, struct timespec *rmtp);

To suspend process execution temporarily using the POSIX timer interface, call the nanosleep function. The nanosleep function suspends execution for a specified number of nanoseconds, providing a high-resolution sleep. If the rmtp argument is non-NULL, the timespec structure referenced by it is updated to contain the amount of time remaining in the interval (the requested time minus the time actually slept).

EXAMPLE
int main() { sigemptyset(&s); sigaddset(&s,SIGINT); event.sigev_notify=SIGEV_SIGNAL; timer_t t; event.sigev_signo=SIGINT; int x,y; signal(SIGINT,handler); struct itimerspec s1,s2; timer_create(CLOCK_REALTIME, struct sigevent event; &event, &t); sigset_t s; s1.it_value.tv_sec=1; s1.it_interval.tv_sec=1; void handler(int p) timer_settime(t,0,&s1,NULL); { sigprocmask(SIG_BLOCK,&s,NULL); int q; printf("GOT THE SIGNAL\n"); sleep(6); sigprocmask(SIG_UNBLOCK,&s, y=timer_getoverrun(t); NULL); printf("%d\n",q); } } #include<stdio.h> #include<time.h> #include<signal.h>

Asynchronous Input and Output

I/O operations on a file can be either synchronous or asynchronous. For synchronous I/O operations, the process calling the I/O request is blocked until the I/O operation is complete and regains control of execution only when the request is completely satisfied or fails. For asynchronous I/O operations, the process calling the I/O request immediately regains control of execution once the I/O operation is queued to the device. When the I/O operation is completed (either successfully or unsuccessfully), the calling process can be notified of the event by a signal passed through the aiocb structure for the asynchronous I/O function. Alternatively, the calling process can poll the aiocb structure for completion status.

Data Structures Associated with Asynchronous I/O

The POSIX.1b asynchronous I/O functions use the asynchronous I/O control block aiocb.
The aiocb structure contains the following members:

int aio_fildes; /* File descriptor */ off_t aio_offset; /* File offset */ volatile void *aio_buf; /* Pointer to buffer */ size_t aio_nbytes; /* No. of bytes to transfer */ int aio_reqprio; /* Request priority offset */ struct sigevent aio_sigevent; /* Signal structure */ int aio_lio_opcode; /* Specifies type of I/O operation */

Data Structures Associated with Asynchronous I/O int aio_fildes

This element specifies the file descriptor to be used for the operation. It must be a legal descriptor, otherwise the operation will fail. The device on which the file is opened must allow the seek operation. I.e., it is not possible to use any of the AIO operations on devices like terminals where an lseek call would lead to an error. off_t aio_offset This element specifies the offset in the file at which the operation (input or output) is performed. volatile void *aio_buf This is a pointer to the buffer with the data to be written or the place where the read data is stored.

Data Structures Associated with Asynchronous I/O

size_t aio_nbytes This element specifies the length of the buffer pointed to by aio_buf. int aio_reqprio If the platform has defined _POSIX_PRIORITIZED_IO, AIO requests are processed based on the current scheduling priority. The aio_reqprio element can then be used to lower the priority of the AIO operation. struct sigevent aio_sigevent This element specifies how the calling process is notified once the operation terminates. If the sigev_notify element is SIGEV_NONE, no notification is sent. If it is SIGEV_SIGNAL, the signal determined by sigev_signo is sent.

Data Structures Associated with Asynchronous I/O

int aio_lio_opcode This element is only used by the lio_listio function. Since this function allow an arbitrary number of operations to start at once, and each operation can be input or output (or nothing), the information must be stored in the control block. The possible values are:
LIO_READ Start a read operation. Read from the file at position aio_offset and store the next aio_nbytes bytes in the buffer pointed to by aio_buf. LIO_WRITE Start a write operation. Write aio_nbytes bytes starting at aio_buf into the file starting at position aio_offset. LIO_NOP Do nothing for this control block. This value is useful sometimes when an array of struct aiocb values contains holes, i.e., some of the values must not be handled although the whole array is presented to the lio_listio function.

Asynchronous Read and Write Operation

A typical application using asynchronous I/O includes the following steps:
Create and fill the asynchronous I/O control block (aiocb). Call the open function to open a specified file and get a file descriptor for that file. After a call to the open function, the file pointer is set to the beginning of the file. Select flags as appropriate. If you use signals, establish a signal handler to catch the signal returned on completion of the asynchronous I/O operation. Call the aio_read, aio_write, function to request asynchronous I/O operations. Call aio_suspend if your application needs to wait for the I/O operations to complete; or continue execution and poll for completion with aio_error; or continue execution until the signal arrives. After completion, call the aio_return function to retrieve completion value. Call the close function to close the file. The close function waits for all asynchronous I/O to complete before closing the file

Asynchronous Read and Write Operation

Function: int aio_read (struct aiocb *aiocbp) This function initiates an asynchronous read operation. It immediately returns after the operation was enqueued or when an error was encountered. The first aiocbp->aio_nbytes bytes of the file for which aiocbp->aio_fildes is a descriptor are written to the buffer starting at aiocbp->aio_buf. Reading starts at the absolute position aiocbp->aio_offset in the file. The calling process is notified about the termination of the read request according to the aiocbp->aio_sigevent value. When aio_read returns, the return value is zero if no error occurred or returns -1.

Asynchronous Read and Write Operation

Function: int aio_write (struct aiocb *aiocbp)
This function initiates an asynchronous write operation. The function call immediately returns after the operation was enqueued or if before this happens an error was encountered. The first aiocbp->aio_nbytes bytes from the buffer starting at aiocbp->aio_buf are written to the file for which aiocbp->aio_fildes is an descriptor, starting at the absolute position aiocbp->aio_offset in the file. The calling process is notified about the termination of the read request according to the aiocbp->aio_sigevent value. When aio_write returns, the return value is zero if no error occurred or returns -1.

Using List-Directed Input/Output

Function: int lio_listio (int mode, struct aiocb *const list[], int nent, struct sigevent *sig) The lio_listio function can be used to enqueue an arbitrary number of read and write requests at one time. The requests can all be meant for the same file, all for different files or every solution in between. lio_listio gets the nent requests from the array pointed to by list. The operation to be performed is determined by the aio_lio_opcode member in each element of list. The mode argument determines how lio_listio behaves after having enqueued all the requests. If mode is LIO_WAIT it waits until all requests terminated. Otherwise mode must be LIO_NOWAIT and in this case the function returns immediately after having enqueued all the requests.

Getting the Status of AIO Operations

Function: int aio_error (const struct aiocb *aiocbp) This function determines the error state of the request described by the struct aiocb variable pointed to by aiocbp. If the request has not yet terminated the value returned is always EINPROGRESS. Once the request has terminated the value aio_error returns is either 0 if the request completed successfully or it returns the value which would be stored in the errno variable if the request would have been done using read, or write. Function: ssize_t aio_return (const struct aiocb *aiocbp) This function can be used to retrieve the return status of the operation carried out by the request described in the variable pointed to by aiocbp. The return value itself is the value which would have been returned by the read or write call.

Getting the Status of AIO Operations

Function: int aio_suspend (const struct aiocb *const list[], int nent, const struct timespec *timeout) When calling this function, the calling thread is suspended until at least one of the requests pointed to by the nent elements of the array list has completed. If any of the requests has already completed at the time aio_suspend is called, the function returns immediately Function: int aio_cancel (int fildes, struct aiocb *aiocbp) The aio_cancel function can be used to cancel one or more outstanding requests. If the aiocbp parameter is NULL, the function tries to cancel all of the outstanding requests which would process the file descriptor fildes (i.e., whose aio_fildes member is fildes). If aiocbp is not NULL, aio_cancel attempts to cancel the specific request pointed to by aiocbp.

POSIX Interface" is the POSIX or "Portable Operating System

collective name of a family of related standards specified by the IEEE to define the application programming interface (API) for software compatible with variants of the Unix operating system.

Why POSIX? POSIX is an industry-standard operating system specification that enables applications to be easily ported across different hardware and operating systems implementations. POSIX.1b, Real-time extensions POSIX.1c, POSIX Threads

Semaphores
Semaphores are used to control access to shared resources by processes. There are named and unnamed semaphores. Named semaphores provide access to a resource between multiple processes. Unnamed semaphores provide multiple accesses to a resource within a single process or between related processes. Some semaphore functions are specifically designed to perform operations on named or unnamed semaphores. Semaphores are global entities and are not associated with any particular process.

Creating and Opening a Semaphore

A call to the sem_init function creates an unnamed counting semaphore with a specific value. If you specify a non-zero value for the pshared argument, the semaphore can be shared between processes. If you specify the value zero, the semaphore can be shared among threads of the same process. The sem_open function establishes a connection between a named semaphore and the calling process. Subsequent to creating a semaphore with either sem_init or sem_open, the calling process can reference the semaphore by using the semaphore descriptor address returned from the call. The semaphore is available in subsequent calls to the sem_wait, sem_trywait, and sem_post functions, which control access to the shared resource. You can also retrieve the semaphore value by calls to sem_getvalue.

Creating and Opening a Semaphore

All POSIX semaphore functions and types are prototyped or defined in semaphore.h. int sem_init(sem_t *sem, int pshared, unsigned int value); sem_init() initialises the unnamed semaphore at the address pointed to by sem. The value argument specifies the initial value for the semaphore. The pshared argument indicates whether this semaphore is to be shared between the threads of a process, or between processes. If pshared has the value 0, then the semaphore is shared between the threads of a process, and should be located at some address that is visible to all threads (e.g., a global variable, or a variable allocated dynamically on the heap). If pshared is non-zero, then the semaphore is shared between processes

Creating and Opening a Semaphore

sem_t *sem_open(const char *name, int oflag, ...); The sem_open() function creates a connection between a named semaphore and a process. One the connection has been created for the semaphore name specified by the name argument with a call to sem_open(), the process can use the address returned by the call to reference that semaphore. The oflag argument controls whether the semaphore is created or merely accessed by the call to sem_open(). It can have the following flag bits set:
O_CREAT O_EXCL

The O_CREAT flag requires two additional arguments: mode of type mode_t, and value of type unsigned int. The value argument specifies the initial value assigned to the newly created semaphore

Locking and Unlocking Semaphores

After you create the semaphore with a call to the sem_init or sem_open function, you can use the sem_wait, sem_trywait, and sem_post functions to lock and unlock the semaphore. To lock a semaphore, you can use either the sem_wait or sem_trywait function. If the semaphore value is greater than zero, the sem_wait function locks the specified semaphore. If the semaphore value is less than or equal to zero, the process is blocked (sleeps) and must wait for another process to release the semaphore and increment the semaphore value. To be certain that the process is not blocked while waiting for a semaphore to become available, use the sem_trywait function. The sem_trywait function will lock the specified semaphore if, and only if, it can do so without waiting. That is, the specified semaphore must be available at the time of the call to the sem_trywait function.

Locking and Unlocking Semaphores

int sem_wait(sem_t *sem); int sem_trywait(sem_t *sem); The sem_wait() function locks the specified semaphore by performing a semaphore lock operation on that semaphore. The sem_trywait() function locks the specified semaphore only if that semaphore is currently not locked; that is, if the semaphore value is currently positive. Otherwise, it does not lock the semaphore. int sem_post(sem_t *sem);

The sem_post() function unlocks the specified semaphore by performing a semaphore unlock operation on that semaphore.

Closing a Semaphore
When an application is finished using an unnamed semaphore, it should destroy the semaphore with a call to the sem_destroy function. For named semaphores, the application should deallocate the semaphore with a call to the sem_close function. The semaphore name is disassociated from the process. A named semaphore is removed using the sem_unlink function, which takes effect once all processes using the semaphore have deallocated the semaphore with calls to sem_close. If needed, the semaphore can be reopened for use through a call to the sem_open function. Since semaphores are persistent, the state of the semaphore is preserved, even though the semaphore is closed. When you reopen the semaphore, it will be in the state it was when it was closed, unless altered by another process.

Closing a Semaphore
int sem_destroy(sem_t *sem); sem_destroy() destroys the unnamed semaphore at the address pointed to by sem. Only a semaphore that has been initialised by sem_init(3) should be destroyed using sem_destroy(). int sem_close(sem_t *sem); The sem_close() function closes the semaphore specified by the sem argument when the calling process is finished with it. When a semaphore is closed, sem_close() frees up any system resources allocated to be used by that semaphore. These resources are then available for use by a subsequent invocation of the sem_open() function. int sem_unlink(const char *name); The sem_unlink() function removes the specified named semaphore.

EXAMPLE
#include<stdio.h> #include<semaphore.h> #include<unistd.h> #include<sys/types.h> #include<fcntl.h> #include<stdio.h> #include<semaphore.h> #include<unistd.h> #include<sys/types.h> #include<fcntl.h>

int main() { int x,z; sem_t *s; s=sem_open("/xyz",O_CREAT,0666,1); sem_wait(s); sleep(6); write(1,"hello",5); sem_post(s); sem_getvalue(s,&z); printf(value: %d\n",z); sem_close(s); sem_unlink("xyz"); }

int main() { sem_t *s; int z; s=sem_open("xyz",O_CREAT); sem_wait(s); write(1,"world",5); sem_getvalue(s,&z); printf("z5%d\n",z); sleep(5); sem_post(s); sem_close(s); }

Message queues
Message queues work by exchanging data in buffers. Any number of processes can communicate through message queues, regardless of whether they are related. If a process has adequate access permission, it can send or receive messages through the queue. If your application involves heavy message traffic, you can prioritize the order in which processes receive messages by assigning a priority to the message or controlling the priority of the receiving process. Asynchronous notification of the availability of a message on a queue allows a process to do useful work while waiting to receive a message. Realtime message passing is designed to work with shared memory in order to accommodate the needs of realtime applications with an efficient, deterministic mechanism to pass arbitrary amounts of data between cooperating processes.

Message queues
General usage for message queues is as follows:
Get a message queue descriptor with a call to the mq_open function. Send and receive messages with calls to the mq_send and mq_receive functions. Close the message queue with a call to the mq_close function. Remove the message queue with a call to the mq_unlink function

One can identify message queue attributes with a call to the mq_getattr function. One can specify whether the message operation is blocking or non-blocking by calling the mq_setattr function.

Opening a Message Queue

#include <mqueue.h>
mqd_t mq_open(const char *name, int oflag, ...); The mq_open() function shall establish the connection between a process and a message queue with a message queue descriptor. This message queue descriptor is used by other functions to refer to that message queue. The name argument points to a string naming a message queue. It is unspecified whether the name appears in the file system and is visible to other functions .

Opening a Message Queue

The value of oflag is the bitwise-inclusive OR of values from the following list. Applications shall specify exactly one of the first three values (access modes) below in the value of oflag:
O_RDONLY O_WRONLY O_RDWR

Any combination of the remaining flags may be specified in the value of oflag:
O_CREAT O_EXCL O_NONBLOCK Determines whether an mq_send() or mq_receive() waits for resources or messages that are not currently available.

Create a message queue. It requires two additional arguments: mode, which shall be of type mode_t, and attr, which shall be a pointer to an mq_attr structure.

Sending and Receiving Messages

Once a message queue is open, you can send messages to another process using the mq_send function

#include <mqueue.h> #include <time.h>

int mq_send(mqd_t mqdes, const char *msg_ptr, size_t msg_len, unsigned msg_prio); int mq_timedsend(mqd_t mqdes, const char *msg_ptr, size_t msg_len, unsigned msg_prio, const struct timespec *abs_timeout); The mq_send() function shall add the message pointed to by the argument msg_ptr to the message queue specified by mqdes. The msg_len argument specifies the length of the message, in bytes, pointed to by msg_ptr.

Sending and Receiving Messages

If the specified message queue is not full, mq_send() shall behave as if the message is inserted into the message queue at the position indicated by the msg_prio argument. A message with a larger numeric value of msg_prio shall be inserted before messages with lower values of msg_prio. The mq_timedsend() function shall add a message to the message queue specified by mqdes in the manner defined for the mq_send() function. However, if the specified message queue is full and O_NONBLOCK is not set in the message queue description associated with mqdes, the wait for sufficient room in the queue shall be terminated when the specified timeout expires. If O_NONBLOCK is set in the message queue description, this function shall be equivalent to mq_send().

Sending and Receiving Messages

Data Type: struct timespec
The struct timespec structure represents an elapsed time. It is declared in `time.h' and has the following members: long int tv_sec
This represents the number of whole seconds of elapsed time.

long int tv_nsec

This is the rest of the elapsed time (a fraction of a second), represented as the number of nanoseconds. It is always less than one billion.

Sending and Receiving Messages

ssize_t mq_receive(mqd_t mqdes, char *msg_ptr, size_t msg_len, unsigned *msg_prio);

ssize_t mq_timedreceive(mqd_t mqdes, char *msg_ptr, size_t msg_len, unsigned * msg_prio, const struct timespec * abs_timeout); The mq_receive() function shall receive the oldest of the highest priority message(s) from the message queue specified by mqdes. If the argument msg_prio is not NULL, the priority of the selected message shall be stored in the location referenced by msg_prio. The mq_timedreceive() function shall receive the oldest of the highest priority messages and if no message exists on the queue to satisfy the receive, the wait for such a message shall be terminated when the specified timeout expires.

Asynchronous Notification of Messages

A process that wants to read a message from a message queue has three options:
Set the queue to blocking mode, and wait for a message to be received by calling mq_receive Set the queue to non-blocking mode, and call mq_receive multiple times until a message is received Set the queue to non-blocking mode, and call mq_notify specifying a signal to be sent when the queue goes from empty to non-empty

The last option is a good choice for a realtime application. The mq_notify function is used to register a request for asynchronous notification by a signal when a message becomes available on a previously empty queue. The process can then do useful work until a message arrives, at which time a signal is sent according to the signal information specified in the notification argument of the mq_notify function. After notification, the process can call mq_receive to receive the message.

Asynchronous Notification of Messages

int mq_notify(mqd_t mqdes, const struct sigevent *notification); If the argument notification is not NULL, this function shall register the calling process to be notified of message arrival at an empty message queue associated with the specified message queue descriptor, mqdes. The notification specified by the notification argument shall be sent to the process when the message queue transitions from empty to non-empty. At any time, only one process may be registered for notification by a message queue. If the calling process or any other process has already registered for notification of message arrival at the specified message queue, subsequent attempts to register for that message queue shall fail.

Asynchronous Notification of Messages

If notification is NULL and the process is currently registered for notification by the specified message queue, the existing registration shall be removed. When the notification is sent to the registered process, its registration shall be removed. The message queue shall then be available for registration. If a process has registered for notification of message arrival at a message queue and some thread is blocked in mq_receive() waiting to receive a message when a message arrives at the queue, the arriving message shall satisfy the appropriate mq_receive(). The resulting behavior is as if the message queue remains empty, and no notification shall be sent.

Asynchronous Notification of Messages

struct sigevent { int sigev_notify; /*notification mechanism */ int sigev_signo; /*signal number */ union sigval sigev_value; /* signal data value */ } This structure contains three members. sigev_notify is a flag value that specifies what sort of notification should be used upon timer expirtation--signals, nothing, or something else. Currently, only two values are defined for sigev_notify:
SIGEV_SIGNAL: means to send the signal described by the remainder of the struct sigevent, and SIVEV_NONE means to send no notification at all. The sigval union is defined as:
int sival_int void* sival_ptr integer signal value pointer signal value

Using Message Queue Attributes

#include <mqueue.h> int mq_getattr(mqd_t mqdes, struct mq_attr *mqstat); The mq_getattr() function retrieves status information and attributes of the message queue and the open message queue description identified by the specified message queue descriptor, mqdes. It stores the retriieved information in in the mq_attr structure pointed to by mqstat. After returning from a mq_getattr() call, the following members of the mq_attr structure are set as described:
mq_flags mq_maxmsg mq_msgsize mq_curmsgs
Contains the number of messages currently on the message queue.

int mq_setattr(mqd_t mqdes, const struct mq_attr *mqstat, struct mq_attr *omqstat); The mq_setattr() function sets the attributes associated with the open message queue description associated with specified the message queue descriptor, mqdes.

Closing and Removing a Message Queue

int mq_close(mqd_t mqdes); int mq_unlink(const char *name);
Each process that uses a message queue should close its access to the queue by calling the mq_close function before exiting. When all processes using the queue have called this function, the software removes the queue. A process can remove a message queue by calling the mq_unlink function. However, if other processes still have the message queue open, the mq_unlink function returns immediately and destruction of the queue is postponed until all references to the queue have been closed.

EXAMPLE
int main() { int z; sigset_t set; struct sigevent event; struct mq_attr attr; unsigned int y; attr.mq_maxmsg=5; mqd_t x; attr.mq_msgsize=256; char a[256]={0}; event.sigev_notify = SIGEV_SIGNAL; event.sigev_signo=SIGINT; void handler(int q) signal(SIGINT,handler); { sigemptyset(&set); int s; printf("GOT THE SIGNAL\n"); x=mq_open("/sammq",O_CREAT|O_RDWR| O_NONBLOCK,0666,&attr); s=mq_receive(x,a,256,&y); z=mq_notify(x,&event); write(1,a,5); printf("waiting\n"); } sigsuspend(&set); mq_close(x); } #include<stdio.h> #include<fcntl.h> #include<mqueue.h> #include<signal.h>

EXAMPLE
#include<stdio.h> #include<mqueue.h> #include<fcntl.h>

int main() { mqd_t x; struct mq_attr m; unsigned int y=4; char a[256]="hello"; char b[256]="bye"; m.mq_maxmsg=20; m.mq_msgsize=256; m.mq_flags=0; x=mq_open("/xyz",O_RDWR|O_CREAT,0666,&m); mq_send(x,a,256,y); mq_send(x,b,256,8); mq_close(x); }

POSIX SHARED MEMORY

Shared memory and memory-mapped files allow processes to communicate by incorporating data directly into process address space. Shared memory and memory-mapped files follow the same general usage, as follows:
Get a file descriptor with a call to the open or shm_open function. Map the object using the file descriptor with a call to the mmap function. Unmap the object with a call to the munmap function. Close the object with a call to the close function. Remove the shared-memory object with a call to the shm_unlink function or optionally remove a memory-mapped file with a call to the unlink function.

Files, however, may need to be saved and reused each time the application is run. The unlink and shm_unlink functions remove (delete) the file and its contents. Therefore, if you need to save a shared file, close the file but do not unlink it.

Opening a Shared-Memory Object

shm_open() #include <sys/mman.h> int shm_open(const char *name, int oflag, mode_t mode); DESCRIPTION The shm_open() function establishes a connection between a shared memory object and a file descriptor. The file descriptor is used by other functions to refer to that shared memory object. The name argument points to a string naming a shared memory object. If successful, shm_open() returns a file descriptor for the shared memory object that is the lowest numbered file descriptor not currently open for that process.

Opening a Shared-Memory Object

The open file description is new, and therefore the file descriptor does not share it with any other processes. The file status flags and file access modes of the open file description are according to the value of oflag. The oflag argument is the bitwise inclusive OR of the following flags defined in the header <fcntl.h>. Applications specify exactly one of the first two values (access modes) below in the value of oflag:
O_RDONLY
Open for read access only.

O_RDWR
Open for read or write access.

Any combination of the remaining flags may be specified in the value of oflag:
O_CREAT O_EXCL

Opening Memory-Mapped Files

The open function points to the data you intend to use; the mmap function establishes how much of the data will be mapped and how it will be accessed. After opening a file, call the mmap function to map the file into application address space. When you have finished using a memory-mapped file, unmap the object by calling the munmap function, then close the object with the close function.

Mapping Memory-Mapped Files

#include <sys/mman.h> void *mmap(void *addr, size_t len, int prot, int flags, int fildes, off_t off); DESCRIPTION The mmap() function establishes a mapping between a process' address space and a file or shared memory object. The mmap() function establishes a mapping between the address space of the process at an address sa for len bytes to the memory object represented by the file descriptor fildes at offset off for len bytes. The mmap() function is supported for regular files and shared memory objects. Support for any other type of file is unspecified.

Mapping Memory-Mapped Files

The parameter prot determines whether read, write, execute, or some combination of accesses are permitted to the data being mapped. The prot should be either PROT_NONE or the bitwise inclusive OR of one or more of the other flags in the following table, defined in the header <sys/mman.h>.

Mapping Memory-Mapped Files

The parameter flags provides other information about the handling of the mapped data. The value of flags is the bitwise inclusive OR of these options, defined in <sys/mman.h>: If MAP_SHARED is specified, write references change the underlying object.
If MAP_PRIVATE is specified, modifications to the mapped data by the calling process will be visible only to the calling process and will not change the underlying object. When MAP_FIXED is set in the flags argument, the implementation is informed that the value of sa must be addr, exactly.

Controlling Memory-Mapped Files

Several functions let you manipulate and control access to memory-mapped files and shared memory. These functions include msync and mprotect. Using these functions, you can modify access protections and synchronize writing to a mapped file. int msync(void *addr, size_t len, int flags); The msync function synchronizes the caching operations of a memory-mapped file or shared-memory region. When you use the MS_SYNC flag, the msync function does not return until all write operations are complete and the integrity of the data is assured. All previous modifications to the mapped region are visible to processes using the read parameter. When you use the MS_ASYNC flag, the msync function returns immediately after all of the write operations are scheduled.

Controlling Memory-Mapped Files

int mprotect(void *addr, size_t len, int prot);
The function mprotect() changes the access protections to be that specified by prot for those whole pages containing any part of the address space of the process starting at address addr and continuing for len bytes. The parameter prot determines whether read, write, execute, or some combination of accesses are permitted to the data being mapped. The prot argument should be either PROT_NONE or the bitwise inclusive OR of one or more of PROT_READ, PROT_WRITE and PROT_EXEC.

UNmapping Memory-Mapped Files

#include <sys/mman.h> int munmap(void *addr, size_t len); The function munmap() removes any mappings for those entire pages containing any part of the address space of the process starting at addr and continuing for len bytes. The implementation will require that addr be a multiple of the page size {PAGESIZE}. If a mapping to be removed was private, any modifications made in this address range will be discarded.

Removing Shared Memory

When a process has finished using a shared-memory segment, you can remove the name from the file system namespace with a call to the shm_unlink function. #include <sys/mman.h> int shm_unlink(const char * name); The shm_unlink function unlinks the shared-memory object. Memory objects are persistent, which means the contents remain until all references have been unmapped and the shared-memory object has been unlinked with a call to the shm_unlink function. Every process using the shared memory should perform the cleanup tasks of unmapping and closing.

Pthreads Overview
What is a Thread? Technically, a thread is defined as an independent stream of instructions that can be scheduled to run as such by the operating system. To the software developer, the concept of a "procedure" that runs independently from its main program may best describe a thread. To go one step further, imagine a main program (a.out) that contains a number of procedures. Then imagine all of these procedures being able to be scheduled to run simultaneously and/or independently by the operating system. That would describe a "multi-threaded" program.

Pthreads Overview
Why threads? As process is created by the operating system, it requires a fair amount of information about program resources and program execution state, including: Process ID, process group ID, user ID, and group ID Working directory. Program instructions Registers Stack Heap File descriptors Signal actions Shared libraries Inter-process communication tools (such as message queues, pipes, semaphores, or shared memory).

Pthreads Overview
Threads use and exist within these process resources, yet are able to be scheduled by the operating system and run as independent entities. Is "lightweight" because most of the overhead has already been accomplished through the creation of its process. This independent flow of control is accomplished because a thread maintains its own:
Stack pointer Registers Scheduling properties (such as policy or priority) Set of pending and blocked signals Thread specific data.

Dies if the parent process dies. Reading and writing to the same memory locations is possible, and therefore requires explicit synchronization by the programmer.

The Pthreads API

The subroutines which comprise the Pthreads API can be informally grouped into three major classes: Thread management: The first class of functions work directly on threads - creating, detaching, joining, etc. They include functions to set/query thread attributes (joinable, scheduling etc.) Mutexes: The second class of functions deal with synchronization, called a "mutex", which is an abbreviation for "mutual exclusion". Mutex functions provide for creating, destroying, locking and unlocking mutexes. They are also supplemented by mutex attribute functions that set or modify attributes associated with mutexes. Condition variables: The third class of functions address communications between threads that share a mutex. They are based upon programmer specified conditions. This class includes functions to create, destroy, wait and signal based upon specified variable values. Naming conventions: All identifiers in the threads library begin with pthread_

Creating Threads
Initially, your main() program comprises a single, default thread. All other threads must be explicitly created by the programmer. #include <pthread.h>; int pthread_create (pthread_t *thread, const pthread_attr_t *attr, void *(*start_routine) (void *), void *arg) ; Description pthread_create creates a new thread and makes it executable. Typically, threads are first created from within main() inside a single process. Once created, threads are peers, and may create other threads. The new thread inherits its creating thread's signal mask; but any pending signal of the creating thread will be cleared for the new thread. The maximum number of threads that may be created by a process is implementation dependent.

Creating Threads
pthread_create arguments: thread: An opaque, unique identifier for the new thread returned by the subroutine. attr: An opaque attribute object that may be used to set thread attributes. You can specify a thread attributes object, or NULL for the default values. start_routine: the C routine that the thread will execute once it is created. arg: A single argument that may be passed to start_routine. It must be passed by reference as a pointer cast of type void. NULL may be used if no argument is to be passed.
Return Values If successful, the pthread_create function returns zero. Otherwise, an error number is returned to indicate the error.

Example Code - Pthread Creation and Termination

#include <pthread.h> #include <stdio.h> #define NUM_THREADS 5 void *PrintHello(void *threadid) { printf("\n%d: Hello World!\n", threadid); pthread_exit(NULL); } int main () { pthread_t threads[NUM_THREADS]; int rc, t; for(t=0; t<NUM_THREADS; t++) { printf("Creating thread %d\n", t); rc = pthread_create(&threads[t], NULL, PrintHello, (void *)t); } pthread_exit(NULL); }

Threads th, void **thread_return) Syncronization int pthread_join (pthread_t

pthread_join suspends the execution of the calling thread until the thread identified by th terminates, either by calling pthread_exit or by being canceled. If thread_return is not NULL, the return value of th is stored in the location pointed to by thread_return. The joined thread th must be in the joinable state: it must not have been detached using pthread_detach or the PTHREAD_CREATE_DETACHED attribute to pthread_create. When a joinable thread terminates, its memory resources (thread descriptor and stack) are not deallocated until another thread performs pthread_join on it. Therefore, pthread_join must be called once for each joinable thread created to avoid memory leaks. At most one thread can wait for the termination of a given thread. Calling pthread_join on a thread th on which another thread is already waiting for termination returns an error.

Thread Termination
void pthread_exit (void *retval) pthread_exit terminates the execution of the calling thread. The retval argument is the return value of the thread. It can be retrieved from another thread using pthread_join. The pthread_exit function never returns. int pthread_cancel (pthread_t thread) pthread_cancel sends a cancellation request to the thread denoted by the thread argument. If there is no such thread, pthread_cancel fails and returns ESRCH. Otherwise it returns 0.

pthread_t pthread_self (void) pthread_self returns the thread identifier for the calling thread.

Thread Attributes
By default, a thread is created with certain attributes. Some of these attributes can be changed by the programmer via the thread attribute object. pthread_attr_init and pthread_attr_destroy are used to initialize/destroy the thread attribute object. After thread creation, the thread attributes object can be reused to create another thread

Thread Attributes
Threads have a number of attributes that may be set at creation time. This is done by filling a thread attribute object attr of type pthread_attr_t, then passing it as second argument to pthread_create. The same attribute object can be used for creating several threads once created.

int pthread_attr_init (pthread_attr_t *attr)

pthread_attr_init initializes the thread attribute object attr and fills it with default values for the attributes. int pthread_attr_destroy (pthread_attr_t *attr) pthread_attr_destroy destroys the attribute object pointed to by attr releasing any resources associated with it. attr is left in undefined state, and one cannot use it again in call to any pthreads function until it has been reinitialized.

Thread Attributes
int pthread_attr_setattr (pthread_attr_t *obj, int value) Set attribute attr to value in the attribute object pointed to by obj. The following thread attributes are supported: detachstate
Choose whether the thread is created in the joinable state (value PTHREAD_CREATE_JOINABLE) or in the detached state (PTHREAD_CREATE_DETACHED). The default is PTHREAD_CREATE_JOINABLE. In the joinable state, another thread can synchronize on the thread termination and recover its termination code using pthread_join, but some of the thread resources are kept allocated after the thread terminates, and reclaimed only when another thread performs pthread_join on that thread. In the detached state, the thread resources are immediately freed when it terminates, but pthread_join cannot be used to synchronize on the thread termination. A thread created in the joinable state can later be put in the detached thread using pthread_detach.

Thread Attributes
schedpolicy
Select the scheduling policy for the thread: one of SCHED_OTHER (regular, non-realtime scheduling), SCHED_RR (realtime, round-robin) or SCHED_FIFO (realtime, first-in first-out). The default is SCHED_OTHER. Desktop OS only supports SCHED_OTHER - attempting to set one of the other policies will return an error

schedparam
Change the scheduling parameter (the scheduling priority) for the thread. The default is 0. This attribute is not significant if the scheduling policy is SCHED_OTHER; it only matters for the realtime policies SCHED_RR and SCHED_FIFO.

Thread Attributes
inheritsched
Whether the scheduling policy and scheduling parameter for the newly created thread are determined by the values of the schedpolicy and schedparam attributes (value PTHREAD_EXPLICIT_SCHED) or are inherited from the parent thread (value PTHREAD_INHERIT_SCHED). The default is PTHREAD_EXPLICIT_SCHED.

scope
Choose the scheduling contention scope for the created thread. The default is PTHREAD_SCOPE_SYSTEM, meaning that threads contend for CPU time with all processes running on the machine. In particular, thread priorities are interpreted relative to the priorities of all other processes on the machine. The other possibility, PTHREAD_SCOPE_PROCESS, means that scheduling contention occurs only between the threads of the running process: thread priorities are interpreted relative to the priorities of the other threads of the process, regardless of the priorities of other processes.

Mutexes
A mutex is a MUTual EXclusion device is useful for protecting shared data structures from concurrent modifications. A mutex has two possible states:
unlocked (not owned by any thread), and locked (owned by one thread).

A mutex can never be owned by two different threads simultaneously. A thread attempting to lock a mutex that is already locked by another thread is suspended until the owning thread unlocks the mutex first. None of the mutex functions is a cancellation point, not even pthread_mutex_lock, in spite of the fact that it can suspend a thread for arbitrary durations.

Mutexes
The PTHREAD_MUTEX_INITIALIZER macro initializes the mutex to a variable of type pthread_mutex_t.

For example: pthread_mutex_t mymutex = PTHREAD_MUTEX_INITIALIZER; The mutex is initially unlocked.

int pthread_mutex_lock (pthread_mutex_t *mutex)) pthread_mutex_lock locks the given mutex. If the mutex is currently unlocked, it becomes locked and owned by the calling thread, and pthread_mutex_lock returns immediately. If the mutex is already locked by another thread, pthread_mutex_lock suspends the calling thread until the mutex is unlocked.

int pthread_mutex_trylock (pthread_mutex_t *mutex)

pthread_mutex_trylock behaves identically to pthread_mutex_lock, except that it does not block the calling thread if the mutex is already locked by another thread. Instead, pthread_mutex_trylock returns immediately with the error code EBUSY.

Mutexes
int pthread_mutex_timedlock (pthread_mutex_t *mutex, const struct timespec *abstime) The pthread_mutex_timedlock is similar to the pthread_mutex_lock function but instead of blocking for in indefinite time if the mutex is locked by another thread, it returns when the time specified in abstime is reached. If the mutex is successfully locked, the function returns zero. If the time specified in abstime is reached without the mutex being locked, ETIMEDOUT is returned. int pthread_mutex_unlock (pthread_mutex_t *mutex) pthread_mutex_unlock unlocks the given mutex. int pthread_mutex_destroy (pthread_mutex_t *mutex) pthread_mutex_destroy destroys a mutex object, freeing the resources it might hold. If the mutex is locked by some thread, pthread_mutex_destroy returns EBUSY. Otherwise it returns 0.

Condition Variables
A condition (short for "condition variable") is a synchronization device that allows threads to suspend execution until some predicate on shared data is satisfied. The basic operations on conditions are:
signal the condition (when the predicate becomes true), and wait for the condition, suspending the thread execution until another thread signals the condition.

A condition variable must always be associated with a mutex, to avoid the race condition where a thread prepares to wait on a condition variable and another thread signals the condition just before the first thread actually waits on it. The PTHREAD_COND_INITIALIZER macro initializes the conditional variable to a variable of type pthread_cond_t. For example: pthread_cond_t cv = PTHREAD_COND_INITIALIZER;
[

Condition Variables
int pthread_cond_signal (pthread_cond_t *cond) pthread_cond_signal restarts one of the threads that are waiting on the condition variable cond. If no threads are waiting on cond, nothing happens. If several threads are waiting on cond, exactly one is restarted, but it is not specified which. This function always returns 0. int pthread_cond_broadcast (pthread_cond_t *cond) pthread_cond_broadcast restarts all the threads that are waiting on the condition variable cond. Nothing happens if no threads are waiting on cond. This function always returns 0.

Condition Variables
int pthread_cond_wait (pthread_cond_t *cond, pthread_mutex_t *mutex) pthread_cond_wait atomically unlocks the mutex (as per pthread_unlock_mutex) and waits for the condition variable cond to be signaled. The thread execution is suspended and does not consume any CPU time until the condition variable is signaled. The mutex must be locked by the calling thread on entrance to pthread_cond_wait. Before returning to the calling thread, pthread_cond_wait re-acquires mutex (as per pthread_lock_mutex). Thus, if all threads always acquire the mutex before signaling the condition, this guarantees that the condition cannot be signaled between the time a thread locks the mutex and the time it waits on the condition variable. This function always returns 0.

Condition Variables
int pthread_cond_timedwait (pthread_cond_t *cond, pthread_mutex_t *mutex, const struct timespec *abstime) pthread_cond_timedwait atomically unlocks mutex and waits on cond, as pthread_cond_wait does, but it also bounds the duration of the wait. If cond has not been signaled before time abstime, the mutex mutex is re-acquired and pthread_cond_timedwait returns the error code ETIMEDOUT. int pthread_cond_destroy (pthread_cond_t *cond) pthread_cond_destroy destroys the condition variable cond, freeing the resources it might hold. If any threads are waiting on the condition variable, pthread_cond_destroy leaves cond untouched and returns EBUSY. Otherwise it returns 0, and cond must not be used again until it is reinitialized.
[

Threads and Signal Handling

int pthread_sigmask (int how, const sigset_t *newmask, sigset_t *oldmask) pthread_sigmask changes the signal mask for the calling thread as described by the how and newmask arguments. If oldmask is not NULL, the previous signal mask is stored in the location pointed to by oldmask. The meaning of the how and newmask arguments is the same as for sigprocmask.
If how is SIG_SETMASK, the signal mask is set to newmask. If how is SIG_BLOCK, the signals specified to newmask are added to the current signal mask. If how is SIG_UNBLOCK, the signals specified to newmask are removed from the current signal mask.

Signal masks are set on per-thread basis, but signal actions and signal handlers, as set with sigaction, are shared between all threads. int pthread_kill (pthread_t thread, int signo) pthread_kill sends signal number signo to the thread thread. pthread_kill returns 0 on success.

Threads and Signal Handling

int sigwait (const sigset_t *set, int *sig) sigwait suspends the calling thread until one of the signals in set is delivered to the calling thread. It then stores the number of the signal received in the location pointed to by sig and returns. The signals in set must be blocked and not ignored on entrance to sigwait. If the delivered signal has a signal handler function attached, that function is not called. sigwait is a cancellation point. It always returns 0. For sigwait to work reliably, the signals being waited for must be blocked in all threads, not only in the calling thread, since otherwise the POSIX semantics for signal delivery do not guarantee that it's the thread doing the sigwait that will receive the signal. The best way to achieve this is block those signals before any threads are created, and never unblock them in the program other than by calling sigwait.

The Make command

As you begin to write larger programs, you will notice that recompiling larger programs takes much longer than re-compiling short programs. Moreover, you notice that you usually only work on a small section of the program (such as a single function that you are debugging), and much of the rest of the program remains unchanged. The make program aids you in developing your large programs by keeping track of which portions of the entire program have been changed, compiling only those parts of the program which have changed since the last compile. make is a powerful tool that is most commonly used to automatically compile programs, particularly those whose source is contained in multiple files make accomplishes this by reading a description file that you write. The description file describes the program's dependencies on other software modules.

A simple compilation
Compiling a small C program requires at least a single .c file, with .h files as appropriate. There are 3 steps to obtain the final executable program, as shown:
Compiler stage: Assembler stage: Linker stage:

Compiling with several files

When your program becomes very large, it makes sense to divide your source code into separate easily-manageable .c files. The figure above demonstrates the compiling of a program made up of two .c files and a single common.h file. The command is as follows: cc green.c blue.c where both .c files are given to the compiler. The two .o files are linked together at the Linker stage to create one executable program, a.out.

Separate compilation
The steps taken in creating the executable program can be divided up in to two compiler/assembler steps circled in red, and one final linker step circled in yellow. The two .o files may be created separately, but both are required at the last step to create the executable program. The three different tasks required to produce the executable program are as follows: Compile green.o: cc -c green.c Compile blue.o: cc -c blue.c Link the parts together: cc green.o blue.o

Splitting your C program

When you separate your C program into many files, keep these points in mind: Be sure no two files have functions with the same name in it. The compiler will get confused. Similarly, if you use global variables in your program, be sure no two files define the same global variables. If you use global variables, be sure only one of the files defines them, and declare them in your .h as follows: extern int globalvar; To use functions from another file, make a .h file with the function prototypes, and use #include to include those .h files within your .c files. At least one of the files must have a main() function. Note: When you define a variable, it looks like this: int globalvar;. When you declare a variable, it looks like this: extern int globalvar;. The main difference is that a variable definition creates the variable, while a declaration indicates that the variable is defined elsewhere. A definition implies a declaration.

Dependencies
The principle by which make operates was described to you in the last section. It creates programs according to the file dependencies. For example, we now know that in order to create an object file, program.o, we require at least the file program.c. (There may be other dependencies, such as a .h file.) This section involves drawing what are called "dependency graphs", which are very similar to the diagrams given in the previous section. As you become proficient using make, you probably will not need to draw these diagrams, but it is important to get a feel for what you are doing.

Dependency graphs
The lines which radiate downwards from a file are the other files which it depends on. For example, to create main.o, the three files data.h, io.h, and main.c are needed.

How dependency works

Suppose that you have gone through the process of compiling the program, and while you are testing the program, you realize that one function in io.c has a bug in it. You edit io.c to fix the bug. By going up the graph, you notice that io.o needs to be updated because io.c has changed. Similarly, because io.o has changed, project1 needs to be updated as well.

How make works

Make looks for a file called "makefile" or "Makefile", within the makefile are variables and things called dependencies. This file determines the relationships between the source, object and executable files. Format of Makefiles Dependencies dependecy1: dependencyA dependencyB ... dependencyN command for dependency1 The dependency line is made of two parts. The first part (before the colon) are target files and the second part (after the colon) are called source files. It is called a dependency line because the first part depends on the second part. Multiple target files must be separated by a space. Multiple source files must also be separated by a space.

Translating the dependency graph

Each dependency shown in the graph is circled with a correspondingly in the Makefile. Comments can be placed in a Makefile by placing a pound s ign (#) in front of it.

How does make do it?

Make checks the modification times of the files, and whenever a file becomes "newer" than something that depends on it, (in other words, modified) it runs the compiler accordingly. For example, the previous page explained io.c was changed. If you edit io.c, it becomes "newer" than io.o, meaning that make must run cc -c io.c to create a new io.o, then run cc data.o main.o io.o -o project1 for project1.

Using the Makefile with make

Once you have created your Makefile and your corresponding source files, you are ready to use make. If you have named your Makefile either Makefile or makefile, make will recognize it. If you do not wish to call your Makefile one of these names, you can use make -f mymakefile. The order in which dependencies are listed is important. If you simply type make and then return, make will attempt to create or update the first dependency listed. You can also specify one of the other targets listed in the Makefile, and only that target (and its corresponding source files) would be made.

Macros in make
Macros in make work similarly to macros used in C programming. The make program allows you to use macros, which are similar to variables, to store names of files. The format is as follows: OBJECTS = data.o io.o main.o Whenever you want to have make expand these macros out when it runs, type the following corresponding string $(OBJECTS). Here is our sample Makefile again, using a macro. OBJECTS = data.o main.o io.o project1: $(OBJECTS) cc $(OBJECTS) -o project1 data.o: data.c data.h cc -c data.c main.o: data.h io.h main.c cc -c main.c io.o: io.h io.c cc -c io.c

Special macros
In addition to those macros which you can create yourself, there are a few macros which are used internally by the make program. Here are some of those, listed below: CC
Contains the current C compiler. Defaults to cc.

CFLAGS
Special options which are added to the built-in C rule.

$@
Full name of the current target.

$?
A list of files for current dependency which are out-of-date.

$<
The source file of the current (single) dependency.

Predefined rules
By itself, make knows already that in order to create a .o file, it must use cc -c on the corresponding .c file. These rules are built into make, and you can take advantage of this to shorten your Makefile. If you just indicate just the .h files in the dependency line of the Makefile that the current target is dependent on, make will know that the corresponding .c file is already required. You don't even need to include the command for the compiler. This reduces our Makefile further, as shown: OBJECTS = data.o main.o io.o project1: $(OBJECTS) cc $(OBJECTS) -o project1 data.o: data.h main.o: data.h io.h io.o: io.h

Special dependencies
Usually, make uses the same command to create or update a target, regardless of which file changes. Some other files, such as libraries allow users to replace a portion of its code. For this kind of different behavior, make allows a special form of the dependency, where the action specified can differ, depending on which file has changed. Here is an example for this rule: target :: source1
command1

target :: source2
command2

As we have described, if source1 changes, target is created or updated using command1; command2 is used if source2 is modified instead.