Processes and Threads

CS 202: Advanced Operating Systems

 OS Abstractions

Applications

Process File system Virtual memory

Operating System

CPU Hardware
Disk RAM

Today, we start discussing the first abstraction that enables us to

virtualize (i.e., share) the CPU – processes!

2
 Program
• Program is a file describing a computation
– Executable code (machine instructions)
– Data (info manipulated by the instructions

3
 Process
• The process is the OS abstraction for execution
– It is the unit of execution
– It is the unit of scheduling

• A process is a program in execution

– Programs are static entities with the potential for execution
– Process is the dynamic/active entity of a program
• Includes dynamic state
• As the representative of the running program, it is the “owner” of other
resources (memory, files, sockets, …)

4
 Process = Program ???
• A program is passive
– Code + data
• A process is alive:
– Code + data + stack +registers + PC…
• Same program can be run simultaneously
– 2 processes
• Why processes?

5
 Process Components
• A process contains all the state for a program in execution
– An address space containing
• Static memory:
– The code and input data for the executing program
• Dynamic memory:
– The memory allocated by the executing program
– An execution stack encapsulating the state of procedure calls
– Control registers such as the program counter (PC)
– A set of general-purpose registers with current values
– A set of operating system resources
• Open files, network connections, etc.

• A process is named using its process ID (PID)

6
Address Space (memory abstraction)
0xFFFFFFFF
Stack
SP

Dynamic

Address Heap
Space (Dynamic Memory Alloc)

Static Data
(Data Segment)
Static
Code PC
(Text Segment)
0x00000000

7
 Process Execution State
• A process is born, executes for a while, and then dies

• The process execution state that indicates what it is

currently doing
– Running: Executing instructions on the CPU
• It is the process that has control of the CPU
• How many processes can be in the running state simultaneously?
– Ready: Waiting to be assigned to the CPU
• Ready to execute, but another process is executing on the CPU
– Waiting: Waiting for an event, e.g., I/O completion
• It cannot make progress until event is signaled (disk completes)

8
 Execution state (cont’d)
• As a process executes, it moves from state to state
– Unix: “ps -x”: STAT column indicates execution state
– What state do you think a process is in most of the time?
– How many processes can a system support?

9
 Execution State Graph

Create
Process

New Ready
I/O Done

Unschedule Schedule
Process Process Waiting

I/O, Page
Terminated Running Fault, etc.

Process
Exit

10
 How does the OS support this model?
We will discuss three issues:
1. How does the OS represent a process in the kernel?
– The OS data structure representing each process is called
the Process Control Block (PCB)

2. How do we pause and restart processes?

– We must be able to save and restore the full machine state

3. How do we keep track of all the processes in the system?

– A lot of queues!

11
 PCB Data Structure
• PCB is also where OS keeps all of a process’ hardware execution
state when the process is not running
• Process ID (PID)
• Execution state
• Hardware state: PC, SP, regs
• Location in memory
• Scheduling info
• User info
• Pointers for state queues
• Etc.
• This state is everything that is needed to restore the hardware to
the same configuration it was in when the process was switched
out of the hardware

12
xv6/proc.h: struct proc

13
 How to pause/resume a process?
• When a process is running, its dynamic state is in memory and
some hardware registers
– Hardware registers include program counter, stack pointer, control registers, data
registers, …
– To stop and restart a process, we need to completely restore this state

• When the OS stops running a process, it saves the current values

of the registers (usually in PCB)
• When the OS resumes a process, it loads the hardware registers
from the stored values in PCB

• Changing CPU hardware state from one process to another is

called a context switch
– This can happen 100s or 1000s of times a second!
14
 xv6: context switching
• Switching from one user process to another (shell to cat)
– First moves to the kernel space, then calls swtch() for context switching

• In xv6, each CPU has a scheduler thread (scheduler() in proc.c)

15
xv6: PCB and context

16
 xv6: swtch()
• Kernel/swtch.S

17
How does the OS track process states?
• The OS maintains a collection of queues that represent the state
of all processes in the system

• Typically, the OS maintains at least one queue for each state

– Ready, waiting, etc.

• Each process (namely PCB) is queued on a state queue according

to its current state

• As a process changes state, its PCB is unlinked from one queue

and linked into another

18
 State Queues
Ready Queue Firefox PCB X Server PCB Outlook PCB

Disk I/O Queue Emacs PCB ls PCB

Console Queue
Sleep Queue There may be many wait queues,
. one for each type of wait (disk,
. console, timer, network, etc.)
.

19
 Process System Call APIs
• Process creation: how to create a new process?

• Process termination: how to terminate and clean up a process

• Coordination between processes

– wait, waitpid, signal, inter-process communication, synchronization

• Other
– e.g., set quotas or priorities, examine usage, …

20
 Process Creation
• A process is created by another process
– Parent is creator, child is created (Unix: ps “PPID” field)
– What creates the first process (Unix: init (PID 1))?

• After creating a child, the parent may either wait for it to finish
its task or continue in parallel (or both)

21
 Process Creation: Windows
• The system call on Windows for creating a process is called,
surprisingly enough, CreateProcess:
BOOL CreateProcess(char *prog, char *args) (simplified)

• CreateProcess
– Creates and initializes a new PCB
– Creates and initializes a new address space
– Loads the program specified by “prog” into the address space
– Copies “args” into memory allocated in address space
– Initializes the saved hardware context to start execution at main (or
wherever specified in the file)
– Places the PCB on the ready queue

22
 Process Creation: Unix
• In Unix, processes are created using fork(): int fork()

• fork()
– Creates and initializes a new PCB
– Creates a new address space
– Initializes the address space with a copy of the entire contents of the
address space of the parent
• Child has the same memory and registers: same SP & PC
– Initializes the kernel resources to point to the resources used by parent
(e.g., open files)
– Places the PCB on the ready queue

• fork() returns twice

– Returns the child’s PID to the parent, “0” to the child
– Child and parent resume execution from this point: they have same PC
23
 fork()
int main(int argc, char *argv[])
{
char *name = argv[0];
int child_pid = fork();
if (child_pid == 0) {
printf(“Child of %s is %d\n”, name, getpid());
return 0;
} else {
printf(“My child is %d\n”, child_pid);
return 0;
}
}
What does this program print?

24
 Example Output
[well ~]$ gcc t.c
[well ~]$ ./a.out
My child is 486
Child of a.out is 486

25
 Duplicating Address Spaces

child_pid = 486 child_pid = 0

PC child_pid = fork(); child_pid = fork(); PC

if (child_pid == 0) { if (child_pid == 0) {
printf(“child”); printf(“child”);
} else { } else {
printf(“parent”); printf(“parent”);
} }

Parent Child

26
 Divergence

child_pid = 486 child_pid = 0

child_pid = fork(); child_pid = fork();

if (child_pid == 0) { if (child_pid == 0) {
printf(“child”); printf(“child”); PC
} else { } else {

PC printf(“parent”); printf(“parent”);
} }

Parent Child

27
 Example Continued
[well ~]$ gcc t.c
[well ~]$ ./a.out
My child is 486
Child of a.out is 486
[well ~]$ ./a.out
Child of a.out is 498
My child is 498

Why is the output in a different order?

28
 Why fork()?
• Simple approach to enable concurrent execution
• Useful when the child…
– Is cooperating with the parent
– Relies upon the parent’s data to accomplish its task

• Example: Web server

while (1) {
int sock = accept();
if ((child_pid = fork()) == 0) {
Handle client request
} else {
Close socket
}
}
29
 Process Creation: Unix (2)
• Wait a second. How do we actually start a new program?
int exec(char *prog, char *argv[])

• exec()
– Stops the current process
– Loads the program “prog” into the process’ address space
– Initializes hardware context and args for the new program
– Places the PCB onto the ready queue
– Note: It does not create a new process

• What does it mean for exec to return?

30
 Process Termination
• All good processes must come to an end. But how?
– Unix: exit(int status), Windows: ExitProcess(int status)

• Essentially, free resources and terminate

– Terminate all threads (next lecture)
– Close open files, network connections
– Allocated memory (and VM pages out on disk)
– Remove PCB from kernel data structures, delete

• Note that a process does not need to clean up itself

– OS will handle this on its behalf

31
 wait() a second…
• Often it is convenient to pause until a child process has finished
– Think of executing commands in a shell

• Use wait() (WaitForSingleObject)

– Suspends the current process until a child process ends
– Returns the process ID of the terminated child
– waitpid() suspends until the specified child process ends

• Unix: Every process must be reaped by a parent

– Basically, reading its child’s exit status using wait()
– What happens if a parent process exits before a child?
• Orphan process; adopted by init (PID 1)
– “zombie” process: undead; a process that has died but has not yet been
reaped by the parent
32
 Unix Shells
while (1) {
char *cmd = read_command();
int child_pid = fork();
if (child_pid == 0) {
Manipulate STDIN/OUT/ERR file descriptors for pipes, redirection,
etc.
Child
exec(cmd); (new prog)
panic(“exec failed”);
} else {
if (!(run_in_background))
waitpid(child_pid); Parent
} (shell)
}

33
 Some issues with processes
• Creating a new process is costly because of new address space
and data structures that must be allocated and initialized
– Recall struct proc in xv6

• Communicating between processes is costly because most

communication goes through the OS
– Each process has a separate address space
– Inter Process Communication (IPC) – we will discuss later
– Overhead of system calls and copying data

34
 Process-associated overheads
• Context switch: high Process 1 Process n
CPU CPU
– CPU state: low State State
– Memory / IO state: high ….
Mem. Mem.
State State
• Process creation: high
I/O I/O
• Protection State State

– CPU: yes
– Memory / IO: yes
• Sharing overhead: high OS
scheduler
– At least one context switch

35
 Parallel Programs
• Recall our Web server example that forks off copies of itself to
handle multiple simultaneous requests

• To execute these programs, we need to

• Create several processes that execute in parallel
• Cause each to map to the same address space to share data
• They are all part of the same computation
• Have the OS schedule these processes in parallel

• This could be very inefficient

• Space: PCB, page tables, etc.
• Time: create data structures, fork and copy address space, etc.

36
 Rethinking Processes
• What is similar in these cooperating processes?
– They all share the same code and data (address space)
– They all share the same privileges
– They all share the same resources (files, sockets, etc.)

• Why don’t they share resources?

– While each keeps its own execution state: PC, SP, and registers

• Key idea: Separate resources from execution state

37
 Threads
• Separate execution and resource container roles
– The thread defines a sequential execution stream within a process
(PC, SP, registers)
– The process defines the address space, resources, and general process
attributes (everything but threads)

• Threads become the

unit of scheduling
– Processes are now the
containers in which
threads execute
– Processes become static,
threads are dynamic
schedulable entities
38
 Thread-associated overheads
Process 1 Process 1
• Context switch: low Threads

– Only CPU state CPU

State
– No more memory / IO state
Mem. Mem.
State State
• Thread creation: low
I/O CPU CPU I/O
• Protection State State State State

– CPU: yes
– Memory / IO: no
OS
• Sharing overhead: low scheduler
– Due to low thread switch

39
 Thread-associated overheads
Process 1 Process 1
• Context switch: low Threads

– Only CPU state CPU

State
– No more memory / IO state
Mem. Mem.
State State
• Thread creation: low
I/O CPU CPU I/O
• Protection State State State State

– CPU: yes
– Memory / IO: no
OS
• Sharing overhead: low scheduler
– Due to low thread switch

Context switching mainly depends on the process or

thread’s hunger for memory (i.e., working set size)

40
Recap: Process Address Space
0xFFFFFFFF
Stack
SP

Address Heap
Space (Dynamic Memory Alloc)

Static Data
(Data Segment)

Code PC
(Text Segment)
0x00000000

41
 Threads in a Process

Stack (T1) Thread 1

Thread 2 Stack (T2)

Stack (T3) Thread 3

Heap

Static Data

PC (T3)
PC (T2) Code
PC (T1)

42
 Threads: Concurrent Servers
• Using fork() to create new processes to handle requests in
parallel is overkill for such a simple task

• Recall our forking Web server:

while (1) {
int sock = accept();
if ((child_pid = fork()) == 0) {
Handle client request
Close socket and exit
} else {
Close socket
}
}

43
 Threads: Concurrent Servers
• Instead, we can create a new thread for each request

web_server() {
while (1) {
int sock = accept();
thread_fork(handle_request, sock);
}
}

handle_request(int sock) {
Process request
close(sock);
}

44
 Sample Thread Interface
• thread_fork(procedure_t)
– Create a new thread of control
– Also thread_create(), thread_setstate()
• thread_stop()
– Stop the calling thread; also thread_block
• thread_start(thread_t)
– Start the given thread
• thread_yield()
– Voluntarily give up the processor
• thread_exit()
– Terminate the calling thread; also thread_destroy

45
 Implementing threads
• Kernel-Level Threads
– All thread operations are implemented in the kernel
– The OS schedules all of the threads in the system
• Requires a full thread control block (TCB) for each thread
– Don’t have to separate from processes

• OS-managed threads are called kernel-level threads or

lightweight processes
– Windows: threads
– Solaris: lightweight processes (LWP)
– POSIX Threads (pthreads): PTHREAD_SCOPE_SYSTEM

46
 Alternative: User-Level Threads
• Implement and manage threads entirely at user level
– Kernel knows nothing about user-level threads

• ULTs are small and fast

– A thread is simply represented by a PC, registers, stack, and small thread
control block (TCB)
– Creating a new thread, switching between threads, and synchronizing
threads are done via user-level procedure call
• No kernel involvement; No mode switching
– User-level thread operations can be 100x faster than kernel threads
– ULTs are required to use non-blocking system calls. (Why?)
– pthreads: PTHREAD_SCOPE_PROCESS

47
KLT and ULT combined

48
 Summary KLT vs. ULT
• Kernel-level threads
– Integrated with OS (informed scheduling)
– Slow to create, manipulate, synchronize

• User-level threads
– Fast to create, manipulate, synchronize
– Not integrated with OS (uninformed scheduling)

• Understanding the differences between kernel and user-level

threads is important
– For programming (correctness, performance)
– For fundamental systems knowledge

49
 Simultaneous multithreading &
hyperthreading
• Traditional multithreading can start execution of instructions from only
a single thread at a given cycle
– Low utilization if not enough instructions from a thread to dispatch in one
cycle
– Bad for machines with multiple execution units (i.e., superscalar architecture)
• Idea: dispatch instructions from multiple threads in the same cycle to
keep multiple execution units utilized
– Hirata et al., “An elementary processor architecture with simultaneous instruction issuing from multiple
threads,” ISCA 1992
– Tullsen et al., “Simultaneous Multithreading: maximizing on-chip parallelism,” ISCA 1995

50
 Multithreading
Time

• Intra-thread dependencies
• Single thread performance suffers
51
 SMT
Time

• Utilize functional units with independent operations

from multiple threads

52
SMT scalability

53
 Commercial SMT implementations
• Intel Pentium 4 (Hyperthreading)
• IBM POWER5
• Intel Nehalem

54
 Intel Pentium 4 HT

• Marr et al., “Hyper-threading technology architecture

and microarchitecture,” Intel technology journal, 2002
55
Summary: why processes & threads?
• Goals
– Multiprogramming: run multiple applications concurrently
– Protection: don’t want a bad application to crash system
• Solution
– Process: given process/program illusion that it owns
hardware resources
• Challenges
– Process creation & switching is expensive
– Concurrency within a same application
• Solution
– Thread: decouple allocation and execution
– Multiple threads within same process with less context
switch overhead (only CPU state) 56