Slides for Chapter 7:

Operating System support

From Coulouris, Dollimore, Kindberg and Blair

Distributed Systems:
Concepts and Design
Edition 5, © Addison-Wesley 2012
 Outline
7.1 Introduction
7.2 The operating system layer
7.3 Protection
7.4 Processes and threads
7.5 Communication and invocation
7.6 Operating system architecture
7.7 Virtualization at the operating system level

2
 Communication and Invocation
• An invocation
– A construct (e.g., RMI, RPC or event notification) whose purpose is
to bring about an operation on a resource in a different address
space

• In a DS, an invocation is supported by the OS’s underlying

communication mechanisms.
• Ask the following questions about the OS:
1) Which communication primitives does it supply?
2) Which protocols does it support and how open is the
communication implementation?
3) What steps are taken to make communication as efficient as
possible?
4) What support is provided for high-latency and disconnected
operations? 3
 Communication and Invocation (cont.)
• Communication Primitives:
– Examples of kernel-provided communication primitives
• Amoeba (doOperation, getRequest, getReply)
• UNIX (write, read)
– In practice, middleware, and not the kernel, provides most high-level
communication facilities in the DS. Why?
– Kernel-provided primitives vs middleware-provided primitives?
Kernel-provided primitives middleware-provided primitives

Pros?

Cons?

4
Communication and Invocation (cont.)
• Protocols and Openness
– Goal: Standardized protocols that enable interworking
between middleware implementations on different
platforms
– An open design allows the protocol
• to be expandable (when new features are needed), and
• to be integrable with other protocols

– Protocols are arranged in a stack of layers.

– A protocol provides standardized interface to support
protocols at the layer above it.
• Many OSs allow new layers to be integrated statically.
• Dynamic Protocol Composition – Protocol stack can be composed
on the fly to meet the requirements of a particular application.
5
 Communication and Invocation (cont.)
• Invocation Performance:
– Examples of invocation mechanisms
• Calling a conventional procedure
• Invoking a conventional method
• Making a system call
• Sending a message
• Remote procedure calling
• Remote method invocation
– When network communication are involved
• Total invocation cost =
software overheads on the caller node
+ network overheads to send the message
+ software overheads on the callee
+ [network overheads to send the response back to the caller]6
 Communication and Invocation (cont.)
• Invocation Performance (cont.)
• Total invocation cost =
software overheads on the caller node
+ network overheads to send the message
+ software overheads on the callee
+ [network overheads to send the response back to the caller]
– For the case of a LAN or intranet, software overheads often
predominate over network overheads in invocation times
– c.f., On the Internet, network latencies are highly variable and
relatively high on average; throughput may be relatively low, and
server load often predominates over per-request processing costs.
– [Bridges et al. 2007]
- UDP message round-trips taking average times of ~400 milliseconds over
the Internet between two computers connected across US geographical
regions
- ~0.1 milliseconds when identical computers were connected over a single
7
Ethernet
 Communication and Invocation
(cont.)

• Invocation Performance (cont.)

– Major factors impacting invocation costs
• On the nodes (client/server, sender/receiver):
– Multiprocessing: Is there domain transition (or context
switching) between processes?
– Multithreading: Is there thread scheduling and switching?
• Between the nodes:
– Communication: Is there communication over the network?

8
Figure 7.11 Invocations between address spaces

9
 Communication and Invocation (cont.)
• Invocation Performance (cont.)
– Invocation over the network
• A null RPC or null RMI (no params, execute a null procedure,
no return): carry system but not user data.
• A null RPC (over a LAN) takes ~0.1 millisecond [Bridges et al.
2007]
c.f., A null local call takes a fraction of a microsecond (~0.0001-
0.000001 millisecond)
– For a null RPC, about 100 bytes are passed over the network, so with
a raw 100 Mbits /sec, the network transfer time is 0.01 millisecond.
Q: What may have cost the other 0.09 millisecond?
• Null invocations (null RPCs or null RMIs) are important since
they measure a fixed overhead (latency) w/o user data.
• Of course the overall overhead increases when data is
involved.
10
 Communication and Invocation
(cont.)

• Actual remote invocation cost is more than just

network latency (as incurred in null RPCs/RMIs).
• Other factors?
– Processing cost on the caller and the callee nodes
• Packetization (on the caller node): Fragmentation of the
user data into two or more packets
• Assembly of packets (on the callee node)
– Data transmission cost
• Affected by throughput (e.g., the average number of
transmitted packets per second) and bandwidth (i.e., rate
of data transfer)
11
 Communication and Invocation (cont.)
• RPC Steps: Figure 5.10 + Figure 7.5, 7.6
– Client stub marshals the call arguments into a message and sends the
request message and unmarshals the reply.
– Server: a worker thread receives incoming request which in turn calls
the appropriate server stub.
– Server stub unmarshals, calls the procedure, and marshals and sends
back.
• So besides network transmission, what else are involved in
remote invocation delay?
– Marshalling
– Data copying (across user-kernel boundary, across protocol layers,
between network interface and kernel buffer)
– Packet Initialization
– Thread Scheduling and context switching
– Waiting for acknowledgements 12
 Figure 7.12
RPC delay against parameter size

13
 Communication and Invocation (cont.)
• Other performance factors:
– Use of shared memory (to reduce memory copying overheads)
• Shared regions may be used for rapid communication between a user
process and the kernel, or between user processes.
– Choice of protocol (TCP/UDP)
• When implementing request-reply interactions on top of a protocol
such as TCP, TCP’s buffering behavior can hinder good performance,
and its connection overheads put it at a disadvantage compared with
UDP.
• UNLESS a TCP connection is used for multiple requests.
– Buffering of data before being dispatched over the network
• OS’s default buffering is to collect several small messages (in the
buffer) and then send them together in a single packet.
• This behavior may cause unnecessary delays.
14
 Communication and Invocation
(cont.)
• Invocation within a computer
• [Bershad et al. 1990] Most cross-address-space invocation took place
within a computer and not, as might be expected in a client-server
installation, between computers.
– Data are cached in a local server, and only retrieved from the remote server
when necessary.

• Lightweight RPC (LRPC)

• Implemented as if it is RPC across computers, but the underlying
message passing (between the client and the server) is between two
local processes
• The underlying message passing should be optimized to reduce the
overall invocation cost.
– Data copying
– Thread scheduling

15
 Figure 7.13
A lightweight remote procedure call

• An A (Argument) stack is used to pass arguments and get return results

between a client and the server processes.

16
 LRPC vs RPC
LRPC RPC
Processing of Arguments are copied Arguments are copied four times:
arguments once: when they are • from the client stub’s stack
marshalled onto the A onto a message,
stack. • from the message to a kernel
buffer,
• from the kernel buffer to a
server message, and
• from the message to the
server stub’s stack.

Processing The server exports a A worker thread is created by the

on the server set of procedures that server to handle each of the
it is prepared to be client’s request.
called by the clients.

17
 Communication and Invocation (cont.)
- Asynchronous Operation
• In the Internet environment, the effects of relatively high
latencies, low throughput and high server loads may
outweigh any benefits that the OS can provide to reduce
invocation overheads.
– Network may not be available all the time.
• disconnected and then reconnected
– Servers may be overloaded or have crashed.

• Asynchronous operation can be used to defeat high

latency and low throughput.
– Concurrent invocations: Making invocations concurrent
– Asynchronous invocations: Make non-blocking calls.
18
 Figure 7.14
Times for serialized vs concurrent invocations

19
 Communication and Invocation
- Asynchronous Operation (cont.)

• An asynchronous invocation is one that is performed

asynchronously with respect to the caller.
– The caller makes a non-blocking call.
• Persistent asynchronous invocation
– Rationale: A conventional invocation mechanism (synchronous
or asynchronous) is designed to fail after a given number of
timeouts have occurred.
• These short-term timeouts are often not appropriate where
disconnections or very high latencies occur.
– A persistent asynchronous invocation tries indefinitely to
perform the invocation, until it is known to have succeeded or
failed, or until the application cancels the invocation.
e.g., Queued RPC (QRPC)

20
 Communication and Invocation
- Asynchronous Operation (cont.)

• Queued RPC (QRPC)

– Outgoing invocation requests are queued.
– The requests are then dispatched to servers when there is
a network connection.
– Invocation results (i.e., responses from servers) are queued
in the client’s invocation ‘mailbox’ until the client
reconnects and collects them.

Q: Trade-offs between synchronous and asynchronous

operations?

21
 Operating System Architecture
• Examine the architecture of a kernel suitable for a distributed
system
• The requirements of openness in kernel architecture
• An open distribute system should make it possible to:
– Run only that system software at each computer that is necessary for
it to carry out the particular role.
• Loading redundant or unneeded modules wastes memory resources.
• Ideally, the kernel would provide only the most basic mechanisms upon which
the general resource management tasks at a node are carried out.
– Allow the software implementing a service to be changed
independently of other facilities.
– Allow for alternatives of the provided services.
– Introduce new services without harming the existing ones.
• Server modules would be dynamically loaded as required, to implement the
required resource management policies for the currently running applications.
22
Operating System Architecture (cont.)
• Two kernel design approaches: Monolithic kernels vs.
Microkernels
– They differ in what functionality belongs to the kernel and what is
left for the server processes that can be dynamically loaded to run
on top of the kernel.
– Monolithic design is ‘massive’.
• performs all the basic OS functions and takes up in the order of megabytes
of code and data
• undifferentiated – It is coded in a non-modular way.
• intractable – altering individual software components is difficult.
– Microkernel design has a smaller kernel.
• The kernel provides only the most basic abstractions – address spaces,
threads, and local IPC.
• All other system services are provided by servers that are dynamically
loaded when needed. 23
 Figure 7.15
Monolithic kernel and microkernel

24
 Figure 7.16 The role of the microkernel

• If performance is the main goal, then middleware may use the facilities of
the microkernel directly.
• If portability is the main goal, then middleware uses a language runtime
support subsystem, or a higher-level operating system interface provided
by an operating system emulation subsystem.
25
 Operating System Architecture (cont.)
• Comparison:
– Microkernel OSs are extensible.
• Modularity is enforced behind memory protection boundaries.
• Small kernels are relatively free of bugs. (software maintenance)
– Monolithic Kernels
• Hard to maintain
• Efficiency in invoking operations.
– System calls can be expensive but still cheaper than invocations to a separate
user-level address space.
• Lacks structure but that can be avoided using software development
techniques such as layering.
– Hybrid Approaches:
• Mach and Chorus are both originally microkernels.
• Because of performance problems, they eventually changed to allow
servers to be loaded dynamically either into the kernel address space or
the user address space.
26
 Virtualization at the OS level
• We have seen virtualization already in overlay networks.
– a virtual or logical network that is created on top of an existing
physical network
• System virtualization: virtualization of an OS
– Examples : VirtualBox, Parallels, VMWare, …
– Goal: provide multiple virtual machines (hardware images) over the
underlying physical machine. Each VM may be running a different OS
instance.
– Modern computer have the ability to do this without a performance
hit.
• System virtualization enables the processor and other resources
to be shared between multiple tasks running on behalf of one
or several users.
27
 Virtualization
• Motivation through different uses of virtualization:
– Resource sharing
– Flexible server allocation: Unlike processes, VMs can be migrated
quite simply to other physical machines, adding flexibility in
managing the server infrastructure.
– It is very relevant to cloud computing.
• Infrastructure as a service, Platform as a service, and Software as a
service
• In particular, VMs can be used to enable infrastructure as a service.
– Dynamic resource allocation: Some distribute applications need
to create and destroy VMs with very little overhead.
– Convenient access to different OS environments on a single
desktop computer.
28
 Virtualization (cont.)
• Implementation:
– Virtualization is implemented as a thin layer of software
on top of the underlying physical architecture (known as
“virtual machine monitor” or “hypervisor”).
• The monitor provides interface based closely on the
underlying physical architecture.
• “Full Virtualization”
– The virtual machine monitor provides identical interface to the
underlying physical architecture.
– Advantage of Virtualization:
• Applications can run in virtualized environments without
being rewritten or recompiled.
29
• The Xen virtual machine monitor (aka the Xen hypervisor)
 to provide VMs with a virtualization of the hardware, providing the
appearance that each VM has its own (virtualized) physical machine
and multiplexing the virtual resources onto the underlying physical
resources 30
 Summary
• how the operating system supports the middleware
layer in providing invocations upon shared resources
– Clients invoke operations upon shared resources.
• Concurrent access to shared resources
• Protection of resources and processes
• Processes, Execution environment, Threads
• Communication between remote processes
• Kernel architectures
• Virtualization

31