System Models

Chapter 2: Coulouris +
Chapter notes from K. Birman’s that
in turn was based on Professor Paul
Francis notes, Cornell University

1/14/2018 1
Distributed system models
 Model: “a simplified representation of a
system or phenomenon, as in the sciences
or economics, with any hypotheses
required to describe the system or explain
the phenomenon, often mathematically.”

1/14/2018 2
System Models
 Architectural model defines the way in which the
components of the system are placed and how they interact
with one another and the way in which they are mapped
onto the underlying network of computers.
 Fundamental models:
 Interaction model deals with communication details among the
components and their timing and performance details.
 Failure model gives specification of faults and defines reliable
communication and correct processes.
 Security model specifies possible threats and defines the concept of
secure channels.
 We will discuss the various models at a high level in this
discussion and will elaborate on each of these as we discuss
other systems.

1/14/2018 3
Architectural Model
 Concerned with placement of its parts and
relationship among them.
 Example: client-server model, peer-to-peer
model
 Abstracts the functions of the individual
components.
 Defines patterns for distribution of data and
workload.
 Defines patterns of communication among the
components.
 Example: Definition of server process, client
process and peer process and protocols for
communication among processes; definition
client/server model and its variations.
1/14/2018 4
Software and hardware service layers
in distributed systems

Applic ations, services

Middlew are

Operating sy stem

Platform

Computer and netw ork hardw are

1/14/2018 5
National Weather
Service Web Site RMI WeatherInfo
Client
1 Application 3

IP Socket API Data Aggregator RMI

RMI WeatherInfo
Http Server
2 LAN

Relation Weather Google Map

MySQL
Database Client
4 7

SOAP/REST
Analytics XML Weather
Weather Web Service Web Service
Server Http Web Client 6
5
Middleware
 Layer of software whose purpose is to mask
the heterogeneity and to provide a convenient
programming model for application
programmers.
 Middleware supports such abstractions as
remote method invocation, group
communications, event notification, replication
of shared data, real-time data streaming.
 Examples: Java RMI, grid software (Globus,
Open grid Services), Web services.

1/14/2018 7
Clients invoke individual servers

Client inv oc ation Server

inv oc ation

result
Server
result EX: 1. File server,
2. Web crawler
EX: Web server
Client
Key:
Proc es s: Computer:
EX: browser,
web client

1/14/2018 8
A service provided by multiple
servers
Service

Server
Client

Server

Client
Server

EX: akamai (data duplication), now amazon aws (zones)

1/14/2018 9
Web proxy server and caches

Client Web
s erv er
Prox y
s erv er

Client Web
s erv er

Proxy servers + cache are used to provide increased

Availability and performance. They also play a major role
Firewall based security. http://www.interhack.net/pubs/fwfaq/
1/14/2018 10
A distributed application based on peer processes

Applic ation Applic ation

Coordination Coordination
c ode c ode

Ex: distributed Applic ation

Whiteboard Coordination
Application; c ode
Music sharing
1/14/2018 11
Web applets
a) client request res ults in the dow nloading of applet c ode

Client Web
Applet code s erv er

b) client interac ts w ith the applet

Web
Client Applet s erv er

EX: Code streaming; mobile code

1/14/2018 12
Interaction Models
 Within address space (using path as
addresses)
 Socket based communication: connection-
oriented, connection-less
 Socket is an end-point of communication
 Lets look at some code + details

1/14/2018 13
Socket based communication
int sockfd;
struct sockaddr_in addr;

addr.sin_family = AF_INET;
addr.sin_addr.s_addr =
inet_addr(SERV_HOST_ADDR);
addr.sin_port = htons(SERV_TCP_PORT);

sockfd = socket(AF_INET, SOCK_STREAM, 0);

connect(sockfd, (struct sockaddr *) &addr,
sizeof(serv_addr));
do_stuff(stdin, sockfd);
1/14/2018 14
Classic view of network API
 Start with host name
(maybe) foo.bar.com

1/14/2018 15
Classic view of network API
 Start with host name
 Get an IP address foo.bar.com
gethostbyname()
10.5.4.3

1/14/2018 16
Classic view of network API
 Start with host name
 Get an IP address foo.bar.com
gethostbyname()
 Make a socket 10.5.4.3
(protocol, address) socket();connect();…
sock_id

1/14/2018 17
Classic view of network API
 Start with host name
 Get an IP address foo.bar.com
gethostbyname()
 Make a socket 10.5.4.3
(protocol, address) socket();connect();…
 Send byte stream sock_id
(TCP) or packets
(UDP) 1,2,3,4,5,6,7,8,9 . . .
…

TCP sock UDP sock

Network
Eventually May or may
1/14/2018
arrive in order not arrive 18
Protocol layering
 Communications stack consists of a set of
services, each providing a service to the layer
above, and using services of the layer below
 Each service has a programming API, just like any
software module
 Each service has to convey information one or
more peers across the network
 This information is contained in a header
 The headers are transmitted in the same order as the
layered services

1/14/2018 19
Protocol layering example

Browser Web server

process process

HTTP HTTP

TCP TCP
Router

IP IP IP

Link1 Link1 Link2 Link1

Physical Link 1 Physical Link 2

1/14/2018 20
Protocol layering example
Browser wants to request a page. Calls
Browser HTTP with the web address (URL). Web server
process HTTP’s job is to convey the URL to the process
web server.
HTTP HTTP learns the IP address of the web HTTP
server, adds its header, and calls TCP.
H
TCP TCP
Router

IP IP IP

Link1 Link1 Link2 Link1

Physical Link 1 Physical Link 2

1/14/2018 21
Protocol layering example
TCP’s job is to work with server to
Browser make sure bytes arrive reliably and Web server
process in order. process
TCP adds its header and calls IP.
HTTP (Before that, TCP establishes a HTTP
connection with its peer.)

TCP TCP
H T Router

IP IP IP

Link1 Link1 Link2 Link1

Physical Link 1 Physical Link 2

1/14/2018 22
Protocol layering example
IP’s job is to get the packet routed to
Browser the peer through zero or more Web server
process routers. process
IP determines the next hop from the
HTTP destination IP address. HTTP
IP adds its header and calls the link
layer (i.e. Ethernet) with the next
TCP hop address. TCP
Router

IP IP IP
H T I
Link1 Link1 Link2 Link1
Physical Link 1 Physical Link 2

1/14/2018 23
Protocol layering example
The link’s job is to get the packet to
Browser the next physical box (here a Web server
process router). process
It adds its header and sends the
HTTP resulting packet over the “wire”. HTTP

TCP TCP
Router

IP IP IP

Link1 Link1 Link2 Link1

Physical Link 1 Physical Link 2

H T I L1
1/14/2018 24
Protocol layering example
The router’s link layer receives the
Browser packet, strips the link header, and Web server
process hands the result to the IP forwarding process
process.
HTTP HTTP

TCP TCP
Router

IP IP IP
H T I
Link1 Link1 Link2 Link1
Physical Link 1 Physical Link 2

1/14/2018 25
Protocol layering example
The router’s IP forwarding process
Browser looks at the destination IP address, Web server
process determines what the next hop is, process
and hands the packet to the
HTTP appropriate link layer with the HTTP
appropriate next hop link address.

TCP TCP
Router

IP IP IP
H T I
Link1 Link1 Link2 Link1
Physical Link 1 Physical Link 2

1/14/2018 26
Protocol layering example
The packet goes over the link to the
Browser web server, after which each layer Web server
process processes and strips its process
corresponding header.
HTTP HTTP
H
TCP TCP
Router H T
IP IP IP
H T I
Link1 Link1 Link2 Link1
Physical Link 1 Physical Link 2

H T I L2
1/14/2018 27
Basic elements of any protocol
header
 Demuxing field
 Indicates which is the next higher layer (or
process, or context, etc.)
 Length field or header delimiter
 For the header, optionally for the whole packet
 Header format may be text (HTTP, SMTP
(email)) or binary (IP, TCP, Ethernet)

1/14/2018 28
Demuxing fields
 Ethernet: Protocol Number
 Indicates IPv4, IPv6, (old: Appletalk, SNA, Decnet, etc.)
 IP: Protocol Number
 Indicates TCP, UDP, SCTP
 TCP and UDP: Port Number
 Well known ports indicate FTP, SMTP, HTTP, SIP, many others
 Dynamically negotiated ports indicate specific processes (for these and
other protocols)
 HTTP: Host field
 Indicates “virtual web server” within a physical web server

1/14/2018 29
IP (Internet Protocol)
 Three services:
 Unicast: transmits a packet to a specific host
 Multicast: transmits a packet to a group of hosts
 Anycast: transmits a packet to one of a group of hosts
(typically nearest)
 Destination and source identified by the IP address (32
bits for IPv4, 128 bits for IPv6)
 All services are unreliable
 Packet may be dropped, duplicated, and received in a different
order

1/14/2018 30
IP(v4) address format
 In binary, a 32-bit integer
 In text, this: “128.52.7.243”
 Each decimal digit represents 8 bits (0 – 255)
 “Private” addresses are not globally unique:
 Used behind NAT boxes
 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16
 Multicast addresses start with 1110 as the first 4 bits
(Class D address)
 224.0.0.0/4
 Unicast and anycast addresses come from the same
space

1/14/2018 31
UDP (User Datagram Protocol)
 Runs above IP
 Same unreliable service as IP
 Packets can get lost anywhere:
 Outgoing buffer at source
 Router or link
 Incoming buffer at destination
 But adds port numbers
 Used to identify “application layer” protocols or
processes
 Also a checksum, optional

1/14/2018 32
TCP (Transmission Control
Protocol)
 Runs above IP
 Port number and checksum like UDP
 Service is in-order byte stream
 Application does not absolutely know how the bytes are
packaged in packets
 Flow control and congestion control
 Connection setup and teardown phases
 Can be considerable delay between bytes in at source
and bytes out at destination
 Because of timeouts and retransmissions
 Works only with unicast (not multicast or anycast)

1/14/2018 33
UDP vs. TCP
 UDP is more real-time
 Packet is sent or dropped, but is not delayed
 UDP has more of a “message” flavor
 One packet = one message
 But must add reliability mechanisms over it
 TCP is great for transferring a file or a bunch of email,
but kind-of frustrating for messaging
 Interrupts to application don’t conform to message boundaries
 No “Application Layer Framing”
 TCP is vulnerable to DoS (Denial of Service) attacks,
because initial packet consumes resources at the
receiver

1/14/2018 34
Figure 2.8
Real-time ordering of events
s end receiv e receiv e
X
1 m1 4
m2
s end
receiv e
2 3 Phy sical
Y
receiv e time

s end
Z
receiv e receiv e

m3 m1 m2
A
receiv e receiv e receiv e
t1 t2 t3

Instructor’s Guide for Coulouris,

Dollimore and Kindberg Distributed
Systems: Concepts and Design Edn. 4
Figure 2.9
Processes and channels

proc es sp proc es s q

send m receive

Communic ation c hannel

Outgoing mes sage buffer Inc oming mes sage buffer

Instructor’s Guide for Coulouris,

Dollimore and Kindberg Distributed
Systems: Concepts and Design Edn. 4
Figure 2.10
Omission and arbitrary failures
Class of failure Affects Description
Fail-stop Process Process halts and remains halted. Other processes
detect this state.
Crash Process Process halts and remains halted. Other processes
not be able to detect this state.
Omission Channel A message inserted in an outgoing message buffer n
arrives at the other end’s incoming message buffer.
Send-omission Process A process completes send,
a but the message is not put
in its outgoing message buffer.
Receive-omissionProcess A message is put in a process’s incoming message
buffer, but that process does not receive it.
Arbitrary Process or
Process/channel exhibits arbitrary behaviour: it may
(Byzantine) channel send/transmit arbitrary messages at arbitrary times,
commit omissions; a process may stop or take an
incorrect step.

Instructor’s Guide for Coulouris,

Dollimore and Kindberg Distributed
Systems: Concepts and Design Edn. 4
Figure 2.11
Timing failures

Class of Failure Affects Description

Clock Process Process’s local clock exceeds the bounds on
rate of drift from real time.
Performance Process Process exceeds the bounds on the interval
between two steps.
Performance Channel A message’s transmission takes longer than
stated bound.

Instructor’s Guide for Coulouris,

Dollimore and Kindberg Distributed
Systems: Concepts and Design Edn. 4
Figure 2.12
Objects and principals
Ac cess rights Object
inv oc ation

Client
result Server

Principal (user) Netw ork Principal (s erv er)

Instructor’s Guide for Coulouris,

Dollimore and Kindberg Distributed
Systems: Concepts and Design Edn. 4
Figure 2.13
The enemy

Copy ofm

The enemy
m’
Processp m Processq
Communication channel

Instructor’s Guide for Coulouris,

Dollimore and Kindberg Distributed
Systems: Concepts and Design Edn. 4
Figure 2.14
Secure channels

PrincipalA PrincipalB

Processp Secure channel Processq

Instructor’s Guide for Coulouris,

Dollimore and Kindberg Distributed
Systems: Concepts and Design Edn. 4
Summary
 When designing systems or analyzing
systems, you want to examine at the high
level the architectural model.
 Subsequent steps will explore
fundamental models such as interaction
model, security model, failure model,
reliability model etc.

1/14/2018 42