Distributed Systems

Principles and Paradigms

Maarten van Steen
VU Amsterdam, Dept. Computer Science
Room R4.20, [email protected]
Chapter 02: Architectures
Version: October 25, 2009
Contents
Chapter
01: Introduction
02: Architectures
03: Processes
04: Communication
05: Naming
06: Synchronization
07: Consistency & Replication
08: Fault Tolerance
09: Security
10: Distributed Object-Based Systems
11: Distributed File Systems
12: Distributed Web-Based Systems
13: Distributed Coordination-Based Systems
2 / 25
Architectures
Architectures
Architectural styles
Software architectures
Architectures versus middleware
Self-management in distributed systems
3 / 25
Architectures 2.1 Architectural styles
Architectural styles
Basic idea
Organize into logically different components, and distribute those
components over the various machines.
Layer N
Layer N-1
Layer 1
Layer 2
Request
flow
Response
flow
(a) (b)
Object
Object
Object
Object
Object
Method call
(a) Layered style is used for client-server system
(b) Object-based style for distributed object systems.
4 / 25
Architectures 2.1 Architectural styles
Architectural Styles
Observation
Decoupling processes in space (anonymous) and also time
(asynchronous) has led to alternative styles.
(a)
(b)
Component Component
Component
Event bus
Publish
Publish
Event delivery
Component Component
Data delivery
Shared (persistent) data space
(a) Publish/subscribe [decoupled in space]
(b) Shared dataspace [decoupled in space and time]
5 / 25
Architectures 2.2 System Architectures
Centralized Architectures
Basic ClientServer Model
Characteristics:
There are processes offering services (servers)
There are processes that use services (clients)
Clients and servers can be on different machines
Clients follow request/reply model wrt to using services
Client
Request Reply
Server
Provide service
Time
Wait for result
6 / 25
Architectures 2.2 System Architectures
Application Layering
Traditional three-layered view
User-interface layer contains units for an applications user
interface
Processing layer contains the functions of an application, i.e.
without specic data
Data layer contains the data that a client wants to manipulate
through the application components
Observation
This layering is found in many distributed information systems, using
traditional database technology and accompanying applications.
7 / 25
Architectures 2.2 System Architectures
Application Layering
Traditional three-layered view
User-interface layer contains units for an applications user
interface
Processing layer contains the functions of an application, i.e.
without specic data
Data layer contains the data that a client wants to manipulate
through the application components
Observation
This layering is found in many distributed information systems, using
traditional database technology and accompanying applications.
7 / 25
Architectures 2.2 System Architectures
Application Layering
Database
with Web pages
Query
generator
Ranking
algorithm
HTML
generator
User interface
Keyword expression
Database queries
Web page titles
with meta-information
Ranked list
of page titles
HTML page
containing list
Processing
level
User-interface
level
Data level
8 / 25
Architectures 2.2 System Architectures
Multi-Tiered Architectures
Single-tiered: dumb terminal/mainframe conguration
Two-tiered: client/single server conguration
Three-tiered: each layer on separate machine
Traditional two-tiered congurations:
User interface User interface User interface
Application
User interface
Application
User interface
Application
Database
Application Application Application
Database Database Database Database Database
User interface
(a) (b) (c) (d) (e)
Client machine
Server machine
9 / 25
Architectures 2.2 System Architectures
Decentralized Architectures
Observation
In the last couple of years we have been seeing a tremendous growth
in peer-to-peer systems.
Structured P2P: nodes are organized following a specic
distributed data structure
Unstructured P2P: nodes have randomly selected neighbors
Hybrid P2P: some nodes are appointed special functions in a
well-organized fashion
Note
In virtually all cases, we are dealing with overlay networks: data is
routed over connections setup between the nodes (cf. application-level
multicasting)
10 / 25
Architectures 2.2 System Architectures
Decentralized Architectures
Observation
In the last couple of years we have been seeing a tremendous growth
in peer-to-peer systems.
Structured P2P: nodes are organized following a specic
distributed data structure
Unstructured P2P: nodes have randomly selected neighbors
Hybrid P2P: some nodes are appointed special functions in a
well-organized fashion
Note
In virtually all cases, we are dealing with overlay networks: data is
routed over connections setup between the nodes (cf. application-level
multicasting)
10 / 25
Architectures 2.2 System Architectures
Structured P2P Systems
Basic idea
Organize the nodes in a structured overlay network such as a logical
ring, and make specic nodes responsible for services based only on
their ID.
0
15
2 14
3 13
4 12
8
7 9
6 10
5 11
1
Actual node
{2,3,4}
{5,6,7}
{8,9,10,11,12}
{13,14,15} {0,1}
Associated
data keys
Note
The system provides an operation
LOOKUP(key) that will efciently
route the lookup request to the
associated node.
11 / 25
Architectures 2.2 System Architectures
Structured P2P Systems
Other example
Organize nodes in a d-dimensional space and let every node take the
responsibility for data in a specic region. When a node joins split a
region.
(0.2,0.8)
(0.6,0.7)
(0.9,0.9)
(0.2,0.3)
(0.7,0.2)
(0.9,0.6)
(0,0)
Keys associated with
node at (0.6,0.7)
(0.2,0.8)
(0.6,0.7)
(0.9,0.9)
(0.2,0.45)
(0.7,0.2)
(0.9,0.6)
(0.2,0.15)
(1,0)
(0,1) (1,1)
Actual node
(a)
(b)
12 / 25
Architectures 2.2 System Architectures
Unstructured P2P Systems
Observation
Many unstructured P2P systems attempt to maintain a random graph.
Basic principle
Each node is required to contact a randomly selected other node:
Let each peer maintain a partial view of the network, consisting of c
other nodes
Each node P periodically selects a node Q from its partial view
P and Q exchange information and exchange members from their
respective partial views
Note
It turns out that, depending on the exchange, randomness, but also
robustness of the network can be maintained.
13 / 25
Architectures 2.2 System Architectures
Topology Management of Overlay Networks
Basic idea
Distinguish two layers: (1) maintain random partial views in lowest layer;
(2) be selective on who you keep in higher-layer partial view.
Protocol for
randomized
view
Protocol for
specific
overlay
Random peer
Links to randomly
chosen other nodes
Links to topology-
specific other nodes
Random
overlay
Structured
overlay
Note
Lower layer feeds upper layer with random nodes; upper layer is selective
when it comes to keeping references.
14 / 25
Architectures 2.2 System Architectures
Topology Management of Overlay Networks
Constructing a torus
Consider a NN grid. Keep only references to nearest neighbors:
(a
1
, a
2
) (b
1
, b
2
) =d
1
+d
2
d
i
=min{N|a
i
b
i
|, |a
i
b
i
|}
Time
15 / 25
Architectures 2.2 System Architectures
Superpeers
Observation
Sometimes it helps to select a few nodes to do specic work:
superpeer.
Superpeer
Regular peer
Superpeer
network
Examples
Peers maintaining an
index (for search)
Peers monitoring the
state of the network
Peers being able to setup
connections
16 / 25
Architectures 2.2 System Architectures
Hybrid Architectures: Client-server combined with P2P
Example
Edge-server architectures, which are often used for Content Delivery
Networks
Edge server
Core Internet
Enterprise network
ISP
ISP
Client
Content provider
17 / 25
Architectures 2.2 System Architectures
Hybrid Architectures: C/S with P2P BitTorrent
Node 1
Node 2
Node N
.torrent file
for F
A BitTorrent
Web page
List of nodes
storing F
Web server File server Tracker
Client node
K out of N nodes
Lookup(F)
Ref. to
file
server
Ref. to
tracker
Basic idea
Once a node has identied where to download a le from, it joins a
swarm of downloaders who in parallel get le chunks from the source,
but also distribute these chunks amongst each other.
18 / 25
Architectures 2.3 Architectures versus Middleware
Architectures versus Middleware
Problem
In many cases, distributed systems/applications are developed
according to a specic architectural style. The chosen style may not be
optimal in all cases need to (dynamically) adapt the behavior of the
middleware.
Interceptors
Intercept the usual ow of control when invoking a remote object.
19 / 25
Architectures 2.3 Architectures versus Middleware
Interceptors
Client application
B.do_something(value)
invoke(B, &do_something, value)
send([B, "do_something", value])
Request-level interceptor
Message-level interceptor
Object middleware
Local OS
Application stub
To object B
Nonintercepted call
Intercepted call
20 / 25
Architectures 2.3 Architectures versus Middleware
Adaptive Middleware
Separation of concerns: Try to separate extra functionalities and later
weave them together into a single implementation only toy
examples so far.
Computational reection: Let a program inspect itself at runtime and
adapt/change its settings dynamically if necessary mostly at
language level and applicability unclear.
Component-based design: Organize a distributed application through
components that can be dynamically replaced when needed
highly complex, also many intercomponent dependencies.
Fundamental question
Do we need adaptive software at all, or is the issue adaptive systems?
21 / 25
Architectures 2.3 Architectures versus Middleware
Adaptive Middleware
Separation of concerns: Try to separate extra functionalities and later
weave them together into a single implementation only toy
examples so far.
Computational reection: Let a program inspect itself at runtime and
adapt/change its settings dynamically if necessary mostly at
language level and applicability unclear.
Component-based design: Organize a distributed application through
components that can be dynamically replaced when needed
highly complex, also many intercomponent dependencies.
Fundamental question
Do we need adaptive software at all, or is the issue adaptive systems?
21 / 25
Architectures 2.3 Architectures versus Middleware
Adaptive Middleware
Separation of concerns: Try to separate extra functionalities and later
weave them together into a single implementation only toy
examples so far.
Computational reection: Let a program inspect itself at runtime and
adapt/change its settings dynamically if necessary mostly at
language level and applicability unclear.
Component-based design: Organize a distributed application through
components that can be dynamically replaced when needed
highly complex, also many intercomponent dependencies.
Fundamental question
Do we need adaptive software at all, or is the issue adaptive systems?
21 / 25
Architectures 2.3 Architectures versus Middleware
Adaptive Middleware
Separation of concerns: Try to separate extra functionalities and later
weave them together into a single implementation only toy
examples so far.
Computational reection: Let a program inspect itself at runtime and
adapt/change its settings dynamically if necessary mostly at
language level and applicability unclear.
Component-based design: Organize a distributed application through
components that can be dynamically replaced when needed
highly complex, also many intercomponent dependencies.
Fundamental question
Do we need adaptive software at all, or is the issue adaptive systems?
21 / 25
Architectures 2.3 Architectures versus Middleware
Adaptive Middleware
Separation of concerns: Try to separate extra functionalities and later
weave them together into a single implementation only toy
examples so far.
Computational reection: Let a program inspect itself at runtime and
adapt/change its settings dynamically if necessary mostly at
language level and applicability unclear.
Component-based design: Organize a distributed application through
components that can be dynamically replaced when needed
highly complex, also many intercomponent dependencies.
Fundamental question
Do we need adaptive software at all, or is the issue adaptive systems?
21 / 25
Architectures 2.4 Self-management in Distributed Systems
Self-managing Distributed Systems
Observation
Distinction between system and software architectures blurs when
automatic adaptivity needs to be taken into account:
Self-conguration
Self-managing
Self-healing
Self-optimizing
Self-*
Warning
There is a lot of hype going on in this eld of autonomic computing.
22 / 25
Architectures 2.4 Self-management in Distributed Systems
Self-managing Distributed Systems
Observation
Distinction between system and software architectures blurs when
automatic adaptivity needs to be taken into account:
Self-conguration
Self-managing
Self-healing
Self-optimizing
Self-*
Warning
There is a lot of hype going on in this eld of autonomic computing.
22 / 25
Architectures 2.4 Self-management in Distributed Systems
Feedback Control Model
Observation
In many cases, self-* systems are organized as a feedback control
system.
Core of distributed system
Metric
estimation
Analysis
Adjustment
measures
+/-
+/-
+/-
Reference input
Initial configuration
Uncontrollable parameters (disturbance / noise)
Observed output
Measured output
Adjustment triggers
Corrections
23 / 25
Architectures 2.4 Self-management in Distributed Systems
Example: Globule
Globule
Collaborative CDN that analyzes traces to decide where replicas of
Web content should be placed. Decisions are driven by a general cost
model:
cost = (w
1
m
1
) +(w
2
m
2
) + +(w
n
m
n
)
24 / 25
Architectures 2.4 Self-management in Distributed Systems
Example: Globule
Replica server
Core Internet
Enterprise network
ISP
ISP
Client
Origin server
Client Client
Globule origin server collects traces and does what-if analysis by
checking what would have happened if page P would have been
placed at edge server S.
Many strategies are evaluated, and the best one is chosen.
25 / 25