CHAPTER

8
AGREEMENT
PROTOCOLS

8.1 INTRODUCTION
In distributed systems, where sites (or processors) often compete as well as cooperate
to achieve a common goal, it is often required that sites reach mutual agreement. For
example, in distributed database systems, data managers at sites must agree on whether
to commit or to abort a transaction [11]. Reaching an agreement
typically requlres
that sites have knowledge about the values of other sites. For example, in distributed
commit, a site should knowthe outcome of local commit at each site.
When the system is free from failures, an agreement can easily be
the processors (or sites). For example, reached among
nicating their values to each other and processors can reach an agreement by cOni
maximum, mean, etc. of those values, then by taking minimum,
However, when athemajority
systemvote or a
is prone tofailure,
this method does not work. This is
to other processors because faulty processors can send conflicting vaalues
faults, processors mustpreventing them from reaching an agreement. In the presenceof
exchange their values with other processors and relay the values
received from other processors several
value as it learnstimes to isolate the effects of
Aprocessor refines its processors.
faulty
Drocess of reaching an agreement is of the values of other processors (This entire
In this chapter, we called an agreement
sor failures. Avery study agreement protocols for protocol).systems under proces-
may send general
spurious messages model of faults is assumed.distributed processor
For example, afaulty received
to other
178 processors, may lie, may not respond to
 AGREEMENT PROTOCOLS 179

messages Correctly, etc. Also, nonfaulty processors do not know which processors are
faulty.
In agreement problens, nonfaulty processors in adistributed system should be able
to reach a common agreement, even if certain components in the system are faulty. 1ne
agreement is achieved through an agreement protocol that involves several rounds of
message exchange among the processors.

8.2 THE SYSTEM MODEL

Agreement problems have been studied under the following system model:
There are n processors in the system and at most m of the processors can be faulty.
"The processors can directly communicate with other processors by message passing.
Thus, the system is logically fully connected.
A receiver processor always knows the identity of the sender processor of the mes
sage.
The communication medium is reliable (i.e., it delivers all messages without intro
ducing any errors) and only processors are prone to failures.

For simplicity, we assume that agreement is to be reached between only two

values, 0 and 1. Results can easily be extended to multivalue agreement [23].
Early solutions to agreement problems assumed that only processors could be
faulty and that communication links did not fail. Limiting faults solely to the processors
simplifies the solution toagreement problems. Recently, agreement problems have been
studied under the failure of communication inks only [24] and under the failure of
both processors and communication inks (25] In this chapter, we limit the treatment
of agreement problems solely to processor failures.

8.2.1 Synchronous vs. Asynchronous Computations

In a(synchronous computation, processes in the system runin lock step manner, where
in each step, a process receives messages (sent to it in the previous step), performs_a
computation, and sends messages to other processes (received in the next step). A step of
a synchronous computation is also referred to as a round |In synchronous computation,
a process knows all the messages it expects to receive in a round. A meSsage delay or
a slow process can slow down the entire system or _computation
Ih an asynchronous computation, on the other hand, the computation at processes
does not proceed in lock steps. A process can send and receive messages and perform
computation at any time.
In this chapter, synchronous models of computation are assumed. The
assumption
of synchronous computation is critical to agreement protocols. In fact, the agreement
problem is not solvable in an asynchronous system, even for a single processor fail
ure [10].
 SYSTEMS
180 AD\ANCED ONCEPTS IN OPERATING

Failures
8.2.2 Model of Processor model of processOr failures. A pro-
very gencral maliciouss
In agreement problems, we consider a fault, and fault. In a crash
three modes: çrash fault, omission
resumes operation. In an omission fault,
cessor can fail in functioning and never processors.(These are the meSsages that
fault, a processor stops
messages to some protocol or algorithm it is
a processor "omits"
should
to send
have sent according to
the
broadcast a message to
all other executing.)
processors,
the processor supposed to malicious fault, a processor be-
processor is processors. In a
For example, a only a few may send fictitious messages
but it sends the message to For example, a processor broad in nature and thus
haves randomly and arbitrarily.them. Malicious faults are very Malicious faults
confuse
to other processors to faults can be treated as malicious faults.
most other conceivable mav.
Byzantine faults.
also referred to as processor to send a message, a nonfaulty processor wa
can refuse a situation
Since a faulty message from a faulty processor. ln such acts as if the
never receive an expectedprocessor simply chooses an arbitrary value and situatione
assume that the nonfaulty received [16]. Of course, we assume that such receiver
expected message has been message, can be detected by the
respective
to send a then thic
where a processor refuses systems, if the duration of each round is known,
processors. In synchronous round were
the expected messages not received by the end of a
detection is simple-all
not sent.

Non-Authenticated Messages
8.2.3 Authenticated vs.
agreement, processors have to exchange their values and relay the
Note that toreach an several times. The capability of faulty processors
received values to other processors
what they receive from other processors greatly depends upon the type of
to distort
underlying messages. authenticated and non-authenticated. In an
There are two types of messages:
forge a message or change
authenticated message system, a (faulty) processor cannot
message to other processors). A
the contents of a received message (before it relays the An authenticated message
processor can verify the authenticity of a received message.
is also called a signed message [14].
forge a message
In a non-authenticated message system, a (faulty) processor can
receiveu
and claim to have receivedit from another processor or change the contents of a
no way ol
message before it relays the,message to other processors. A processor has
verifying the authenticity of a received message. A non-authenticated message 1s also
called an oral message [14]. It is easier to reach agreement in an authenticated messago
system because faulty processors are capable of doing less damage.
8.2.4 Performance Aspects
The performance (or the computational complexity) of agreernentprotocols is generally
Overhead.
determined by the following three metrics: time, message traffic, and storage
Time refers to the time taken to rench an agreement under a protocol. The time is usually
is
expressed as the number of rounds needed to reach an agreement. MeSsag traffic
 AGREEMENT PROTOCOLS 181

measured by the number of messages exchanged to reach an agreement. Sometimes,

the message traffic is also measured by the total number of bits exchanged to reach an
agreement [5]. Storage overhead measures the amount of information that needs to be
stored at processors during the execution of a protocol.
Next, we discuss three agreement problems for non-authenticated messages under
processor failures.

8.3 A CLASSIFICATION OF AGREEMENT PROBLEMS

There are three wellknown agreement problems in distributed systems: the Byzantine
agreement problem, the consensus problem, and the interactive consistency problem. In
the Byzantine agreement problem, a single value, which is to be agreed on, is initialized
by an arbitrary processor and all nonfaulty processors have to agree on that value. In the
consensus problem, every processor has its own initial value and allnonfaulty processors
must agree on asingle common value. In the interactive consistency problem, every
processor has its own initial value and all nonfaulty processors must agree on a set of
common values.
In all three problems, fall nonfaulty processors must reach a common agreement.
In the Byzantine agreement àndthe consensus problems, the agreement is about a single
value. Whereas in the interactive consistency problem, the agreement is about a set of
common values. In the Byzantine agreement problem, only one processor initializes the
initial value. Whereas in the consensus and the interactive consistency problems, every
values and final
processor has its own initial value. Table 8.1 summarizes the starting
outcomes of the three problems.
Next, we define these three agreement problems in a precise manner.

8.3.1 The Byzantine Agreement Problem

processor, called the source
In the Byzantine agreement problem, an arbitrarily chosen solution to the Byzantine
processor, broadcasts its initial value to all other processors. A
objectives:
agreement problem should meet the following two
Agreement. All nonfaulty processors agree on the same value.
agreed upon value
Validity. If the source processor is nonfaulty, then the common
source.
initial value of the
by all nonfaulty processors should be the

TABLE 8.1
The three agreement problems
Consensus Interactive
Problem ’ Byzantine
Agreement Consistency

Who initiates One processor All processors Allprocessors

the value A vector of values
Final agreement Single value Single value
182 ADVANCED ONCEPTS IN
OPERATING SYSTEMS

ho points should be noted: (1) 1f the source processor is faulty, then all non
faultypOcessorS can agree on any'common value. (2) It is irrelevant what value faulty
processors agree on or whether theycommon
agree on a value at all
8.3.2 The Consensus Problem
lt ne consensus problem, every processor broadcasts its initial value to all other
prOCes
SOS. Iniial values of the processors maybe different. A protocol for reaching
should meet the consensus
following conditions:
Agreement All nonfaulty processors agree on the same single value.
Validity If the initial value of every nonfaulty processor is u,
upon common value by all then the agreed
nonfaulty processors must be v.
Note that if the initial values of
Taulty processors can agree on any nonfaulty processors are different, then all non
faulty processors agree on. common value. Again, we don't care what value

8.3.3 The Interactive

In the interactive
Consistency Problem
consistency problem, every
initial values of the processor broadcasts
all other
processors. The
for the interactive
its initial value to
consistencyproblem shouldprocessors may be
meet the followingdifferent. A protocol
Agreement. All nonfaulty processors agree On the conditions:
Validity. If the ith processor is nonfaulty and its same vector, (v1, U, .... Un).
nonfaulty processors must beinitial value is Vis then the ith
value to be agreed on by all
v;.
Note that if the jth
any common value for v;.processor
is faulty, then all
It is irrelevant what value nonfaulty
processors can agree
faulty processors on
8.3.4 Relations Among the agree on.
Agreement Problems
All