Synchronization Algorithms

and Concurrent Programming

Gadi Taubenfeld
Chapter 3
Mutual Exclusion using atomic registers:
Advanced Topics

Version: June 2014

Chapter 3 Synchronization Algorithms and Concurrent Programming 1
Gadi Taubenfeld © 2014
 Synchronization Algorithms
and Concurrent Programming
ISBN: 0131972596, 1st edition

A note on the use of these power-point slides:

I am making these slides freely available to all (faculty, students, readers).
They are in PowerPoint form so you can add, modify, and delete slides and slide
content to suit your needs. They obviously represent a lot of work on my part.
In return for use, I only ask the following:

q That you mention their source, after all, I would like people to use my book!
q That you note that they are adapted from (or perhaps identical to)
my slides, and note my copyright of this material.

Thanks and enjoy!

Gadi Taubenfeld
All material copyright 2014
Gadi Taubenfeld, All Rights Reserved

To get the most updated version of these slides go to:

http://www.faculty.idc.ac.il/gadi/book.htm
Chapter 3 Synchronization Algorithms and Concurrent Programming 2
Gadi Taubenfeld © 2014
 Chapter 3
Mutual Exclusion using atomic
registers: Advanced Topics

3.1 Local Spinning Algorithms

3.2 Adaptive Algorithms
3.3 Fault-tolerant Algorithms Algorithms
3.4 Symmetric Algorithms

Chapter 3 Synchronization Algorithms and Concurrent Programming 3

Gadi Taubenfeld © 2014
 Spinning

P1 P2 P3 P4

Entry

Critical Section

Exit

spinning – good for short delays Contention!

“go to sleep” – when delays are long Sequential Bottleneck

Chapter 3 Synchronization Algorithms and Concurrent Programming 4

Gadi Taubenfeld © 2014
 Model 1: simple shared memory

P1 P2 P3 P4

Shared memory

• All memory accesses are remote

Chapter 3 Synchronization Algorithms and Concurrent Programming 5

Gadi Taubenfeld © 2014
 Model 2: Coherent Caching (CC)

P1 P2 P3 P4

C1 C2 C3 C4

Shared memory

• Local and remote memory accesses

Chapter 3 Synchronization Algorithms and Concurrent Programming 6

Gadi Taubenfeld © 2014
 Model 3: Distributed Shared Memory (DSM)

P1 P2 P3 P4

M1 M2 M3 M4

• Local and remote memory accesses

• What is local spinning ?

Chapter 3 Synchronization Algorithms and Concurrent Programming 7

Gadi Taubenfeld © 2014
 Local Spinning Algorithms
Section 3.1

Chapter 3 Synchronization Algorithms and Concurrent Programming 8

Gadi Taubenfeld © 2014
 Local-spinning Algorithms

An algorithm satisfies local-spinning if it is possible

to physically distribute the shared memory among
the processes in such a way that the only type of
spinning required is local-spinning.

P1 P2 P3 P4

M1 M2 M3 M4

Chapter 3 Synchronization Algorithms and Concurrent Programming 9

Gadi Taubenfeld © 2014
 Question

Is it true that in local-spinning algorithms for

the CC model, different processes must spin
on different memory locations?

Answer: No.

Chapter 3 Synchronization Algorithms and Concurrent Programming 10

Gadi Taubenfeld © 2014
 Question

Is it true that in local-spinning algorithms for

the DSM model, different processes must
spin on different memory locations?

Answer: Yes.

Chapter 3 Synchronization Algorithms and Concurrent Programming 11

Gadi Taubenfeld © 2014
 Section 3.1.2

The Local spinning Black-White Bakery Algorithm

Bakery

Black-White Bakery
Local-spinning

Local-spinning

Chapter 3 Synchronization Algorithms and Concurrent Programming 12

Gadi Taubenfeld © 2014
 The Bakery Algorithm
code of process i , i Î {1 ,..., n}

choosing[i] = true
number[i] = 1 + max {number[j] | (1 £ j £ n)}
choosing[i] = false
for j = 1 to n {
await choosing[j] = false 1. Wait for j to choose a number
await (number[j] = 0) Ú (number[j],j)
2. Wait for j³to(number[i],i)
finish its critical section
}
critical section
number[i] = 0

1 2 3 4 n
choosing false false false false false false bits

number 0 0 0 0 0 0 integer

Chapter 3 Synchronization Algorithms and Concurrent Programming 13

Gadi Taubenfeld © 2014
 The Local spinning Bakery Algorithm
code of process i , i Î {1 ,..., n}

choosing[i] = true
number[i] = 1 + max {number[j] | (1 £ j £ n)}
choosing[i] = false
for j = 1 to n {spin.ch[j,i] = false}
for j = 1 to n {
spin.ch[i,j] = true
if choosing[i] = true then await spin.ch[i,j] = false fi
spin.nu[i,j] = true
if (number[j] = 0) Ú (number[j],j) ³ (number[i],i)
then skip else await spin.nu[i,j] = false fi
}
critical section
number[i] = 0
for j = 1 to n {spin.nu[i,j] = false}

Chapter 3 Synchronization Algorithms and Concurrent Programming 14

Gadi Taubenfeld © 2014
 The Black-White Bakery Algorithm

choosing[i] = true
mycolor[i] = color
number[i] = 1 + max{number[j] | (1 £ j £ n) Ù (mycolor[j] = mycolor[i])}
choosing[i] = false
for j = 0 to n do
1. await choosing[j] = false 1. Wait for j to choose a number
if mycolor[j] = mycolor[i]
then await (number[j] = 0) Ú (number[j],j) ³ (number[i],i) Ú
2. (mycolor[j] ¹ mycolor[i])
2. Wait for j to finish its critical section
else await (number[j] = 0) Ú (mycolor[i] ¹ color) Ú
(mycolor[j] = mycolor[i]) fi od
critical section
if mycolor[i] = black then color = white else color = black fi
number[i] = 0

Chapter 3 Synchronization Algorithms and Concurrent Programming 15

Gadi Taubenfeld © 2014
 The Local spinning Black-White Bakery Algorithm

q Mutual exclusion, starvation-freedom, FIFO

q Satisfies local spinning in both the CC model and
DSM model
m time complexity is O(n) in both the CC model
and DSM model
q 0(n^2) bounded size shared registers are used

Chapter 3 Synchronization Algorithms and Concurrent Programming 16

Gadi Taubenfeld © 2014
 Adaptive Algorithms
Section 3.2

An algorithm is adaptive if its time complexity

is a function of the actual number of contending
processes.

Chapter 3 Synchronization Algorithms and Concurrent Programming 17

Gadi Taubenfeld © 2014
 Contention

T1
p1
T2
p2
T3
p3
time

• The point contention over T1 is 2.

• The interval contention over T1 is 3.

Chapter 3 Synchronization Algorithms and Concurrent Programming 18

Gadi Taubenfeld © 2014
 Section 2.3

Fast Mutual exclusion Algorithm

q Mutual exclusion and deadlock-freedom
q Starvation of individual processes is possible
q fast access
m in the absence of contention, only 7
accesses to shared memory are needed
q With contention è not adaptive
m Even if only 2 processes contend, the
winner may need to check all the 0(n)
shared registers
m System response time is of order n time
units
q n+2 shared registers are used

Chapter 3 Synchronization Algorithms and Concurrent Programming 19

Gadi Taubenfeld © 2014
 Section 3.2.2

Algorithm #1
A simple adaptive algorithm
for unbounded concurrency

q Mutual exclusion and deadlock-freedom

q Starvation of individual processes is possible
q fast access
m in the absence of contention, only 8
accesses to shared memory are needed
q Adaptive w.r.t. system response time
m system response time is O(k) time units
where k is point contention
q Works also for unbounded # of processes
q Symmetric
q Infinite number of shared registers are used

Chapter 3 Synchronization Algorithms and Concurrent Programming 20

Gadi Taubenfeld © 2014
 MA-splitter

q At most n-1 can move right

n
q At most n-1 can move down
q At most 1 can win 1 n-1
win right
q In solo run è 1 win
n-1
down

Chapter 3 Synchronization Algorithms and Concurrent Programming 21

Gadi Taubenfeld © 2014
 MA-splitter
implementation

x=i
if y = 1 then go right fi n
y=1
if x ¹ i then go down fi 1 n-1
win right
win
x n-1
down
y 0

Chapter 3 Synchronization Algorithms and Concurrent Programming 22

Gadi Taubenfeld © 2014
 MA-splitter
implementation

y=1 £ n-1
x=i right
(and all
y¹1
latecomer)
£1 x=i
win y=1
x¹1

£ n-1 or there is a latecomer

down

Chapter 3 Synchronization Algorithms and Concurrent Programming 23

Gadi Taubenfeld © 2014
 MT-splitter
Redefine n to be the earlybirds (non-latecomers)

q At most n-1 can move right

n
q At most n-1 can move down
q At most 1 can win 1 n-1
win right
q In solo run è 1 win
n-1
q All latecomers move right
down
q If a process wins then
nobody moves down

Chapter 3 Synchronization Algorithms and Concurrent Programming 24

Gadi Taubenfeld © 2014
Mutual exclusion using a infinite
chain of the MT-splitters: infinite-1
1
win right
n-1

n’

1 n’-1
win right
n’-1
down

Chapter 3 Synchronization Algorithms and Concurrent Programming 25

Gadi Taubenfeld © 2014
 MT-splitter
implementation

x=i
if y = 1 then b = 1; go right fi
x
y=1
if x = i then z = 1 y 0
if b = 0 then go win else go down fi
else await b = 1 or z = 1 z 0
if z = 1 then go right else go down fi
fi
b 0

win right

down

Chapter 3 Synchronization Algorithms and Concurrent Programming 26

Gadi Taubenfeld © 2014
 MT-splitter
implementation

b=1 1 n-1
win z=1 MA-
b=1 right
splitter
n-1
b¹1 z=1
await (b = 1 or z = 1)

z¹1

down

Chapter 3 Synchronization Algorithms and Concurrent Programming 27

Gadi Taubenfeld © 2014
 Adaptive mutex for unbounded concurrency

start: level = next

repeat
x[level] = i
if y[level] then b[level] = 1; await level < next;
goto start fi
y[level] = 1
¹
if x[level] i then await b[level] = 1 or z[level] = 1
if z[level] = 1 then await level < next;
goto start
else level = level+1 fi
else z[level] = 1
if b[level] = 0 then win = 1 else level = level+1 fi
fi
until win = 1
critical section
next := level+1

Chapter 3 Synchronization Algorithms and Concurrent Programming 28

Gadi Taubenfeld © 2014
 Section 3.2.3

Algorithm #2
An Adaptive Tournament Algorithm

q Mutual exclusion and starvation-freedom

q Adaptive w.r.t. time complexity in both the CC
model and DSM model
m time complexity is O(min(k, log n) remote
memory accesses, where k is point contention
q 0(n^2) bounded size shared registers are used

Chapter 3 Synchronization Algorithms and Concurrent Programming 29

Gadi Taubenfeld © 2014
 Section 3.2.3

Algorithm #2
An Adaptive Tournament Algorithm

critical
start here section

2 get 3 compete
name

4 5 6 7

renaming tree three-based tree

Chapter 3 Synchronization Algorithms and Concurrent Programming 30

Gadi Taubenfeld © 2014
 Section 3.2.3

Algorithm #2
An Adaptive Tournament Algorithm

p q
2-process
mutex

O(log n)
height

renaming three-based overflow

tree tree tree

Chapter 3 Synchronization Algorithms and Concurrent Programming 31

Gadi Taubenfeld © 2014
 Section 3.2.4

The Adaptive Black-White Bakery Algorithm

q Mutual exclusion, deadlock-freedom, FIFO-fairness

q Adaptive w.r.t. time complexity in the CC model
m can be modified to be adaptive w.r.t. time
complexity in the DSM model
q Finite number of bounded size shared registers are
used

Bakery

Black-White Bakery

Adaptive
Chapter 3 Synchronization Algorithms and Concurrent Programming 32
Gadi Taubenfeld © 2014
 Active set

• join(S)

• leave(S) P1 P2 P3

• Get-set(S)

P1 P2

Chapter 3 Synchronization Algorithms and Concurrent Programming 33

Gadi Taubenfeld © 2014
 Active set
A simple non-adaptive implementation

• join(S) – set bit to 1

• leave(S) – set bit to 0

• Get-set(S) – scan S return all bits which are 1

1 2 3 4 n
S 0 0 0 0 0 0 bits

• What about an adaptive implementation ?

• wait-free vs. fault-free

Chapter 3 Synchronization Algorithms and Concurrent Programming 34

Gadi Taubenfeld © 2014
 The Adaptive Black-White Bakery Algorithm
code of process i , i Î {1 ,..., n}
join(S)
choosing[i] = true
localS = getset(S)
mycolor[i] = color
number[i] = 1 + max{number[j] | jÎ localS Ù (mycolor[j] = mycolor[i])}
choosing[i] = false
localS = getset(S)
for every j Î localS do
await choosing[j] = false
if mycolor[j] = mycolor[i]
then await (number[j] = 0) Ú (number[j],j) ³ (number[i],i) Ú
(mycolor[j] ¹ mycolor[i])
else await (number[j] = 0) Ú (mycolor[i] ¹ color) Ú
(mycolor[j] = mycolor[i]) fi od
critical section
if mycolor[i] = black then color = white else color = black fi
number[i] = 0
leave(S)
Chapter 3 Synchronization Algorithms and Concurrent Programming 35
Gadi Taubenfeld © 2014
 The Adaptive Black-White Bakery Algorithm
code of process i , i Î {1 ,..., n}
join(S)
choosing[i] = true
localS = getset(S)
mycolor[i] = color
number[i] = 1 + max{number[j] | jÎ localS Ù (mycolor[j] = mycolor[i])}
choosing[i] = false
localS = getset(S) Can we omit No. mutex would
for every j Î localS do this line? not be satisfied
await choosing[j] = false
if mycolor[j] = mycolor[i]
then await (number[j] = 0) Ú (number[j],j) ³ (number[i],i) Ú
(mycolor[j] ¹ mycolor[i])
else await (number[j] = 0) Ú (mycolor[i] ¹ color) Ú
(mycolor[j] = mycolor[i]) fi od
critical section
if mycolor[i] = black then color = white else color = black fi
number[i] = 0
leave(S)
Chapter 3 Synchronization Algorithms and Concurrent Programming 36
Gadi Taubenfeld © 2014
 Question
Design alg. that satisfies all 4 properties

Bakery (FIFO, unbounded)

Black-White Bakery

FIFO
Bounded space
Local-spinning Adaptive Adaptive
Local-spinning

Chapter 3 Synchronization Algorithms and Concurrent Programming 37

Gadi Taubenfeld © 2014
 Question
Show that there is no adaptive algorithm
that uses only single-writer bits.
Bakery

Black-White Bakery

Single-writer
Local-spinning bits
Adaptive

Yes No

All
Chapter 3 Synchronization Algorithms and Concurrent Programming 38
Gadi Taubenfeld © 2014
 Section 3.2.5
Impossibility

Theorem: There is no two process mutual exclusion

algorithm with an upper bound on the number of
times a winning process may need to access the
shared memory in order to enter its CS in presence
of contention.

èThere is no adaptive mutual exclusion algorithm

when time is measured by counting all accesses
(local and remote) to shared registers. That is,
there is no adaptive mutual exclusion algorithm
w.r.t. process time complexity.

Chapter 3 Synchronization Algorithms and Concurrent Programming 39

Gadi Taubenfeld © 2014
 Section 3.2.5
Impossibility

v v
1,2 1,2
e1 e2 e1 e2

v1 v2 v1 v2
1 2 2 1

r1 r2 r1 r2
v12 v21 v12 v21
1 2 2 1

2 1
1 2 infinite infinite
path path
p1 is in CS p2 is in CS

Chapter 3 Synchronization Algorithms and Concurrent Programming 40

Gadi Taubenfeld © 2014
 Fault-tolerant Algorithms
Section 3.3

Immediate failures
Continuous failures
Katseff’s fault-tolerant algorithm
Self-Stabilization

Chapter 3 Synchronization Algorithms and Concurrent Programming 41

Gadi Taubenfeld © 2014
 Section 3.3.3

Self-Stabilization

An algorithm is self-stabilizing for a given

property, if it guarantees that after any
transient failure occurs, the property will
eventually hold.

Chapter 3 Synchronization Algorithms and Concurrent Programming 42

Gadi Taubenfeld © 2014
 The One-Bit Algorithm
code of process i , i Î {1 ,..., n}

repeat
b[i] = true; j = 1;
while (b[i] = true) and (j < i) do
if b[j] = true then b[i] = false; await b[j] =false fi
j = j+1
od
until b[i] = true
for j = i+1 to n do await b[j] = false od
critical section
b[i] = false

• not self-stabilizing for deadlock-freedom

• not self-stabilizing for mutual exclusion

Chapter 3 Synchronization Algorithms and Concurrent Programming 43

Gadi Taubenfeld © 2014
 The Self-Stabilizing One-Bit Algorithm
code of process i , i Î {1 ,..., n}

repeat
b[i] = true; j = 1;
while (b[i] = true) and (j < i) do
if b[j] = true then b[i] = false;
repeat
if b[i] = true then b[i] = false fi
until b[j] = false
fi
j = j+1
od
until b[i] = true
for j = i+1 to n do
repeat
if b[i] = true then b[i] = false fi
until b[j] = false
od
critical section
b[i] = false
Chapter 3 Synchronization Algorithms and Concurrent Programming 44
Gadi Taubenfeld © 2014
 Symmetric Algorithms
Section 3.4

Definitions
Symmetric deadlock-free algorithms
Symmetric starvation-free algorithms

Chapter 3 Synchronization Algorithms and Concurrent Programming 45

Gadi Taubenfeld © 2014
 Next Chapter
We have completed covering
solutions to the mutual exclusion
problem using atomic registers.

Next we look at blocking and

non-blocking algorithms using
synchronization primitives
stronger than atomic registers.

-- Gadi

Chapter 3 Synchronization Algorithms and Concurrent Programming 46

Gadi Taubenfeld © 2014