Distributed Shared Memory

Memory Coherence
• si memory operations by Pi
• (s1 + s2 + . . . sn )!/(s1!s2! . . . sn !) possible
interleavings
• Memory coherence model
• which interleavings are permitted
• Read returns the value written by the most recent
Write ”Most recent”
• Write - ambiguous with replicas and concurrent
accesses
• DSM consistency model
• a contract between DSM system and application
programmer
2
Strict Consistency/Linearizability/Atomic Consistency

A Read
 should return the most recent value written, per a
global time axis.

For operations that overlap per the global time axis, the
following must hold.
 All operations appear to be atomic and sequentially
executed.
 All processors see the same order of events,
equivalent to the global time ordering of non-
overlapping events.
Strict Consistency / Linearizability: Examples
Write(x,4) Read(y,2)
1P
P2 Write(y,2) Read(x,0)
(a) Sequentially consistent but not linearizable

P Write(x,4) Read(y,2)
1
P2
Write(y,2) Read(x,4)
(b) Sequentially consistent and linearizable
P1
P2 Write(x,4) Read(y,0)

Initial valuesWrite(y,2)
are zero. (a),(c) not linearizable. Read(x,0)
(b) is linearizable
(c) Not sequentially consistent (and hence not linearizable
Linearlzability: Implementation

●Simulating global time axis is

expensive.
●Assume full replication, and total
order broadcast support.

5
Sequential Consistency .
• The result of any execution - the same as
if all operations of the processors were

“ executed in some sequential order.

• The operations of each individual
processor - in the local program order.
• Any interleaving of the operations from the
different processors – possible
• all processors must see the same
interleaving
• Even if two operations from different
processors (on the same or different
variables) do not overlap in a global time
scale, they may appear in reverse order in
the common sequential order seen by all.
6
Sequential Consistency ●Only Writes participate in total order BCs. Reads
do not because:
● all consecutive operations by the same processor
are ordered in that same order (no pipelining),

“
and
● Read operations by different processors are
independent of each other; to be ordered
only with respect to the Write operations.
● Direct simplification of the
LIN algorithm.
●Reads execute atomically.

●Not so for Writes.

● Suitable for Read intensive

programs.

7
Sequential Consistency using Local Reads
●(shared var)
●int: x ;

(1)When the Memory Manager at Pi receives a

“ Read or Write from application:

(1a) case Read: return value from local replica;
(1b) case Write(x,val): total order broadcasti (Write(x,val))
to all processors including itself.

(2)When the Memory Manager at Pi receives a total order

broadcastj (Write, x, val) from networ
(2a) write val to local replica of x ;
(2b) if i = j then return ack to application

8
Sequential Consistency using Local Writes

9
Causal Consistency
• In SC, all Write ops - seen in common order.
• For causal consistency - causally
related Writes should be seen in common order
Causal relation for shared memory systems
• At a processor, local order of events is the causal order
• A Write causally precedes Read issued by another
processor if the Read returns the value written by the
Write.
• The transitive closure of the above two orders is the causal
order

10
Examples for sequential consistency (SC),causal consistency (CC), and PRAM consistency.

11
Pipelined RAM or Processor Consistency
●Only Write ops issued by the same processor are
seen by others in the order they were issued,
●but Writes from different processors may be seen by
other processors in different orders
●can be implemented by FIFO broadcast
●can exhibit counter-intuitive behavior

12
Slow Memory
●Only Write operations issued by the same processor and to the
same memory location must be seen by others in that order.

13
Hierarchy of Consistency Models

14
Synchronization-based Consistency Models: Weak Consistency
• Consistency conditions apply only to special ”synchronization”
instructions,
• e.g., barrier synchronization
• Non-sync statements may be executed in any order by various
processors.
• e.g.,weak consistency, release consistency, entry consistency
Weak consistency:
• All Writes propagated to other processes
• all Writes done elsewhere brought locally, at a sync instruction.
•Accesses to sync variables - sequentially consistent
•Access to sync variable not permitted unless all Writes elsewhere have
completed
• No data access allowed until all previous synchronization variable accesses
have been performed
• Drawback
 cannot tell whether beginning access to shared variables (enter CS),
or finished access to shared variables (exit CS) 15
 Synchronization based Consistency Models: Release
Consistency and Entry Consistency

Release
• Acquire indicates CS is to be entered. Hence all Writes from other processors
should be locally reflected at this instruction
• Release indicates access to CS is being completed. Hence, all Updates made locally
should be propagated to the replicas at other processors.
• Acquire and Release can be defined on a subset of the variables.

• If no CS semantics are used, then Acquire and Release act as barrier

synchronization variables.
• Lazy release consistency: propagate updates on-demand, not the PRAM way.

16
Synchronization based Consistency Models: Release
Consistency and Entry Consistency

Entry
• Each ordinary shared variable is associated with
a synchronization variable
• (e.g., lock, barrier)
• For Acquire /Release on a synchronization
variable, access to only those ordinary variables
guarded by the synchronization variables is
performed.

17