INTRODUCTION TO DISTRIBUTED ~ SYSTEMS 1.1 FUNDAMENTALS OF DISTRIBTED COMPUTING “The process of computation was started from working on a single processor: This uni= processor computing can be termed as centralized computing. As the demand for the increased processing capability grew high, multiprocessor systems came to existence. The advent of multiprocessor systems, led to the development of distributed systems with high degree of scalability and resource sharing, The modern day parallel computing is a subset of distributed computing A distributed system is @ collection of independent computers, interconnected via a ‘network, capable of collaborating on a task. Distributed computing is computing performed in a distributed system. Distributed computing is widely used due to advancements in machines; faster and cheaper networks. In distributed systems, the entire network.will be viewed as a computer” The multiple systems connected to the network will appear as a single system to the user. Thus the distributed systems hide the complexity of the underlying architecture to the/uset Distributed computing is a special version of parallel computing where the processors are in different computers and tasks are distributed to computers over a network. s _ The definition of distributed systems deals with two aspects that y Deals with hardware: The machines linked in a distributed system are autonomous Y Deals with software: A distributed system gives an impression to the users that they ‘are dealing with a single system. : Features of Distributed Systems: * Communication is hidden from users‘They hove stared memory-and chased Th ave « variables1b troduction vo Distrib Diba appa Extent i T Aisuibuted 5 Ftc Disuibuted sofware (siddloware Meri) Operating Network toc sack Fig 1.2: Interaction of layers of network 1.12 Motivation Access to geographically remote data and resourees:As meitioned previously, pen al remote locations, Resources such as eeutalized servers can also be accessed from distant locations. + Enhanced reliability:DS provides enhanced rel distribution of resources at distant locations makes them "The term reliability com resource! service provided by the resoureshould be Tess susceptible for fa of the resource should be conect and consist from system failures ‘The resource sharing and remote sccess performance / costae Scalable: The number of systems operating in a distributed envionment can be increased asthe demand increases Dustrbuted Syuiem 15 12 RELATION TO PARALLEL SYSTEMS Parallel Processing Systems divide the program into multiple segments and proces | cach Ony Menon Asss COMA) ay 0 Unto emer cee (MEA) ‘+ Here, all the processors share the physical memory ia a cenlined mane ith equa access time tall the memory words. ‘+ Bach processor may have a private cache memory: Same mle i flloned for peripheral devices Bi ‘When all the processors Rave equal access to all the peripbenl devices the system is called a symmetric multiprocessor. ‘+ When only one of 2 few processors can access the pepisal devices he stem is called an asymmetric multiprocessor.: F ee |. $ i i i ae foes Efe 2 : BoE Kon-unler Newry Aes del Fi Ina meno fr ol adres pe wich an be ae = fe pee Zi +The ae no ope of NUMA sens NanCoching NUMA SC NEMA, ‘also share CPUs and the address space, luit each processor has = = Metal oter cca oe lated in Sbe number OF prosesso1s, Fee eee - comms nF pbDistributed Systems SPU Memory CFU Mamery CPU Mamery CPU Memony leoherency isthe dizectoy-bsed protocol. ‘The base idea isto associate each ode blocks: a database stating in which cache is When a block of memory is addressed, Tocated,js queried. to know ifthe lock ‘changed respect tothe copy in RAM Fig LGsCache Coherent last eas, the copy in RAM wil be eventually overwritten withthe py preset inthe cache ofthe processor that modified it the relationship berween processors and memory inio a ‘COMA. there iso longer a ome adress and the entire physical adiess considered a ge, single cache urate within the wbole system, from a memory bank to anor, sotding to the reuestofa specific CPU, that requses hat atacod jon te binarylot pocesirs An amy proceso ini s excited on al dams seca an the aoe a1 incapable fopenting aommony dine 12.3 Flynn's Taxonomy Fiyuu's taxono 1 1 a pect elssficeton of parallel eapueran are based om the numberof co icon (sing or al Streams singe or mulipl) aval the acltetre A following: {5 taxonomy based on the numberof instruction streams and data steams are the ‘+ GSISD) Single insiruction, single data ‘+ (MISD) multiple instruction, single data + (SIMD) single instruction, mult (SIMD) multiple instruction, multiple data ‘Single Data stream) ata ‘+ Single Instruction, Single Data (SISD) refers to an Instution Set Architecture in ‘which a single processor (one CPU) executes exacly one instuction steam at 2 time, “+ Ihalso fetches or stores one item of data ata time to operate oa data sored in a single memory unit. + Most of the CPU design is based on the von Neumann achteture and te follow SISD. : 4+ The SISD model is a nompipelined architectue with general-purpose reise, Program Counter (PC) inaction Register (IR), Memory Address Reyisters (MAR) and Memory Data Registers (MDR). Distnbuted System 113 throug the CU. single instruction steam over PUs, Handled + The CU generates the contol signals for all of the PUs and by which executes the same operation on + The SIMD archi ASD (Mettple stew ferent dala streams ure is capable of achieving data level pall 0 (Taaar] FO} tion Set Arcitccr for ps ilfren operations by enseutingSingle [S0 ree | Yon Newnann Singecompuier | May be pipe ing ‘+ A multiprocessor system ie the capably of executing Mi Data (MIMD) programs with natiples CPU alors wilh ferent sot of + Using. the MIMD, each processor in.» mithigrocessor © ifferet set of te insretons independent computer systems cn used the shated memory ina memory poot or buted memory across Heterogeneous network computers ina ~ on ‘The degree of coupling tong a se of modules, whether hardware oF wofovare, ts ‘esed interns of the terdependency ond binding anor homogeneity aig the modutes, = sic contun multiple CPUs that are connsst at oul shared memory duc faster aceess to meniony at power They are+ Some examples of tig coupled mal ‘communicate by message passing ae Loosely Coupled systems: ‘SGI Origin 2000 ‘+ Loosely coupled multiprocessors consist has its own memory and 10 channels +The processors communicate with each other switching + Esch processor may als runa diferent operating system and have is own bus conta loge. bigger and have alow performance con dual nodes in a loosely coupled usually inexpensive. ‘+ They require more power and are more robust and can resist + The extra hardware required to provide communication beiween the individual processors makes them complex and less portable, ‘= Loosely coupled multicomputers without shared memory ar and that ae physically emote, ae teed a distributed 5) ism on specifi system the se of multiple processing elements simultanco ‘are broken down into chic as bees applied 830, 10 the fine Tin) Its expressed asthe ratio of the time T¢ swith m processors Distributed System JIT Pain he amplicon into small amounts of work done ng ta low compunton a comanication ratio, 2. Coarse-grained paraleisn:This bas high computa smunication ratio, greater than fne- and lover tan coarse-grained ined paalelism are best suited upled systems rocesing systems: Systems: The opeiting system henselves running loss coupled ‘0n loosely coupled13 The communications betweon : ‘two main modes: ye liprocessor systems take place through 3 them, Message queuss used by most operating When stared memory mode asdistributed shared memory | posing must execute athe same time may Disinbuted Sytem 1.19 Differences between message passing and shared memory models [oeeses They ean be protected from another by having private address spaces This technigee can be used in This heterogeneous computers, compu Synchronization between processes is Svnchrovization is through locks and though message passing primitives. semaphores, Processés communicating via message Processes communicating thoudh DSM execute with nonoverlapping Hifevimes. Emicieney: All remote data accesses are explicit and” Any particular read ot wpe ay or may therefore the programmer is always E Emulatng messageiassiog on a sharéd memory ‘The shared memory system can be made to address spe ean be partitioned into disjoint pats, can be reserved as the emits for each ondted pir of processes1.20 _tntroduetion to Disribuied Systems fed memory on a message-passing system (SM —+ M ad and verte operations. Each shared location can ted by send als implemented separate process, Wi corresponding. owner pro stedby sending a query message This emulation ‘memory location shared memory ives Fox oisrumuueD communication /\ tom eet ser buffer to the kernel he hemel buffer ono the network. F required because the is invoked, ad neds a storage pl % Buffered: The standard opti n acknowledgement is it for te expected message fiom the sen ling comvon butter o up ommuniction Hocking iF contol retumne 40 the invoking process the Distrwed Sytem — 1.21 control setunstac othe invoking process immediatly ‘+ For anon-blocking Receive, contol returns tothe process cien before the ditamay hhave arsved from he sender. ‘ ‘Synchronous +A Send or a Receive primitive is synehronous if both te Send) ad Rete) fandsake with each ctr, Send primitive completes only afer the invoking processor responding Receive primitive hat allo been ivoked anda been completed. Receive primitive completes when the data to be recived i copied into the receiver's user buffer Asynchronous ‘+A Send primitive is said to be asynehronous, if contol returns tack to the iming process aferDistributed Sp riled Stems omplets when the data Has been copied out of tne completion may be necessary if the user wants to touse ata wat nt 20 be implemented in four modes + Blocking synchronous + Non: blocking synchronous + Blocking asynchronous + None oct ronous Four modes of send operation ‘Blocking synchronous Send: * EIQ Bs copied trom te wer baer to he Kemal ite dis ho + After the date ja-blacking synchronous sens iodo the reciver'sssscrs baler and @ blocking receive vement bok fo the sender eases conte "ation and completes the Sond ‘Non-blockinz synchronous Send: locking synchronous send ard locking receive joking process 2s soa asthe copy of das fm the use = is iitinted| * A paramcicr in the non-blocking eal eso gets stv ‘user process can Ister cheek for x cmplsnon of the handte ofa location athe ergs sshd operation. * The location gets posted afer an acknowledgemeatfeuins tion the ic + The wer process can keep checking for the completion o synchronous Send by testing the returned handle, or ¢an invone the bloc S ‘operation on the retuned handle ig asynchronous Send: bs wt he a expecta aes an is writen in the spc Gite ie an Bee = iS returned to the wser process. = user process that invokes the Send is Booked wi the data cae orn he = butler tothe eral butler as the Kernel to reser the call c=8 can er check forthe ception es that imokes the Send is blocked ua Her tothe Ketnelbufler is initiated es the Lema afl the expect dat, butter, The use process can heh far the & 'y invoking the Wat operation on the retired hanDitferences between Bi PC of one process fo tat of another process thvolzh coopera foneach process. ‘The pimary goal of the Message Pasting Interface is to provide a widely wed standard for whiting message passing rosrams ‘The commonalities between RMI aad RPC ae 35 fellows: They both support programming with interfaces. < ‘They ae consiucted 08 tp of request-tply ts, + Parallel Virtual Machine (PVM): It is a software foo! for parallel networking of y op of requs-eply protocot ‘computers Its designed to allow a network of heterogencous Unix andlor Windows They both aera sinaar evel of transparency. ee rae ebeeaaiee paral procemeee + Common Object Request Broker Architecture (CORBA): CORBA describes a *+ Remote Procedare Call (RPC): The Remote Procedure Call (RPC) is « common ‘messaging mechanism by which object distibuicd over 6 wees ea ‘model of request reply protocol. In RPC, the procedure need not exist in the sine # ser inespetve of the platform and language used to develop those © é Is 3S ies 2] ; + nouppertound ons tne ken bya yaces 5 = opoeamDistribwed Sen 127umber of higher evel objec sere functions we channels, access contol,” key management on. authori ‘eure gop Waagement a SOME ributed Systems. =~ " jerface (API) and_transparency: The wer e pope cach pce trasnaeng bites diferences in da epesention Le? z ides rences inion y jill inifor ses M &. fe ofthe system across space involves the role of time jon for consistency. This ean be done with extra effon in 2 cootinaed res of concureney also involves the timedimension, 25 munication speed of threads may ary aot tion mechanisms forthe stud process fit concuret ber process rechanisns also involveresource mamsement aad jure transparency: system being reliable and faultsolernt “The algorithms, data and servicesmist be asi Ties such os replicaioneaching and eache management, ip toachieveseslabiy rout automata model and the Texamples of snodels that provide Temporal Lozic of different degrees of infrastructure.‘@ Monitoring distributed events and predicates porithms for monitoring sch predicates + An important paradigm for monitoring distributed ‘wherein streams of relevant events reported fom ‘collectively to detect predicates + The specification of such predicates uses physical > Distributed program design and verification tol + _Methodically designed and venfbly cor ‘heoterhead of software design. debugging. a big challenge F Debugging distributed programs ‘+ Fastaccess to data and ote resources buted system n and aching, prefetching of objects have to be dome Jn and ravigationon the web ate important functions inthe operation Shared memory abstraction ory Is easier to implement since it does not involve manasing the asks is done bythe middleware by mestaze passing lusion Message passing the fementing distributed mutual exclusion sucha wa that, fon the goncuereneythe comeetiy perform a proc to produce a single opt stabilizing algorithms require some variables ofthe state and do extra work. Checkp and recovery algorithms: Checkpointing is peri recording the current state on secondary storage so that, in case ofa fl nlite computation is not lost but can be recovered flom oneof the rece checkpoints. Checkpointing ina distributed envi c’cheekpoints atthe different processes are may become useless because theyare inconsis processes Failure detectors: The asynchronous distributed do not ha ‘message transmission time. This makes the message passing very di the receiver do not Know the wailing time. Failure detectors prok Suspect another process 2s havingfailed and then converge on a determination of the upidown statis of thesuspected process. Load balancing and rede he he objective of lal tlaning is 10 gain ihr tough, and Pa ea ead balancing maybe necesay Because of a valet of {Rot suchas high ewer tli oF {ees bees orig amputation balancing: wve_a bound on the ‘etworkeonnertion ‘some forms of lod Y Performance User perceived latency in dstibted systems must be reduced. The common issues ia Appropriate meties must be defined for measuring the performance of iste algoritims and its implementation. methodvtook: ‘The distributed system is a complexenity Tmethodalogy and tools must be developed for meastring the the environments have their own challenges:he proces ig and network-cenrie ole Rew set of challenges, such a5 position en designing adstbuted system. Processors are embedded in the environment opeform ackyround, th recent advancements Sperling in wi nputing over an application Jay sors are a same level sat mechanism for agaregating. lage volumes of published Information and Fi 5 per the users subserption fie, ‘fers to a mechanism tat categorizes the information based ton ovedlap each oer. ‘+ Multimedia data introduces special iste because of is large sie. > Distributed agents oA software processes or sometimes abots that move sound the syst ic tasks for which they are programmed. + Agents collect and process information and can exchange such infomation wth ‘otber agents z + Challenges in disibutd ayemt systems include cooniaton wectassns among agents, contoling the mobility of the agentsthet software desien and ss amount of data to dete pater and tends useful information, ‘+The mining can te done by app toads repository’ abuse and artificial itligence echrses ces. For instance idle CPU eyes of ‘be available to oters 8 Hamework for implementing quality ofW order Ifsay A Bf A toppers before B. ADB. pi produces a sequence of event cre es produced by Here hare states prod 1 eccur on same process and A occurs before B, > 6) for any message,ondeting is happen before 7 9°90.iF a and bare events inthe same pe taking place in interleaved fashion or multi CPU machines 1.9 MODELS OF COMMUNICATION NETWORK cach chanel acts 35a FIFO message qucuc chanel acts ikea set i which a sender process adds esever removes messages in random onde. (CO}: It follows Lamport aw. between he tee modes i given by CO < FIFO c N-FIFO, 1.20 GLOBAL STATE as eee ted Snapshot represents a state in which the distributed system might have in-A snapshot ofthe system sa single configuration of the system. EE Re ee Concureton ef ke stem | ute Snapsot should refet a consistent state. global state is con observed by an extemal obsener. For a successful Global 8 we have recorded that a process Phas received a message from a process Q. Should have also recorded that process Q had actully send tat 4 snapshot will conta the reconing of messages that have been ceived but never sen hat P has not reecived) is an be graphically represented by what i called a cut A been recorded foreach process “Trdtwdual state € esch pron 4 padivad uch140 _Ihiraducton o Dot of the process. We denote by sf the state of tlobal she § event procesed by p inthe d retains Consistent states: The slates should not violate causility. Such sites are called consistent glotal ses and are meaningful global states, Inconsistent global states: They afe not meaningful in the sense that a system can never be in an inconsistent state 1.10 CUTS OF A DISTRIBUTED COMPUTATION ribted A eat isa set of cut events one per node, each of which captures the sate of the node feat cu: A consistent gloal state corresponds toa cut in which every message PAST ofthe eu was Sent in the PAST of that cut cut Aca is inconsistent if'a message crosses the cul fom the FUTURE In distibued computation, an eve veh tate, e and all the information avail wont =fe,1¥e, Hee) “The term mapas() denotes the latest event of process py that has affected e, This ‘will tways be a message sent event, etteatey)mibonete) S that corresponds fo aeut in which every message received in the PAST of the ext was set inthe PAST ofthat eut ‘The future ofan event ¢, denoted by Future(s)) contains all the eventse, that are oAfi] 4 UAL < URL 2, Strong conssteacy {fo compare wo he system of vector elocks is strongly thus, by examining the imestamp of wo evens, We ean determine i the evens are causally related. 2 execute RI = > an iiss i W] denctes he numberof events exec by - rocess ptt eau e. ‘3. deliver the messegem § process py ly precede Fig 126: Featution of veetorseale‘This drastically reduces (O(0) storage. This is du ‘eachother process, the state This also requires FIFO message pa ‘pon the guarantee of knowing have changed fo vi.¥2 is then proses pi ‘message size, communication bandwidth and buffer equitements. message fo process p + LUI...0) CLast Update’: LLUJj) indicates the value of vt, i] when process pls update Disiribued Sytem 149 rte Sten 1.09 Fig 1.27: Vector clocks progress in Singhal-Kshemkalyani technique + Fowler-Zwaenepoe! direct-dependency technique! ~ This teenigue further reduces the message size by ony sending he single lock value ofthe sending proces witha message full view ofall dependenci search must be made ross pr = Bach process pinantains a dependency vector D, Intl, (]= 0 for jet. updated as fol Wheneveran event occu at p such that, ‘When a process pi sends a message to process pit pisgybacks the ‘pdatedvalue of Di in the message 3. When p, receives a message fron p, wath pigeybacked vabbe & updates its dependency vector as Follows: DU= max{D Gl.180 trreat ‘the direct dependency of py on pe Because ofthis messge. Process py sends a messaze to proces pz pigaytoching salar to indicate the ducet dependency of ps on p; because of this message “This technique fesulls in considerable saving inthe east: only one scalar is pigeybacked on every message. LId PHYSICAL CLOCK SYNCHRONIZATION: NEWTWORK TIME PROTOCOL (STP) 1 nced clock synchronization, as they work under 3 common 9 not follow common clock: each system functions based 7 notion of time The tine in distnbuted 5) between two evens that happened on diferent machines in sing of events that happened on different machines in the Disinbuted Syteor 151 cess of ensuring tat physically dsribated the difference between the time reported by aclockaod the the elock C, is given by Cut Theuset of eock Ca even by CA CA) ne skew ofa clock the difference in the frequencies ofthe edekand the Jock. The skew of a elock C, relative to clock Cy 3 timet Cy (0> Co Primary server a the rot sypcondzes wth ers, which at 38 a teckep fo psmany 3c syncivonizabonsubpct which haste centsREVIEW QUESTIONS PART-A collection of independent compe , capable of collaborating on a task Distributed ina distibuted system 2. What are the aspects of distributed systems? Cocks. € tion of di buted systems dal with wo aspects that: rdvare: The machines linked ina dstibuted system ae autonomous Deals with softwar xy ae dealing wi istibuted system gives an impression to th uses th singe system. he features of Distributed Systems. ‘Communication i hidden ftom users Te + Ani onsite in rif ad case ay A righ ders ofseaiy 1" A distributed system is fonctionally equivalent to the systems of whic itis ‘composed (Py Resource sharing is possible in distributed systems. i t \ et #7 Distributed systems act as fault tolerant systems 1 Bahanced pertormasce Fig 130.a) Offset and delay esfimation ee between processes from same server © No common clo a Let T1, 12, T3, T4 be the values ofthe four mos recent timestamps. The clocks A and ime speed, Leta T1=T3 and b= 12 ~ T4. If the network delay difference from A to B and from B to A, ele k delayé of B relative to A al time T4 ae approximately given by the > Geographical isolation —Fallowing 0-28, scab 6 Each NTP message includes the latest thee timestamps 71, 72, and T3, determined upon aval © Latency Pglyons taxonomy. {SISD) single instruction, singe data (MIS) multiple instruction single dita IMD) stil instraton, slp data (Main) ipl insrecton, tip data 11. Define degree of oupling. ‘The degsee of coupling amone a set of modules. measured in terms of the ftedependency and bit module WI ther hardware oF software, is dior homogeneity among the are Tightly Coupled systems? ly coupled maliprocessor systems contain multiple CPUs that are connecied bus evel with both eel as well as central shared memory Tighily coupled systems perform beter. due to faster aecessto memory and ‘oinmynication ad are physically smaller and use less power. They aft ‘conomically costlier. led multiprocessors with UMA shared memory may be either switeh- YU Uracompurct, RP3} or bus-ba + Some examples of that comm Encore) NUMA shared metaory of igin 2000, = 13, What are Loosely Coupled systems? + Loosely coupled maluperessors consist [osessorhas its own memory and 10 ehantsts Hoty where each Ns communicate with cach other via message passing or ion switching a ‘iconnected By a LAN of Myrinet iques for decomposing a task into subtasks that he Fsks that arise when the program executes larity? Granalarty or pain size is a measure of the amount of work of computation that i performed by that task. 2. Coarse-grined parallelism tation to communication ratio. 3. Mediumgrained: Here the task tion time grate Un fine _grined parallelism and lower than coarse-grained parallelism, 17, What are blocking primitives? rnin commands wat forthe message to be delnered. The excaton ofthe PRES Gecked Te. Sending process must wait afer 4 snd ll ded Sytem 1.57 ive operation has been completed The procesing ‘when the data to be rected ts cop int Be What is Asynehroaeus primitive? [A Seod primitive i said to be asynchronous, if control returns back to He imeking ‘process aller the data item to be sent has been copied out ofthe user-pecifed baer It does not make sens to define asynchronous Receive primitives. Implemetins mo blocking operations are tricky: For non-blocking primitives, a retam arenes 2 ‘the primitive call retusa system-zenerated handle which can be later Ged w check ‘the status of completion ofthe cll 21, What are the modes fsead and receive primitives? + Blocking synchroaoes + Non- blocking synchronous + Blocking asynetrooos + Non- blocking asynchronous er. Then contol is returned tothe user POsess.heterogeneous buted parallel processo 26, Define Remote Procedure Call Disrbwed Sytem 1.59 TC oe oe and os at | eh ogee arc syeote stones Gate hey Sst RM tan employ of fests ang te nection fs he Ket to te scout compra Instead of pusog'» prt eal Rl pose'a eee w Oe ober [intot mci tare cng aid | sommionaliies berween RAM and RPC. ‘+ “They both support programming with interfaces ‘+ Tiny ave eonsuctad ua top oF request-eply protoolsDistributed System Lat Dre Sytem sc site coesponds to ut n which every ms was sent in the PAST of that cut ers nt fa message crosses the cut from the FUTURE. and causal relationships of Logical Clock ‘A logical clock is a component for ca Sequential and causal connections in Aspetsed framework Alogi lions lta oeinrca | ros tom een poem 36. Differenciate between logical and physical concurrency ty feta etnayt Peay ks consists of atime domain T and a logical clock C. Elements dered set over a relation < This relation is usually called the sal precedence counters start with alee 0 37. Define snapshot of system. its counter fr each event. that proces. eternal event, message sending, Distributed Snapshot represents a sise in which the distibuted sy ‘been in. A snapshot of the system isa single configuration ofthe system sends message it incles its (incremented) counter Value with th tecpient is updated to he received message, and then incremented ga message the ‘ana the tin ‘A cut isa set of et events, one per node, each of which captures state ofthe node ‘on shich it occurs.1.62. Introduction to Distributed Systems ronization, Clock synel ization is the process of ensuring that physically distributed processors have a common notion of time. PART-B. 2 1. Explain in detail about types of multiprocessor systems, Describe the various topologies, ae Brief about Flynn’s classification. Classify systems based on granularity: EGE Explain about shared memory Elaborate about messaye'passing system in deta 7. Describe the various primitives for distii ted d communication: s S2-Bxplain synchronous and asynchronous exeentions s 9. Brief abou te design issues and challenges in dist 10. Explain the model of distributes execution: AL. Write in det about global state. s