716 OIstrIIutod IrogrannIng

1lis ¦ricl iniroonciion lcavcs oni many issncs sncl as sccnriiy. naming. rcsonrcc
managcmcni. ano ¦niloing lanli iolcrancc a¦siraciions. Lni ii givcs a gooo
ovcrvicw ol ilc gcncral issncs in ilc arca ol oisiri¦nico programming.
Structuro oI tLo cLaµtor
1lis clapicr consisis ol ilc lollowing paris:
Scciions 11.1 ano 11.2 sci ilc siagc ¦y giving a iaxonomy ol oisiri¦nico
sysicms ano ¦y cxplaining onr oisiri¦nico compniaiion moocl.
Scciions 11.811.6 slow low io program in ilis oisiri¦niion moocl. Wc
lanolc siaic scparaicly ¦ccansc ii involvcs morc soplisiicaico ano cxpcnsivc
oisiri¦niion proiocols. Wc ilcn cxplain ilc conccpi ol n·ìu·:| nun:·n·.
wlicl is imporiani lor pcrlormancc rcasons. Iinally. wc slow somc common
oisiri¦nico programming paiicrns.
Scciion 11.7 cxplains ilc oisiri¦nico proiocols in morc ociail. Ii singlcs oni
iwo pariicnlarly inicrcsiing proiocols. ilc mo¦ilc siaic proiocol ano ilc
oisiri¦nico ¦inoing proiocol.
Scciion 11.8 iniroonccs pariial lailnrc. Ii cxplains ano moiivaics ilc iwo
lailnrcs wc ocicci. pcrmancni proccss lailnrc ano icmporary nciwork inac-
iiviiy. Ii givcs somc simplc programming icclniqncs inclnoing an a¦sirac-
iion io crcaic rcsilicni scrvcr o¦¦ccis.
oisiri¦nico applicaiions.
Scciion 11.10 snmmarizcs ilc clapicr ¦y giving a mcilooology low io ¦nilo
oisiri¦nico applicaiions.
11.1 Taxonony oI dIstrIIutod systons
1lis clapicr is mainly a¦oni a qniic gcncral kino ol oisiri¦nico sysicm. ilc opcn
colla¦oraiivc sysicm. 1lc icclniqncs wc givc can also ¦c nsco lor oilcr kinos ol
oisiri¦nico sysicm. sncl as clnsicr compniing. 1o cxplain wly ilis is so. wc givc
slows lonr iypcs ol oisiri¦nico sysicm. Ior cacl iypc. ilcrc is a simplc oiagram io
illnsiraic ii. In ilcsc oiagrams. circlcs arc proccssors or compnicrs. ilc rccianglc
siaris wiil a |n:·n-n·n·:j n|ì:¡:····:. wlicl is a compnicr ilai consisis
ol scvcral proccssors aiiaclco io a mcmory ilai is slarco ¦ciwccn all ol ilcm.
Copyiighl c 2001-8 Ly I. Van Ioy ano S. Haiioi. All iighls iosoivoo.
11.1 Taxonony oI dIstrIIutod systons 717
(partial Iailure)
?
Distributed memory
multiprocessor
Shared memory
multiprocessor
multiprocessor
Distributed memory
with partial Iailure
Aaa aistribution
Aaa partial failure
Aaa openness
Collaboration
(Internet computing)
Open distributed system
(naming and security)
High performance
(cluster computing)
Iignrc 11.1: A simplc iaxonomy ol oisiri¦nico sysicms
io wriic a mcmory ccll ano anoilcr io rcao ii. Cooroinaiing ilc proccssors. so
Small slarco-mcmory mnliiproccssors wiil onc io cigli proccssors arc com-
mooiiy iicms. Largcr scala¦lc slarco-mcmory caclc-colcrcni mnliiproccssors
arc also availa¦lc ¦ni arc rclaiivcly cxpcnsivc. A morc popnlar solniion is io
conncci a sci ol inocpcnocni compnicrs ilrongl ilcir I,O clanncls. Anoilcr
1lc nciwork can ¦c implcmcnico as a slarco ¦ns (similar io Lilcrnci) or ¦c
poini-io-poini (scparaicly conncciing pairs ol proccssors). Ii can ¦c cnsiom or
nsc sianoaro LAN (local-arca nciwork) icclnology. All sncl maclincs arc nsnal-
ly callco ·|ì·: or n:ì::hì·n-n·n·:j n|ì:¡:····:. 1lcy nsnally can lavc
pariial lailnrc. i.c.. wlcrc onc proccssor lails wlilc ilc oilcrs coniinnc. In ilc
larowarc ano soliwarc ilc clnsicr can kccp rnnning. al¦cii wiil ocgraoco pcrlor-
mancc. cvcn il somc proccssors arc lailco. 1lai is. ilc pro¦a¦iliiy ilai ilc clnsicr
coniinncs io provioc iis scrvicc is closc io 1 cvcn il pari ol ilc clnsicr is lailco.
1lis propcriy is callco |:j| nn:|nh:|:ìj. A clnsicr wiil ilc propcr larowarc ano
soliwarc com¦incs ligl pcrlormancc wiil ligl availa¦iliiy.
In ilc lasi sicp. ilc compnicrs arc connccico ilrongl a wioc-arca nciwork
(WAN) sncl as ilc Inicrnci. 1lis aoos ·¡·nn·. in wlicl inocpcnocni compn-
pniing ano ilc worlo ol colla¦oraiivc compniing. In aooiiion io pariial lailnrc.
opcnncss iniroonccs iwo ncw issncs: naming ano sccnriiy. Ann:nj is low compn-
Copyiighl c 2001-8 Ly I. Van Ioy ano S. Haiioi. All iighls iosoivoo.
71S OIstrIIutod IrogrannIng


Iignrc 11.2: 1lc oisiri¦nico compniaiion moocl
pari ol ilc sysicm callco ilc nnn· ·:·:. o··::ìj is low compniaiions or
compnicrs proicci ilcmsclvcs lrom cacl oilcr.
11.2 TLo dIstrIIutIon nodoI
Wc consiocr a compniaiion moocl wiil ¦oil poris ano cclls. com¦ining ilc moo-
slarco ¦ciwccn Oz proccsscs [71. 107. 72. 201. 78[. Il oisiri¦niion is oisrcgaroco
(i.c.. wc oo noi carc low ilc compniaiion is sprcao ovcr proccsscs) ano ilcrc
arc no lailnrcs. ilcn ilc compniaiion moocl ol ilc langnagc is ilc samc as il ii
cxccnics in onc proccss.
Wc assnmc ilai any proccss can lolo a rclcrcncc io a langnagc cniiiy on any
oilcr proccss. Conccpinally. ilcrc is a singlc glo¦al compniaiion moocl ilai cn-
compasscs all rnnning Mozari proccsscs ano Mozari oaia worlo-wioc (cvcn ilosc
programs ilai arc noi connccico iogcilcr!). 1lc glo¦al siorc is ilc nnion ol all
ilc local siorcs. In ilc cnrrcni implcmcniaiion. connccico Mozari proccsscs pri-
scni ¦ciwccn proccsscs iravcl ilrongl 1CI.
Iignrc 11.2 slows ilc compniaiion moocl. 1o aoo oisiri¦niion io ilis glo¦al
vicw. ilc ioca is ilai cacl langnagc cniiiy las a oisiri¦niion ¦clavior. wlicl oc-
cacl langnagc cniiiy wiil a proccss. wlicl is ilc lomc proccss¨ ol ilai cniiiy. Ii
is ilc proccss ilai cooroinaics ilc oisiri¦niion ¦clavior ol ilc cniiiy. 1ypically.

Wc will somciimcs
nsc ilc plrasc ··n:ì·n·j ¡:·ì···| io ocscri¦c ilc oisiri¦niion ¦clavior ol an cn-

In Mozail, lho cooioinalion of an onlily can Lo oxplicilly movoo fiom ono piocoss lo
anolhoi. 1his issuo will nol Lo oiscussoo in lhis inliooucloiy chaploi.
Copyiighl c 2001-8 Ly I. Van Ioy ano S. Haiioi. All iighls iosoivoo.
11.2 TLo dIstrIIutIon nodoI 719
iiiy. 1lc oisiri¦niion ¦clavior is implcmcnico ¦y cxclanging mcssagcs ¦ciwccn
Mozari proccsscs.
oisiingnisl ¦ciwccn ìnì·]|. ìnì·|·. ano :nj|·-n:jnn·nì langnagc cniiiics.
Siaiclnl cniiiics (ilrcaos. cclls. poris. o¦¦ccis) lavc an inicrnal ìnì·. 1lc
oisiri¦niion ¦clavior las io ¦c carclnl io mainiain a glo¦ally colcrcni vicw
ilai arc possi¦lc. 1lc simplcsi kino ol ¦clavior is io makc ilcm ìnì:·nn:j.
An opcraiion on a siaiionary cniiiy will iravcrsc ilc nciwork lrom ilc
invoking proccss ano ¦c pcrlormco on ilc cniiiy`s lomc proccss. Oilcr
kinos ol ¦clavior arc possi¦lc.
oisiri¦nico rclcrcnccs io ilc samc valnc. 1lis opcraiion is cooroinaico
lrom ilc proccss on wlicl ilc varia¦lc is crcaico.
Siaiclcss cniiiics. i.c.. valncs (procconrcs. lnnciions. rccoros. classcs. lnnc-
iors) oo noi ncco a proccss annoiaiion ¦ccansc ilcy arc consianis. 1lcy
can ¦c ··¡:·n ¦ciwccn proccsscs.
Iignrc 11.8 slows a sci ol proccsscs wiil localizco ilrcaos. cclls. ano nn¦onno
oisiri¦nico ¦clavior. Lni ilis ¦clavior can ¦c clangco wiiloni clanging ilc
langnagc scmaniics. Ior cxamplc. a rcmoic opcraiion on a ccll conlo lorcc ilc
ccll io migraic io ilc calling proccss. ano ilcrcalicr pcrlorm ilc opcraiion locally.
Ior all ocrivco cniiiics cxccpi lor poris. ilc oisiri¦nico ¦claviors ol ilc oc-
is asynclronons (scc Scciion ò.1). 1lis oisiri¦nico ¦clavior oocs noi lollow lrom
ilai ol a ccll. 1lis mcans ilai poris arc ¦asic cniiiics in ilc oisiri¦niion moocl.
¦nsi likc cclls.
¦ni ii las onc limiiaiion: pariial lailnrcs arc noi iakcn inio acconni. In Scc-
iion 11.8 wc will cxicno ilc ¦asic moocl io ovcrcomc ilis limiiaiion.
iri¦nico ¦claviors. Ior cxamplc mo¦ilc¨ o¦¦ccis (also known as caclco¨ o¦-
¦ccis) movc io ilc proccss ilai is nsing ilcm. 1lcsc o¦¦ccis lavc ilc samc
inning nciwork pcrlormancc.
Copyiighl c 2001-8 Ly I. Van Ioy ano S. Haiioi. All iighls iosoivoo.
720 OIstrIIutod IrogrannIng
Process b
...
Sn
S2
S3
S1
X
Threads
Cell
c1:X
DataIlow variable
c2:Z Z
W
Sx
Y
Process a Process n
Iignrc 11.8: Iroccss-oricnico vicw ol ilc oisiri¦niion moocl
11.3 OIstrIIutIon oI docIaratIvo data
Lci ns slow low io program wiil ilc oisiri¦niion moocl. In ilis scciion wc slow
low oisiri¦niion works lor ilc occlaraiivc sn¦sci ol ilc siaiclnl concnrrcni moocl.
11.3.1 Oµon dIstrIIutIon and gIoIaI nanIng
Wc say a oisiri¦nico compniaiion is ·¡·n il a proccss can conncci inocpcnocnily
wiil oilcr proccsscs rnnning a oisiri¦nico compniaiion ai rnn iimc. wiiloni
ncccssarily knowing ¦clorclano wlicl proccss ii may conncci wiil nor ilc iypc
ol inlormaiion ii may cxclangc. A oisiri¦nico compniaiion is ·|··n il ii is
arrangco so ilai a singlc proccss siaris ano ilcn spawns oilcr proccsscs on varions
compnicrs ii las acccss io. Wc will ialk a¦oni closco oisiri¦niion laicr.
An imporiani issnc in opcn oisiri¦nico compniing is nnn:nj. How oo in-
ocpcnocni compniaiions avoio conlnsion wlcn commnnicaiing wiil cacl oilcr´
1lcy oo so ¦y nsing glo¦ally-nniqnc namcs lor ilings. Ior cxamplc. insicao
ol nsing prini rcprcscniaiions (claracicr sirings) io namc procconrcs. poris. or
o¦¦ccis. wc nsc glo¦ally-nniqnc namcs insicao. 1lc nniqncncss slonlo ¦c gnar-
anicco ¦y ilc sysicm. 1lcrc arc many possi¦lc ways io namc cniiiics:
HoIoroncos. A rclcrcncc is an nnlorgca¦lc mcans io acccss any langnagc
cniiiy. 1o programs. a rclcrcncc is iransparcni. i.c.. ii is ocrclcrcncco wlcn
nccoco io acccss ilc cniiiy. Iclcrcnccs can ¦c local. io an cniiiy on ilc
cnrrcni proccss. or rcmoic. io an cniiiy on a rcmoic proccss. Ior cxamplc.
a ilrcao can rclcrcncc an cniiiy ilai is localizco on anoilcr proccss. 1lc
langnagc oocs noi oisiingnisl local lrom rcmoic rclcrcnccs.
^anos. A namc is an nnlorgca¦lc consiani ilai is nsco io implcmcni
ano anilcniicaiion a¦iliiics (scc Scciions 8.7.ò ano 6.4). All langnagc cn-
iiiics wiil iokcn cqnaliiy. c.g.. o¦¦ccis. classcs. procconrcs. lnnciors. cic..
Copyiighl c 2001-8 Ly I. Van Ioy ano S. Haiioi. All iighls iosoivoo.
11.3 OIstrIIutIon oI docIaratIvo data 721
implcmcni ilcir iocniiiy ¦y mcans ol a namc cm¦cooco insioc ilcm (scc
Clapicr 18).
TIckots. A ì:·|·ì. in ilc icrminology ol ilis clapicr. is a glo¦al mcans io
acccss any langnagc cniiiy. A iickci is similar io a rclcrcncc. cxccpi ilai ii is
valio anywlcrc inclnoing onisioc a Mozari proccss. Ii is rcprcscnico ¦y an
ASCII siring. ii is cxpliciily crcaico ano ocrclcrcncco. ano ii is lorgca¦lc. A
compniaiion can gci a rclcrcncc io an inocpcnocni compniaiion ¦y gciiing
a iickci lrom ilai compniaiion. 1lc iickci is commnnicaico nsing any
commnnicaiion proiocol ¦ciwccn ilc proccsscs (c.g.. 1CI. II. SM1I. cic.)
or ¦ciwccn ilc nscrs ol ilcsc proccsscs (c.g.. sncakcrnci. iclcplonc. IosiIi
noics. cic.). Usnally. ilcsc proiocols can only pass simplc oaiaiypcs. noi
ar¦iirary langnagc rclcrcnccs. Lni in almosi all cascs ilcy snppori passing
inlormaiion cooco in ASCII lorm. Il ilcy oo. ilcn ilcy can pass a iickci.
lHLs {lnIIorn Hosourco Locators). A UIL is a glo¦al rclcrcncc io
ilai maclinc. UILs arc nsco io cxclangc pcrsisicni inlormaiion ¦ciwccn
UIL.
Wiilin a oisiri¦nico compniaiion. all ilcsc lonr kinos ol namcs can ¦c passco ¦c-
iwccn proccsscs. Iclcrcnccs ano namcs arc pnrc namcs. i.c.. ilcy oo noi cxpliciily
cncooc any inlormaiion oilcr ilan ¦cing nniqnc. 1lcy can ¦c nsco only insioc a
oisiri¦nico compniaiion. 1ickcis ano UILs arc impnrc namcs sincc ilcy cxplic-
iily cncooc ilc inlormaiion nccoco io ocrclcrcncc ilcmilcy arc ASCII sirings
ano can ¦c rcao as sncl. Sincc ilcy arc cncooco in ASCII. ilcy can ¦c nsco
¦oil insioc ano onisioc a oisiri¦nico compniaiion. In onr casc wc will conncci
ì:·|·ì.
TLo Connection noduIo
1ickcis arc crcaico ano nsco wiil ilc Connection moonlc. 1lis moonlc las
ilrcc ¦asic opcraiions:
¡Connection.offer X ?T¦ crcaics a iickci T lor any rclcrcncc X. 1lc
iickci can ¦c iakcn ¦nsi oncc. Aiicmpiing io iakc a iickci morc ilan oncc
will raisc an cxccpiion.
¡Connection.offerUnlimited X ?T¦ crcaics a iickci T lor any rclcrcncc
X. 1lc iickci can ¦c iakcn any nnm¦cr ol iimcs.
¡Connection.take T ?X¦ crcaics a rclcrcncc X wlcn givcn a valio iickci in
T. 1lc X rclcrs io cxacily ilc samc langnagc cniiiy as ilc original rclcrcncc
Copyiighl c 2001-8 Ly I. Van Ioy ano S. Haiioi. All iighls iosoivoo.
722 OIstrIIutod IrogrannIng
ilcn nciwork commnnicaiion is iniiiaico ¦ciwccn ilc iwo proccsscs.
Wiil Connection
ly simplc. 1lc sysicm oocs a grcai ocal ol work io givc ilis simplc vicw. Ii
implcmcnis ilc conncciion proiocol. iransparcni marslaling ano nnmarslaling.
oisiri¦nico gar¦agc collcciion. ano a carclnlly-ocsignco oisiri¦niion proiocol lor
cacl langnagc cniiiy.
11.3.2 SLarIng docIaratIvo data
SLarIng rocords

X¬the_novel(text."It was a dark and stormy night. ..."
author."E.G.E. Bulwer-Lytton"
year.I80·)
¡Show ¡Connection.offerUnlimited X¦¦
1lis cxamplc crcaics ilc iickci wiil Connection.offerUnlimitedano oisplays
ii in ilc Mozari cmnlaior winoow (wiil Show). Any oilcr proccss ilai wanis io
gci a rclcrcncc io X ¦nsi las io know ilc iickci. Hcrc is wlai ilc oilcr proccss
oocs:
X?¬¡Connection.take ´...ticket comes here...´¦
(1o makc ilis work. yon lavc io rcplacc ilc icxi ´....ticket comes here....´
Connection.take
iakcs ilc iickci ano rcinrns a langnagc rclcrcncc. wlicl wc pni in X?. Lccansc
ol nciwork iransparcncy. ¦oil X ano X? ¦clavc iocniically.
SLarIng IunctIons
insicao ol a rccoro:
fun ¡MyEncoder X¦ (X*4449+I?·4) mod ··88I end
¡Show ¡Connection.offerUnlimited MyEncoder¦¦
1lc sccono proccss can gci ilc lnnciion casily:
E?¬¡Connection.take ´...MyEncoders ticket...´¦
¡Show ¡E? I0000¦¦

Hoio, as in lho suLsoquonl oxamplos, wo loavo oul declare foi Liovily, Lul wo loop
declare ... in foi claiily.
Copyiighl c 2001-8 Ly I. Van Ioy ano S. Haiioi. All iighls iosoivoo.