Table Of ContentUsingM PI
Scientiafincd EngineeriCnogm putation
JanusKzo walik, editor
Data-ParaPlrloeglr amminogn MIMDC omputerPsh,i liJp.H atchearn dM ichaeJl.Q uinn1,9 91
UnstructuSrceide ntiCfiocm putatioonn S calabMluel tiprocesesdoirtse,bdy PiyusMhe hrotra,
JoeSla ltazn,d R oberVto igt1,9 92
ParallCeolm putatioFnlauli Ddy namicsI:m plementatainodnR esultesd,i tebdy Horst
D. Simon1,9 92
EnterprisIen tegratMioodne linPgr:o ceedionfgt sh eF irst InternaCtoinofnearle necdei,t ebdy
CharleJs.P etriJer,. 1,9 92
TheH ighP erformanFcoer traHna ndbookC,h arleHs. KoelbeDla,v idB .L ovemanR,o bert
S.S chreibGeury, L .S teelJer .a ndM aryE .Z osel1,9 94
PVM: ParallVeilr tuaMla chine-UAs ersG'u idea ndT utorifaolrN etworPka rallCeolm puting,
AlG eistA,d am BegueliJna,c kD ongarrWae,i chenJgi angB,o b Mancheka,n d VaiSduyn deram,
1994
PracticPaalr allPerlo gramminGgr,e gorVy. W ilson1,9 95
EnablinTgec hnologifeosrP etafiopCso mputinTgh,o masS terliPnagu,l M essinaan,d P aul
H.S mith1,9 95
An IntroducttiooH ni gh-PerformSacniceen tiCfiocm putinLgl,o ydD .F osdicEkl,i zabeth
R.J essupC,a rolyJn. C .S chaublaen,d G ittDao mik,1 995
ParallPerlo gramminUgs inCg+ +, editebdy GregorVy. W ilsoann dP aulL u,1 996
UsinPgL APACK: ParallLeiln eaArl gebrPaa ckageRo,b erAt. v and eG eijn1,9 97
Fortra9n5 H andbookJ,e annCe. A dams,W alteSr. B rainerJde,a nnTe. M artinB,r iaTn. S mith,
JerroLl.dW agener1,9 97
MPI-The Complete RefereVnocleu:m e1 ,T heM PI CoreM,a rcS nirS,t evOet toS,t even
Huss-LedermDaanv,i dW alkera,n dJ ackD ongarr1a9, 98
MPI-The Complete ReferVeonlcuem:e2 ,T heM PI-2E xtensioWnisl,l iaGmr oppS,t even
Huss-LedermAannd,r ewL umsdainEew,i ngL usk,B ilNli tzberWgi,l liaSma phiarn,d M arcS nir,
1998
A ProgrammerG'usi det oZ PL,L awrencSen yder1,9 99
How toB uilad BeowulTfh,o masL .S terliJnogh,n S almonD,o naldJ .B eckera,n dD aniel
F.S avares1e9,9 9
UsinMgP I:P ortabPlaer allPerlo gramminwgit ht heM essage-PassIinntge rfasceec,o nedd ition,
WilliaGmr oppE,w ingL uska,n dA nthonSyk jellu1m9,9 9
UsinMgP I-2:A dvanced Feaotfut rheesM essage-PassIinntge rfaWciel,l iaGmr opp, Ewing
Luska,n dR ajeeTvh akur1,9 99
UsingM PI
PortabPlaer alPlerolg ramminwgi tht he
Message-PasIsnitnegr face
SeconEdd ition
WilliaGmr opp
EwingL usk
AnthonSyk jellum
TheM IT Press
CambridgeM,a ssachusetts
LondonE,n gland
© 1999M assachusIentsttsi toufTt eec hnology
Allr ighrtess ervNeod p.a rotf t hibso okm ayb er eproduicnea dn yf ormb ya nye lectroonri c
mechanimceaaln s( including photroeccooprydiinongrig, n, f ormatsitoonr aagned r etrieval)
withopuetr missiinow nr itifnrgo mt hep ublisher.
J¥IEX
This bowoakss eitn byt hea uthoarnsd w asp rintaenddb ounidn t heU niteSdt atoefs
America.
LibraorfyC ongreCsast aloging-in-PuDbaltiac ation
GroppW,i lliam.
UsinMgP I: portabplaer alplreolg rammiwnigt ht hem essage-pasisnitnegr f/aW ciel liaGmr opp,
EwingL uskA,n thonSyk jellum.-e2dn.d
p.c m.- (Scientainfidec n gineercionmgp utation)
Includbeisb liograprheifcearle nacnedis n dex.
ISBN-193:7 8-0-262-57(1s3e4tp-:b0 k :. alpka.p er)-978-0-2I36 22--65 7I( : v .p bk. :p aaplekr.)
ISBN-l0O-:2 62-53741- X( setp:b k:. alpka.p er)-0-26I23 -25-73( vI.: pbk:. alpka.p er)
I.P aralplreolg rammi(nCgo mputsecri ence2)..P aralcloemlp uters-programming.
3.C omputeirn terfaceLsu.s kE,Iw .i ng. SIkIj.e llAunmt,h onyI.I I. TitlSee.r ieIsV..
QA 76.642.G7169 99
005.2'75-dc21
99-16613
CIP
10 9 8 7 6 5
To PaBtrtiyga,in dJd,e nnifer
Contents
SerFioerse word
Xlll
Preftaotc heSe e coEnddi tion xv
Preftaotc heFe i rEsdti tion
XIX
1 Background 1
1.1W hyP araClolmeplu ting? 1
1.2O bstatcoPl reosg ress 2
1.3W hyM essPaagses ing? 3
1.3P.a1r aClolmeplu taMtoidoenlasl 3
1.3A.d2v antoaftg heMese ssage-Passing Model 9
1.4E voluotfMi eosns age-SPyasstseimnsg 10
1.5T hMeP IF orum 11
2 IntroductitoonM PI 13
2.1G oal 13
2.2W hatI sM PI? 13
2.3B asMiPcIC oncepts 14
2.4O thIenrt erFeestaitonugfrM ePsI 18
2.5I sM PIL arogrSe m all? 20
2.6D ecisLieoftntots h Iem plementor 21
3 UsingM PI inS impleP rograms 23
3.1A FrisMtP IP rogram 23
3.2R unniYonugFr i rMsPtIP rogram 28
3.3A FrisMtP IP rogrianCm 29
3.4A FrisMtP IP rogrianCm + + 29
3.5T imiMnPgIP rograms 34
3.6A Self-ScEhxeadmuplMliaent:gr ix-Vector Multip3l5i cation
3.7S tudyPianrgaP lelreflo rmance 43
3.7E.l1e menStcaarlya bility Calculations4 3
3.7G.a2t heDraitnoagnP rogrEaxme cution 45
3.7I.n3s trumaeP natriaPnlrglo eglrw aimtM hP EL ogging 46
viii Contents
3.7E.v4e natnsSd t ates 47
3.7I.n5s trumtehnMeta itnrgi x-MMualttriPiprxlo yg ram 47
3.7N.o6t OensI mplemenotfLa otgigoinn g 49
3.7E.x7a miLnoignfigwl ietUshp shot 52
3.8U s inCgo mmunicators 53
3.9A notWhaeyro fF ormNienwgC ommunicators 61
3.1A0 H andGyr aphLiicbsrf aorPrya raPlrloeglr ams 62
3.1C1o mmoEnr roarnMsdi sunderstandings 64
3.1A2p plicQautainotnMu:om n tCea rClaol culianNt uicolnesa r
Physics 66
3.1S3u mmaorfayS impSlueb soefMt P I 67
4 IntermediaMtPeI 69
4.1T hPeo isson Problem 70
4.2T opologies 73
4.3A Codfeot rh Peo isPsroonb lem 81
4.4U siNnogn bloCcokmimnugn ications 93
4.5S ynchrSoennoadunssd" SaPfreo"g rams 96
4.6M oroenS calability 96
4.7J acowbiita h2 -DDe composition 98
4.8A nM PID eriDvaetda type 100
4.9O verlaCpopmimnugn icaantCdio omnp utation 101
4.1M0o roenT imiPnrgo grams 105
4.1T1h rDeiem ensions 107
4.1C2o mmon EarnrMdoi rssu nderstandings 107
4.1A3p plicSaitmiuolnVa:ot rintge x EivnSo ulpuetricoonn ducting
Materials 109
5 AdvancedM essageP assinign M PI 111
5.1M PID atatypes 111
5.1B.a1s Diact ataynpCdeo sn cepts 111
5.1D.e2r iDvaetda types 114
Contents ix
5.1U.n3d erstEaxntdeinntgs 117
5.2T heN -BoPdryo blem 117
5.2G.a1t her 118
5.2N.o2n bloPcikpienlgi ne 123
5.2M.o3v iPnagr tibceltewPser eonc esses 124
5.2S.e4n dDiynnga miAclallolcyDa attead 132
5.2U.s5e r-ConDtartPoaal clkeidn g 134
5.3V isualtihMzeai nndge lSberto t 138
5.3S.e1n dingo fAS rtrraucytsu res 145
5.4G apisnD atatypes 146
5.4M.P1I -F2u nctfiooMrna sn ipulEaxttienngt s 148
5.5N ewM PI-D2a tatRyopuet ines 150
5.6M oroenD atatfyopSret sr uctures 152
5.7D eprecFautnecdt ions 154
5.8C ommoEnr roarnMsdi sunderstandings 156
6 ParallLeilb raries 157
6.1M otivation 157
6.1T.h1eN e efdo Pra raLlilberla ries 157
6.1C.o2m moDne ficieonfPc rieevsiM oeusssa ge-Passing
Systems 158
6.1R.e3v ioefMw P IF eatTuhraeStsu ppLoirbtr aries 160
6.2A FrisMtP IL ibrary 163
6.2M.P1I -A2t tributeR-oCuatcihniensg 172
6.2A. C2+ +A lterntaot ive 172
MPLComlLdup
6.3L inear oAnlG greibdrsa 177
6.3M.a1p pianngLdso giGcraild s 178
6.3V.e2c taonrMdsa trices 181
6.3C.o3m ponoefanP tasr aLlilberla ry 185
6.4T heL INPABCeKn chmianMr PkI 189
6.5S tratfeogLrii ebsrB auriyl ding 190
6.6E xampolfLe isb raries 192
x Contents
7 OtherF eatureosf M PI 195
7.1 SimSuhlaarteidn-gMO epmeorrayt ions 195
7.1.1 vSshD.ai rsetdr iMbeumtoerdy 195
7.1A. C2o unEtxearm ple 196
7.1T.h3Se h arCeodu nUtseirPn ogl liinnsgto efaa nEd x tra
Process 200
7.1F.a4i rinnMe essss Paagses ing 201
7.1E.x5p loRietqiunegs t-RMeesspsoPanagsteet erns 202
7.2A pplicFautlilo-nC:o nfiIgnutreartaicotni on 205
7.3A dvanCcoeldl eOcpteirvaet ions 206
7.3D.a1t Mao vement 206
7.3C.o2l leCcotmipvuet ation 206
7.3C.o3m moEnr roarnMsdi sunderstandings 213
7.4I ntercommunicators 214
7.5H eterogCeonmepouutsi ng 220
7.6T hMeP IP rofiIlnitnegr face 222
7.6F.i1n dBiunffge rPirnogb lems 226
7.6F.i2n dLionagId m balances 228
7.6T.h3Me e chaonfUi scistn hgPe r ofilIintnegr face 228
7.7E rrHoarn dling 229
7.7E.r1r Hoarn dlers 230
7.7A.n2E xampolfEe r rHoarn dling 233
7.7U.s3e r-DEerfirnHoeardn dlers 234
7.7T.e4r minMaPtIPi rnogg rams 237
7.7M.P5I -F2u nctfiooErnr srH oarn dling 239
7.8T hMeP IE nvironment 240
7.8P.r1o ceNsasmoer 242
7.8I.sM2 P II nitialized? 242
7.9D etermtihnVeie nrgso ifMo PnI 243
7.1O0t hFeurn ctiinMo PnIs 245
7.1A1p plicCaotmipount:a FtuliioDdny anla mics 246
7.1P1a.r1a Flolremlu lation 246
Contents xi
7.1P1a.r2a Ilmlpelle mentation 248
8 Understandihnogw MPI ImplementatioWnosr k 253
8.1I ntroduction 253
8.1S.e1n dDiantga 253
8.1R.e2c eiDvaitnag 254
8.1R.e3n dezPvrooutso col 254
8.1M.a4t chPirnogt otcoMo PlIsS' esn Mdo des 255
8.1P.e5r forImmapnlciec ations 256
8.1A.l6t ernMaPtIIi mvpel emenSttartaitoeng ies 257
8.1T.u7n iMnPgII mplementations 257
8.2H owD ifficIusMl PtI t oI mplement? 257
8.3D eviCcaep abialnitdth iMeeP sIL ibrDaerfiyn ition 258
8.4R eliaobfDi altiTrata yn sfer 259
9 ComparingM PI withO therS ystemsf or 261
InterproceCsosm munication
9.1S ockets 261
9.1P.r1o cSetsasr atnuSdph utdown 263
9.1H.a2n dlFianugl ts 265
9.2P VM3 266
9.2T.h1eB a sics 267
9.2M.i2s cellFaunnecotuiso ns 268
9.2C.o3l leOcpteirvaet ions 268
9.2M.P4I C ounteropfOa trhtFeser a tures 269
9.2F.e5a tNuoritens M PI 270
9.2P.r6o cSetsasr tup 270
9.2M.P7I a nPdV Mr elated tools 271
9.3W herteoL eaMronr e 272
10 BeyondM essageP assing 273
10.D1y namPirco cMeasnsa gement 274
10.T2h reads 275
