From the web to the grid and beyond : computing paradigms driven by high-energy physics PDF
Preview From the web to the grid and beyond : computing paradigms driven by high-energy physics
THE FRONTIERS COLLECTION THE FRONTIERS COLLECTION Series Editors: A.C. Elitzur L. Mersini-Houghton M. Schlosshauer M.P. Silverman R. Vaas H.D. Zeh J. Tuszynski Thebooksinthiscollectionaredevotedtochallengingandopenproblemsattheforefrontof modern science, including related philosophical debates. In contrast to typical research monographs, however, they strive to present their topics in a manner accessible also toscientificallyliteratenon-specialistswishingtogaininsightintothedeeperimplications and fascinating questions involved. Taken as a whole, the series reflects the need for a fundamentalandinterdisciplinaryapproachtomodernscience.Furthermore,itisintendedto encourageactivescientistsinallareastoponderoverimportantandperhapscontroversial issues beyond their own speciality. Extending from quantum physics and relativity to entropy,consciousnessandcomplexsystems–theFrontiersCollectionwillinspirereaders topushbackthefrontiersoftheirownknowledge. For a full list of published titles, please see back of book or springer.com/series 5342 René Brun Federico Carminati l Giuliana Galli Carminat i Editors From the Web t o t he Grid a nd Bey on d Computing Paradigms Driven by High Energy Physics 123 Editors Renéé Brun Giuliana Galli Carminati Fedeérico Carminati Hôoˆpitaux U niversitaire de Genèvèe CERN Unitéé de la Psychiatrie du Geneva Dééveloppement Mental Switzerland ch. du Petit Bel Air 2 [email protected] 1225 Cheˆne-Bourg Switz erlan d [email protected] [email protected] SeriesEditors: AvshalomC.Elitzur Bar-IlanUniversity,UnitofInterdisciplinaryStudies,52900Ramat-Gan,Israel email:[email protected] LauraMersini-Houghton Dept.Physics,UniversityofNorthCarolina,ChapelHill,NC27599-3255,USA email:[email protected] MaximilianA.Schlosshauer InstituteforQuantumOpticsandQuantumInformation,AustrianAcademyofSciences, Boltzmanngasse3,A-1090Vienna,Austria email:[email protected] MarkP.Silverman TrinityCollege,Dept.Physics,HartfordCT06106,USA email:[email protected] JackA.Tuszynski UniversityofAlberta,Dept.Physics,EdmontonABT6G1Z2,Canada email:[email protected] Ru¨digerVaas UniversityofGiessen,CenterforPhilosophyandFoundationsofScience,35394Giessen, Germany email:[email protected]. DieterZeh GaibergerStraße38,69151Waldhilsbach,Germany email:[email protected] ISSN1612-3018 ISBN978-3-642-23156-8 e-ISBN978-3-642-23157-5 DOI10.1007/978-3-642-23157-5 SpringerHeidelbergDordrechtLondonNewYork LibraryofCongressControlNumber:2011941933 # Springer-VerlagBerlinHeidelberg2012 Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis concerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation,broadcasting, reproductiononmicrofilmorinanyotherway,andstorageindatabanks.Duplicationofthispublication orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9, 1965,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violations areliabletoprosecutionundertheGermanCopyrightLaw. Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnotimply, evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotective lawsandregulationsandthereforefreeforgeneraluse. Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) Preface ModernHighEnergyPhysics(HEP),asasciencestudyingtheresultsofaccelerator- driven particle collisions, was born after the Second World War, at the same time ascomputers.HEPresearchhasbeenconstantlylimitedbytechnology,bothinthe accelerator and detector domains as well as that of computing. At the same time HighEnergyphysicistshavegreatlycontributedtothedevelopmentofInformation Technology. During all these years, the Conseil Europe´en pour la Recherche Nucle´aire1 (CERN [1], located in Geneva, Switzerland) has been a privileged place for the evolution of HEP computing.Several applicationsconceivedfor HEP have found applications well beyond it, the World Wide Web (see Chap.2) being the most notable example. During all these years HEP computing has faced the chal- lenge of software development within distributed communities, and of exploiting geographically distributed computing resources. HEP computing has been very successfulinfulfillingitsmandate,andthequantityandqualityofscientificresults producedbyHighEnergyPhysicsareadirectconsequenceofthis.Inthisactivity, HEP computing had to face problems and situations that anticipated those that Information Technology at large would meet years later. In many respects, HEP computinghasbeentodaywherecomputingwasgoingtobetomorrow. This book describes the evolution of HEP computing, and in particular those aspectsthathavebeenmostinnovative.Theseaspectsaredescribedbycontributions fromdifferentauthors.Thereaderispresumedtobefamiliarwiththefieldofcom- puting,fromsoftwareengineeringtoprogramminglanguages.Nospecificphysics knowledgeisrequired.Thesubjecthasbeentreatedinthepastmostlyinconference proceedingsorin specialisedbooksonphysicscomputing,but,to ourknowledge, nootherbookhasgivenacompleteaccountoftheevolutionofHEPcomputing. 1TheConseilEurope´enpourlaRechercheNucle´aire,orEuropeanCouncilforNuclearResearch is the name of a provisional body founded in 1952 with the mandate of establishing a world- class fundamental physics research organisation in Europe. At that time, pure physics research concentratedonunderstandingtheinsideoftheatom,hencetheword“nuclear”. v vi Preface HighEnergyPhysics in aNutshell Thediscoveryin1896ofthenaturaltransmutationoftheelementsviaradioactive decay by the French scientist Henri Becquerel, while working on phosphorescent materials,coincidedwiththediscoveryof“rays”composedbysub-nuclearparticles of varying energy. These rays were soon used to probe the structure of matter, and using them the New Zealand scientist Ernest Rutherford discovered in 1909 that matter is composed by atoms with a very dense and small positively charged nucleusandnegativelychargedelectronsfloatinginvirtualemptiness.In1912the Austrian-AmericanphysicistVictorHessduringahot-airballoonflightdiscovered theexistenceofhighlyenergeticrayscomingfromouterspaceandhittingtheearth after havingtraversedthe atmosphere.While the originandnature ofcosmic rays still poses some of the most formidable puzzles in physics, this discovery was instrumental in many ways. The history of the universe and the nature of matter were since then intimately linked;the study of our originand of the infinitely big now relies on advances in our understandingof the microscopic nature of matter, andviceversa. Cosmicraysandthediscoveryofantimatter. In 1936 the American physicist Carl David Anderson, studying cosmic rays, discoveredantimatter,earlierpostulatedbytheBritishphysicistPaulDirac.Cosmic raysalsoprovidedphysicistswithprobesofenergiesmuchhigherthanthosecoming fromnucleartransmutation.This was an importantstep in the investigationof the nature of matter. In his 1924 doctoral thesis, the French physicist Louis-Victor- Pierre-Raymond,7thDucdeBroglie,introducedthesuccessfulhypothesisthatwith eachparticleisassociatedawavewhoselengthvariesastheinverseoftheparticle energy. The “resolution power” of an instrument, that is its ability to distinguish smalldetails, varieswith the inverseof thewavelengthofthe radiationemployed. Highly energetic particles are excellent probes into the sub-nuclear world, as the associatedwaveshaveveryshortlength. Whilecosmicraysprovidedveryshortlengthwaves,theirenergy,aswellastheir arrivalangle,wassubjecttoawidestatisticaldistribution,makingsystematicstudies verydifficult.Physiciststhendecidedtoproducehighenergyraysinthelaboratory byacceleratingchargedsub-nuclearparticlesandnucleiwithelectromagneticfields. In the 1930s the British physicists John D. Cockcroft and Ernest Walton, the American engineer Robert Van de Graaf and a Soviet team in Kharkov managed toacceleratenucleiviathecreationofanintenseelectricfield.Vacuumtubeswere usedtoaccelerateelectrons,andthisledtothediscoveryofthephotoelectriceffect in 1887 by the German physicist Heinrich Rudolf Hertz. The explanation of this effectbyAlbertEinsteinin1905led totheformulationofa corpusculartheoryof lightandwasoneofthefoundingstepstowardsmodernQuantumMechanics. Preface vii ThebirthofHighEnergyPhysics The use of very intense electric fields was however impractical due to the difficulty to generate and control them. Particles could be accelerated by guiding themrepeatedlythroughlowerintensity electricfields, butthisrequiredverylong “linear”machinestoachievehighenergies.Amajorbreakthroughwasachievedin 1929byErnestO.LawrenceattheUniversityofCalifornia,Berkeley.Hemanaged toaccelerateelectronswith a relativelylowvoltagebypassingthemseveraltimes througha potential difference.Lawrence’s machine used a magnetic field to keep the electrons on a circular path, and an alternating voltage gradient to accelerate them. Lawrence’sfirst machine,called a cyclotron,was made outof brass, sealed with wax and had a diameter of ten centimetres;it is said to have cost $25 in all. It was the first particle accelerator and it opened the way to the creation of sub- nuclear and nuclear rays of ever-increasing energy (hence the name High Energy Physics).AlmostallhighenergyparticleacceleratorstodayderivefromLawrence’s machine. Due to fundamentalphysicslaws, the diameter of the accelerator has to increasewithenergy,somodernmachineshavediametersofseveralkilometresand are hosted in underground tunnels. Several particle accelerators have been built since the end of the Second World War. Major laboratories exists in the United States,Japan,EuropeandRussia.ThelargestofthemallisattheEuropeanConseil Europe´enpourlaRechercheNucle´aire(CERN)inGeneva,Switzerland. TheroleofCERN BuiltontheFranco-Swissborder,CERNwasfoundedin1954anditiscurrently fundedby20Europeannations.CERNisnowoperatingtheLargeHadronCollider (LHC)whichiscollidingprotonbeamsatenergiesupto7Tera(1012)electronVolts (to be increased to 14Tera electronVolts after 2013), and lead nuclei beams at energies up to 2.76Tera electronVolts per nucleon (to be increased to 5.5Tera electronVolts after 2013). The LHC is the largest scientific machine ever built. CERN has financed and built most of the accelerator complex, with important contributions from the United States, Russia and Japan. Four experiments have started to analyse the results of particle collisions around the 28km LHC ring. Experimentsaredesigned,financedandbuiltbylargeinternationalcollaborationsof hundredsofphysicsinstitutes,universitydepartmentsandlaboratories–including CERN’sownExperimentalPhysicsDivision–comprisingthousandsofphysicists fromfourcontinents. A modernHigh Energyexperimentis typicallybuilt at the “intersectionpoint” of a particle accelerator. It is here that counter-rotating beams of particles cross each other generating particle collisions. The energy of each collision transforms viii Preface itself into mass, according to Einstein’s famous formula E D mc2, generating a host of particles not observed on Earth under normal conditions. The “detectors” areverylargescientificinstrumentsthatsurroundtheinteractionpointsanddetect theparticles.Severalmillioncollisionsoccureachsecond,and,afterdiscardingthe uninterestingones, the survivingones are registered for furtherprocessingat data ratesthatcanreachfewGigabytespersecond. It is perhaps important to note that particle collisions create a very high temperature and energy density, similar to what is believed to have existed at the beginning of our universe immediately after the Big Bang. In this sense, a particleacceleratorisalsoa“timemachine”whichreproducestheconditionofthe universesoon after its birth,even if onlyfor a very tiny amountof matter. Again, the understanding of our origins is linked with the microscopic composition and behaviourofmatter. Computingin HEP Since its beginnings, High Energy Physics has relied heavily upon computers to extractmeaningfulphysicsresultsfromobservations.Thisisduetoseveralreasons, whichhavetodowiththestatisticalorprobabilisticnatureoftheunderlyingtheory andwiththelargeamountofcollecteddatathathastobetreatedelectronically. Verylargecomputingneedsandcomplexity. This has taken on very large proportions with the latest generation of experi- ments, which have produced several hundred TeraBytes of data. The new experi- mentswhichcameon-linein2009atCERNareproducingafewPetaBytes(1015) of data per year. These data need to be pre-processed several times before being properly analysed. The design and understanding of the experimental apparatus requirethecomputersimulationofitsresponse,andthisisa verydemandingtask intermsofcomputingresources,asitgeneratesanamountofdatacomparablewith thatoftherealexperiments.HoweverthecomplexityofcomputingforHighEnergy Physics is not only due to the sheer size of the resources needed, but also to the way in which the software is developed and maintained. Laboratories that have an accelerator, such as CERN in Geneva, Fermi National Accelerator Laboratory in Chicago, the Brookhaven National Laboratory, or Stanford Linear Accelerator Centrehosthighenergyexperiments.Aswehavementioned,anexperimentisbuilt by a large international collaboration comprising thousands of scientists coming fromhundredsofinstitutesinafewtensofcountries.Theseresearchersworkrather independently on the same code, meeting only rarely and with little hierarchical structure between them. Requirements change very frequently and the problems to be solved often push the boundaries of scientific knowledge, both in physics, Preface ix andalsoincomputerscience.Itcanbesaidthatscientificresearchinfundamental physicshasitslimitnotinthecreativityoftheresearchers,butinthecurrentstate- of-the-artofengineeringandcomputerscience. Ithashappenedseveraltimesin thepastthatphysicistshavebeenattheorigin of important developments in the field of computing. Notable examples are the invention of the Web at CERN and, more recently, the CERN-led development anddeploymentofthelargestGrid(seeChap.3forthedescriptionoftheconcept) in operation in the world for the storage and processing of the data coming from the LHC experiments.And while the Grid is still developingits fullpotential,the attentionoftheHEPcomputingworldisalreadymovingtocomputingCloudsand virtualisation(seeChap.6)attheboundaryofknowledgeindistributedcomputing. Theseresultshavebeengivenconsiderableattentionbythemedia.Howeverthere is relatively little published material on the general history and development of computing in High Energy Physics, which presents many other interesting and innovativeaspects.Forinstance,programmecodehasalwaysbeensharedbetween physicists, even beyond the boundaries of a single experiment, in a very similar way to what has now become known as “open source” software development. The management of the developmentof a large piece of software driven by very dynamicrequirementshas given rise to techniquesvery similar to those that have beenindependentlydevelopedby“AgileSoftwareEngineering”technologies. Most scientific communication between fundamental physicists is centred on their scientific results, with little attention paid to the discussion of their activity asInformationTechnologyprofessionals(seeChap.5). ITinnovationoriginatedinHEPwentoftenunnoticedoutsidethefield. Theresultisthatseveralcomputingtechniquesandconcepts,developedandapplied in the context of High Energy Physics, have had broader success after being “reinvented”anddevelopedindependently,maybeyearslater.Themainreasonfor writingthisbookistodescribetheworldofcomputinginHighEnergyPhysics,not onlythroughitsresults,butbydescribingitschallengesandthewayinwhichHEP is tackling them. The aim is to providea needed“theoretical” justification for the techniquesand “traditions” of HEP computingwith the intentionof making them betterknown,andpossiblypromotingdialogueandinterdisciplinarycollaboration withotherdisciplinesthatusecomputingeitherasa researchfieldperse, orasan importanttoolin their activity.Thisseems to be a veryappropriatemomentto do it,becauseagenerationofexperimentshasnowcomeon-lineafteralmost20years of development, and therefore we can now describe a mature and accomplished situation in computing for HEP. The next generation of experiments will almost certainlyintroduceradicalchangestothisfield,oratleastwehopeso. A large part of this book talks about the development of HEP computing at CERNandthereforethereadermayquestionthelegitimacyoftheclaimthatitdeals
The list of books you might like
