Machine Learning Techniques for Space Weather PDF
Preview Machine Learning Techniques for Space Weather
MACHINE LEARNING TECHNIQUES FOR SPACE WEATHER MACHINE LEARNING TECHNIQUES FOR SPACE WEATHER Editedby Enrico Camporeale Simon Wing Jay R. Johnson Elsevier Radarweg29,POBox211,1000AEAmsterdam,Netherlands TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UnitedKingdom 50HampshireStreet,5thFloor,Cambridge,MA02139,UnitedStates ©2018ElsevierInc.Allrightsreserved. Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicor mechanical,includingphotocopying,recording,oranyinformationstorageandretrievalsystem,without permissioninwritingfromthepublisher.Detailsonhowtoseekpermission,furtherinformationaboutthe Publisher’spermissionspoliciesandourarrangementswithorganizationssuchastheCopyrightClearanceCenter andtheCopyrightLicensingAgency,canbefoundatourwebsite:www.elsevier.com/permissions. ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(other thanasmaybenotedherein). Notices Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroadenour understanding,changesinresearchmethods,professionalpractices,ormedicaltreatmentmaybecomenecessary. Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluatingandusingany information,methods,compounds,orexperimentsdescribedherein.Inusingsuchinformationormethodsthey shouldbemindfuloftheirownsafetyandthesafetyofothers,includingpartiesforwhomtheyhaveaprofessional responsibility. Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeanyliability foranyinjuryand/ordamagetopersonsorpropertyasamatterofproductsliability,negligenceorotherwise,or fromanyuseoroperationofanymethods,products,instructions,orideascontainedinthematerialherein. LibraryofCongressCataloging-in-PublicationData AcatalogrecordforthisbookisavailablefromtheLibraryofCongress BritishLibraryCataloguing-in-PublicationData AcataloguerecordforthisbookisavailablefromtheBritishLibrary ISBN:978-0-12-811788-0 ForinformationonallElsevierpublicationsvisitour websiteathttps://www.elsevier.com/books-and-journals Publisher:CandiceJanco AcquisitionEditor:MarisaLaFleur EditorialProjectManager:KaterinaZaliva ProductionProjectManager:NileshKumarShah CoverDesigner:MatthewLimbert TypesetbySPiGlobal,India Contributors Livia R. Alves National Institute for Space RichardE.Denton DartmouthCollege,Hanover, Research—INPE, São José dos Campos, SP, NH,UnitedStates Brazil MikeHapgood LancasterUniversity,Lancaster; Vassilis Angelopoulos Institute of Geophysics RAL Space, Harwell, Didcot, United andPlanetaryPhysics/Earth,LosAngeles,CA, Kingdom UnitedStates Verena Heidrich-Meisner Institute of Experi- Daniel N. Baker University of Colorado mentalandAppliedPhysics,Kiel,Germany Boulder,Boulder,CO,UnitedStates StefanJ.Hofmeister UniversityofGraz,Graz, Ramkumar Bala Rice University, Houston, TX, Austria UnitedStates George B. Hospodarsky University of Iowa, Michael Balikhin Department of Automatic IowaCity,IA,UnitedStates Control and Systems Engineering, University Paulo R. Jauer National Institute for Space ofSheffield,Sheffield,UK Research—INPE, São José dos Campos, SP, Jacob Bortnik University of California, Los Brazil Angeles,LosAngeles,CA,UnitedStates Jay R. Johnson Andrews University, Berrien Richard Boynton Department of Automatic Springs,MI,UnitedStates Control and Systems Engineering, University Shrikanth G. Kanekal NASA Goddard Space ofSheffield,Sheffield,UK FlightCenter,Greenbelt,MD,UnitedStates EnricoCamporeale CentrumWiskunde&Infor- Adam Kellerman UCLA, Los Angeles, CA, matica,Amsterdam,TheNetherlands UnitedStates AlgoCarè UniversityofBrescia,Brescia,Italy CraigA.Kletzing UniversityofIowa,IowaCity, IA,UnitedStates Mandar Chandorkar Centrum Wiskunde & Informatica,Amsterdam,TheNetherlands Daiki Koga National Institute for Space Research—INPE, São José dos Campos, SP, Xiangning Chu University of California, Los Brazil Angeles,LosAngeles,CA,UnitedStates Alisson Dal Lago National Institute for Space Giuseppe Consolini National Institute for Research—INPE, São José dos Campos, SP, Astrophysics, Institute for Space Astrophysics Brazil andPlanetology,Rome,Italy Zi-Qiang Lang Department of Automatic Con- FLARECASTConsortium AcademyofAthens, trol and Systems Engineering, University of Trinity College Dublin, Università di Genova, Sheffield,Sheffield,UK Consiglio Nazionale delle Ricerche, Centre NationaldelaRechercheScientifique,Université Wen Li Boston University, Boston, MA, United Paris-Sud, Fachhochschule Nordwestschweiz, States MetOffice,NorthumbriaUniversity Marco Loog Delft University of Technology, Véronique Delouille Royal Observatory of Delft, The Netherlands; University of Belgium,Brussels,Belgium Copenhagen,Copenhagen,Denmark xi xii CONTRIBUTORS Qianli Ma Boston University, Boston, MA, HarlanE.Spence UniversityofNewHampshire, United States; University of California, Los Durham,NH,UnitedStates Angeles,LosAngeles,CA,UnitedStates MariaSpasojevic StanfordUniversity,Stanford, Benjamin Mampaey Royal Observatory of CA,UnitedStates Belgium,Brussels,Belgium Manuela Temmer University of Graz, Graz, AnnaM.Massone CNR—SPIN,Genova,Italy Austria Claudia Medeiros National Institute for Space RichardM.Thorne UniversityofCalifornia,Los Research—INPE, São José dos Campos, SP, Angeles,LosAngeles,CA,UnitedStates Brazil Astrid Veronig University of Graz, Graz, Michele Piana CNR—SPIN; Università di Austria Genova,Genova,Italy Luis E. A. Vieira National Institute for Space GeoffreyD.Reeves LosAlamosNationalLabo- Research—INPE, São José dos Campos, SP, ratory; Space Science and Applications Group, Brazil LosAlamos,NM,UnitedStates Hua-LiangWei DepartmentofAutomaticCon- Patricia Reiff Rice University, Houston, TX, trol and Systems Engineering, University of UnitedStates Sheffield,Sheffield,UK MartinA.Reiss SpaceResearchInstitute,Graz, Robert F. Wimmer-Schweingruber Institute Austria of Experimental and Applied Physics, Kiel, Germany YuriY.Shprits HelmholtzCentrePotsdam,GFZ GermanResearchCentreforGeosciences;Uni- SimonWing JohnsHopkinsUniversity,Laurel, versityofPotsdam,Potsdam,Germany;Univer- MD,UnitedStates sityofCaliforniaLosAngeles,LosAngeles,CA, Xiaojia Zhang University of California, Los UnitedStates Angeles,LosAngeles,CA,UnitedStates Ligia A. Da Silva National Institute for Space Irina S. Zhelavskaya Helmholtz Centre Pots- Research—INPE, São José dos Campos, SP, dam, GFZ German Research Centre for Geo- Brazil sciences; University of Potsdam, Potsdam, Vitor M. Souza National Institute for Space Germany Research—INPE, São José dos Campos, SP, Brazil Introduction Enrico Camporeale*, Simon Wing†, Jay R. Johnson‡ *CentrumWiskunde&Informatica,Amsterdam,TheNetherlands †JohnsHopkinsUniversity, Laurel,MD,UnitedStates ‡AndrewsUniversity,BerrienSprings,MI,UnitedStates Acommongoalofscientificdisciplinesistounderstandtherelationshipsbetweenobservable quantitiesandtoconstructmodelsthatencodesuchrelationships.Eventuallyanymodel,and its supporting hypothesis, needs to be tested against observations—the celebrated Popper’s falsifiabilitycriterion(Popper,1959).Hence,experiments,measurements,andobservations— in one word data—have always played a pivotal role in science, at least since the time of Galileo’sexperimentdroppingobjectsfromtheleaningtowerofPisa. Yet, it is only in the last decade that libraries’ bookshelves have started to pile up with books about the data revolution, big data, data science, and various modifications of these terms. While there is certainly a tendency both in science and publishing to re-brand old ideas and to inflate buzzwords, one cannot deny that the unprecedented large amount of collected data of any sort—be it customer buying preferences, health and genetic records, highenergyparticlecollisions,supercomputersimulationresults,orofcourse,spaceweather data—makesthetimewearelivinginuniqueinhistory.Thedisciplinethatbenefitsthemost fromtheexplosionofthedatarevolutioniscertainlymachinelearning.Thisfieldistraditionally seenasasubsetofartificialintelligence,althoughitsboundariesanddefinitionaresomehow blurry. For the purposes of this book, we broadly refer to machine learning as the set of methods and algorithms that can be used for the following problems: (1) make predictions intimeorspaceofacontinuousquantity(regression);(2)assignadatumtoaclasswithina prespecifiedset(classification);(3)assignadatumtoaclasswithinasetthatisdeterminedby thealgorithmitself(clustering);(4)reducethedimensionalityofadataset,byexposingrela- tionshipsamongvariables;and(5)establishlinearandnonlinearrelationshipsandcausalities amongvariables. Machinelearningisinitsgoldenagetodayforthesimplereasonthatmethods,algorithms, andtools,studiedanddesignedduringthelasttwodecades(andsometimesforgotten),have started to produce unexpectedly good results in the last 5 years, exploiting the historically uniquecombinationofbigdataavailabilityandcheapcomputingpower. Thesinglemethodologythathasbeenpopularizedthemostbynonspecialistmediaasthe archetypeofmachinelearning’sgroundbreakingpromiseisprobablythemassivemultilayer neural network, which is often referred to as deep learning (LeCun et al., 2015). For instance, deeplearningisthetechnologybehindtherecentsuccessesinimageandspeechrecognition (withtheformerrecentlyachievingbetter-than-humanaccuracy;Heetal.,2015)andthefirst computereverdefeatingaworldchampioninthegameofGo(Silver,2016). The popular media often focus on the technological applications of machine learning, which has propelled recent advances in many areas, such as self-driving cars, online fraud detection,personalizedadvertisementandrecommendation,real-timetranslation,andmany xiii xiv INTRODUCTION others (Bennett and Lanning, 2007; Sommer and Paxson, 2010; Guizzo, 2011). However, we believethatitmakessensetoaskwhethermachinelearningcouldevenchangetheprocessof scientificdiscovery. Lookingspecificallyatphysics,theprocessofdevelopingamodeloftenreliesonsomeform ofthewell-known Occam’s razor: thesimplest model thatcan explain thedataispreferred. As a consequence, an important characteristic of most physics models is that every step of theprocessthatledtotheirdevelopmentiscompletelyintelligiblebythehumanmind.Such models are referred to as white-box models, suggesting that each component (including the set of assumptions) is transparent. Despite its marvelous achievements, the human brain has a very limited ability to process data, especially in high dimensions. This might be trivially related to the fact that the basic way of understanding data is graphical, and it is hardtovisualizemorethanthreevariablesinasingleplot.Hence,therelationshipsbetween observable quantities that are encoded in white-box physics models usually do not explore highdimensionalspaces.Thishumanlimitationdoesnotmeanthatsuchmodelsare“simple”; on the contrary they can be quite complicated, sometimes requiring formidable numerical methods to produce results that can be compared against observations. Essentially, all first- principlesphysicsmodelsarewhite-boxmodels. Contrarytothemodusoperandiofthewhiteboxes(onecouldperhapssayofthehuman mind), machine learning algorithms focus essentially on two characteristics: being accurate andbeingrobustagainstnewdata(i.e.,beingabletogeneralize).Indeed,theguidingprinciple concernsthetrade-offbetweencomplexityandaccuracytoavoidoverfitting(seeChapter4). Hence,incontrasttowhite-boxmodels,machinelearningmethodsareoftenreferredtoas black-box,signifyingthatthemathematicalstructureandtherelationshipsbetweenvariables aresocomplicatedthatitisoftennotusefultotrytounderstandthem,aslongastheydeliver the expected results. For example, and referring again to deep learning, one can certainly unroll a neural network to the point of deriving a single closed formula that relates inputs andoutputs.However,suchaformulawouldgenerallybeincomprehensibleandcompletely uselessfromascience-basedperspective,althoughsomefeaturesmayberelatedtophysical processes. Weneedtomentionathird,in-betweenparadigm,obviouslycalledgray-boxmodelingthat has recently emerged. Whereas white-box models are accurate but computationally slow (often much slower than real time when it comes to forecasting), and black-box models are fastbutverysensitivetonoiseandoutliers,theideaofgrayboxistoemployreducedphysics models, and to calibrate the assumptions or the free parameters of the models via machine learning techniques. Gray box is often used in engineering modeling, and it is gradually makingitswayintomorefundamentalphysics.Inparticular,webelievethattheskepticism thatsurroundsmachinelearningincertainphysicscommunitieswillbeeventuallyovercome byembracinggray-boxmodels,whichallowtheuseofpriorphysicalinformationinamore transparentway. MACHINE LEARNING AND SPACE WEATHER Space weather is the study of the effect of the Sun’s variability on Earth, on the complex electromagneticsystemsurroundingit,onourtechnologicalassets,andeventuallyonhuman xv INTRODUCTION life. It will be more clearly introduced in Chapter 1, along with its societal and economic importance. This book presents state-of-the-art applications of machine learning to the space weather problem. Artificial intelligence has been applied to space weather at least since the 1990s. In particular, several attempts have been made to use neural networks and linear filters for predicting geomagnetic indices and radiation belt electrons (Baker, 1990; Valdivia etal.,1996;Sutcliffe,1997;Lundstedt,1997,2005;Bobergetal.,2000;Vassiliadis,2000;Gleisner andLundstedt,2001;Li,2001;Vandegriff,2005;Wingetal.,2005).Neuralnetworkshavealso beenusedtoclassifyspaceboundariesandionospherichighfrequencyradarreturns(Newell et al., 1991; Wing et al., 2003), and total electron content (Tulunay et al., 2006; Habarulema et al., 2007). A feature that makes space weather very remarkable and perfectly posed for machinelearningresearchisthatthehugeamountofdataisusuallycollectedwithtaxpayer money and is therefore publicly available. Moreover, the released datasets are often of very highqualityandrequireonlyasmallamountofpreprocessing.Evendatathathavenotbeen conceivedforoperationalspaceweatherforecastingofferanenormousamountofinformation tounderstandprocessesanddevelopmodels.Chapter2willdwellconsiderablyonthenature andtypeofavailabledata. Inparalleltotheabove-mentionedmachinelearningrenaissance,anewwaveofmethods andresultshavebeenproducedinthelastfewyears,whichistherationaleforcollectingsome ofthemostpromisingworksinthisvolume. The machine learning applications to space weather and space physics can generally be dividedintothefollowingcategories: • Automaticeventidentification:Spaceweatherdataistypicallyimbalanced,withmanyhours ofobservationscoveringuninteresting/quiettimes,andonlyasmallpercentageofdataof usefulevents.Theidentificationofeventsisstilloftencarriedoutmanually,following time-consumingandnonreproduciblecriteria.Asanexample,techniquessuchas convolutionalneuralnetworkscanhelpinautomaticallyidentifyinginterestingregions likesolaractiveregions,coronalholes,coronalmassejections,andmagneticreconnection events,aswellastoselectfeatures. • Knowledgediscovery:Methodsusedtostudycausalityandrelationshipswithinhighly dimensionaldata,andtoclustersimilarevents,withtheaimofdeepeningourphysical understanding.Informationtheoryandunsupervisedclassificationalgorithmsfallinto thiscategory. • Forecasting:Machinelearningtechniquescapableofdealingwithlargeclassimbalances and/orsignificantdatagapstoforecastimportantspaceweathereventsfroma combinationofsolarimages,solarwind,andgeospaceinsitudata. • Modeling:Thisissomewhatdifferentfromforecastingandinvolvesahigherlevel approachwherethefocusisondiscoveringtheunderlyingphysicalandlong-term behaviorofthesystem.Historically,thisapproachtendstodevelopfromreduced descriptionsbasedonfirstprinciples,butthemethodsofmachinelearningcanintheory alsobeusedtodiscoverthenonlinearmapthatdescribesthesystemevolution. Wewillcertainlyseeincreasingapplicationsofmachinelearninginspacephysicsandspace weather, falling in one of these categories. Yet, we also believe it is still an open question whethertheamountandthekindofdataatourdisposaltodayissufficienttotrainaccurate models. xvi INTRODUCTION SCOPE AND STRUCTURE OF THE BOOK The aim of this book is to bridge the existing gap between space physicists and machine learningpractitioners.Ononehand,standardmachinelearningtechniquesandoff-the-shelf availablesoftwarearenotimmediatelyusefultoalargepartofthespacephysicscommunity thatisnotfamiliarwiththejargonandthepotentialuseofsuchmethods;ontheotherhand, the data science community is eager to apply new techniques to challenging and unsolved problemswithacleartechnologicalimpact,suchasspaceweather. Thefirstpartofthebookisintendedtoprovidesomecontexttothelattercommunitywhich mightnotbefamiliarwithspaceweatherforecasting.Chapter1summarizestheSocietaland EconomicImportanceofSpaceWeather,whileChapter2describestheDataAvailabilityandForecast ProductsforSpaceWeather. The second part offers a short, high-level overview of the three main topics that will be discussed throughout the book: Information Theory (Chapter 3), Regression (Chapter 4), and Classification(Chapter5).Obviously,wereferthereadertomorespecifictextbooksforin-depth explanationoftheseconcepts. Thelastpartisdevotedtoapplicationscoveringabroadrangeofsubdomains. Chapter 6, Untangling the Solar Wind Drivers of Radiation Belt: An Information Theoretical Approach,isconcernedwithanapplicationofinformationtheorytostudytheclassicalproblem of discerning different solar wind input parameters and quantifying their different roles in drivingtheradiationbeltelectrons. Chapter7,EmergenceofDynamicalComplexityintheEarth’sMagnetosphere,tacklestheEarth’s magnetospherecomplexityfromthestandpointofsystemscience,studyingclassicalconcepts such as scale-invariance, self-similarity, and multifractality in the context of the analysis of timeseriesofgeomagneticdata. Chapter 8, Application of NARMAX to Space Weather, reviews the several uses of the methodology based on Nonlinear AutoRegressive Moving Average with eXogenous inputs modelstospaceweather,focusingongeomagneticindicesandradiationbeltelectrons. Chapter 9, Probabilistic Forecasting of Geomagnetic Indices Using Gaussian Process Models, presents an application of Gaussian process (GP) regression with a particular emphasis on modelselectionanddesignchoice.GPcanbeunderstoodinthecontextofBayesianinference, and it is a particularly promising tool for space weather prediction, for its natural ability to provideprobabilisticforecasts. Chapter 10, Prediction of MeV Electron Fluxes With Autoregressive Models, focuses on rela- tivisticelectronsintheradiationbeltsandonrelevantforecastingverificationtechniquesfor autoregressivemodels.Theapproachemployedinthischapterrepresentsaniceexampleofa gray-boxmodelingdiscussedearlier. Chapter11,ArtificialNeuralNetworkforMagnetosphericConditions,discussesanapplication offeed-forwardneuralnetworkstotheproblemsofelectrondensityestimationintheradiation beltandthespecificationofwavesandfluxproperties. Chapter12,ReconstructionofPlasmaElectronDensityFromSatelliteMeasurementViaArtificial NeuralNetworks,isalsoconcernedwiththestudyofradiationbeltelectrondensityvianeural networks, although using a completely different approach to derive input features, and emphasizingmodelselectionandverification.
The list of books you might like
