A First Course in Random Matrix Theory

371 Pages·2020·3.895 MB·English

by  Marc Potters

A FIRST COURSE IN RANDOM MATRIX THEORY
The real world is perceived and broken down as data, models and algorithms in the eyes
of physicists and engineers. Data is noisy by nature and classical statistical tools have so
far been successful in dealing with relatively smaller levels of randomness. The recent
emergenceofBigDataandtherequiredcomputingpowertoanalyzethemhaverendered
classicaltoolsoutdatedandinsufficient.Toolssuchasrandommatrixtheoryandthestudy
oflargesamplecovariancematricescanefficientlyprocessthesebigdatasetsandhelpmake
senseofmodern,deeplearningalgorithms.Presentinganintroductorycalculuscoursefor
random matrices, the book focuses on modern concepts in matrix theory, generalizing
the standard concept of probabilistic independence to non-commuting random variables.
Concretely worked out examples and applications to financial engineering and portfolio
constructionmakethisuniquebookanessentialtoolforphysicists,engineers,dataanalysts
andeconomists.
marc potters isChiefInvestmentOfficerofCFM,aninvestmentfirmbasedinParis.
Marcmaintainsstronglinkswithacademiaand,asanexpertinrandommatrixtheory,he
hastaughtatUCLAandSorbonneUniversity.
jean-philippe bouchaud isapioneerineconophysics.Hisresearchincludesran-
dommatrixtheory,statisticsofpriceformation,stockmarketfluctuations,andagent-based
modelsforfinancialmarketsandmacroeconomics.HispreviousbooksincludeTheoryof
Financial Risk and Derivative Pricing (Cambridge University Press, 2003) and Trades,
Quotes and Prices (Cambridge University Press, 2018), and he has been the recipient of
severalprestigious,internationalawards.

A FIRST COURSE IN RANDOM
MATRIX THEORY
for Physicists, Engineers and Data Scientists
MARC POTTERS
CapitalFundManagement,Paris
JEAN-PHILIPPE BOUCHAUD
CapitalFundManagement,Paris
UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
314–321,3rdFloor,Plot3,SplendorForum,JasolaDistrictCentre,NewDelhi–110025,India
79AnsonRoad,#06–04/06,Singapore079906
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781108488082
DOI:10.1017/9781108768900
©CambridgeUniversityPress2021
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2021
PrintedintheUnitedKingdombyTJBooksLimited
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
LibraryofCongressCataloging-in-PublicationData
Names:Potters,Marc,1969–author.|Bouchaud,Jean-Philippe,1962–author.
Title:Afirstcourseinrandommatrixtheory:forphysicists,engineers
anddatascientists/MarcPotters,Jean-PhilippeBouchaud.
Description:Cambridge;NewYork,NY:CambridgeUniversityPress,2021.|
Includesbibliographicalreferencesandindex.
Identifiers:LCCN2020022793(print)|LCCN2020022794(ebook)|
ISBN9781108488082(hardback)|ISBN9781108768900(epub)
Subjects:LCSH:Randommatrices.
Classification:LCCQA196.5.P682021(print)|LCCQA196.5(ebook)|
DDC512.9/434–dc23
LCrecordavailableathttps://lccn.loc.gov/2020022793
LCebookrecordavailableathttps://lccn.loc.gov/2020022794
ISBN978-1-108-48808-2Hardback
Additionalresourcesforthistitleatwww.cambridge.org/potters
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracy
ofURLsforexternalorthird-partyinternetwebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
Contents
Preface pageix
ListofSymbols xiv
PartI ClassicalRandomMatrixTheory 1
1 DeterministicMatrices 3
1.1 Matrices,EigenvaluesandSingularValues 3
1.2 SomeUsefulTheoremsandIdentities 9
2 WignerEnsembleandSemi-CircleLaw 15
2.1 NormalizedTraceandSampleAverages 16
2.2 TheWignerEnsemble 17
2.3 ResolventandStieltjesTransform 19
3 MoreonGaussianMatrices* 30
3.1 OtherGaussianEnsembles 30
3.2 MomentsandNon-CrossingPairPartitions 36
4 WishartEnsembleandMarcˇenko–PasturDistribution 43
4.1 WishartMatrices 43
4.2 Marcˇenko–PasturUsingtheCavityMethod 48
5 JointDistributionofEigenvalues 58
5.1 FromMatrixElementstoEigenvalues 58
5.2 CoulombGasandMaximumLikelihoodConfigurations 64
5.3 Applications:Wigner,WishartandtheOne-CutAssumption 69
5.4 FluctuationsAroundtheMostLikelyConfiguration 73
5.5 AnEigenvalueDensitySaddlePoint 78
6 EigenvaluesandOrthogonalPolynomials* 83
6.1 WignerMatricesandHermitePolynomials 83
6.2 LaguerrePolynomials 87
6.3 UnitaryEnsembles 91
v
vi Contents
7 TheJacobiEnsemble* 97
7.1 PropertiesofJacobiMatrices 97
7.2 JacobiMatricesandJacobiPolynomials 102
PartII SumsandProductsofRandomMatrices 109
8 AdditionofRandomVariablesandBrownianMotion 111
8.1 SumsofRandomVariables 111
8.2 StochasticCalculus 112
9 DysonBrownianMotion 121
9.1 DysonBrownianMotionI:PerturbationTheory 121
9.2 DysonBrownianMotionII:Itoˆ Calculus 124
9.3 TheDysonBrownianMotionfortheResolvent 126
9.4 TheDysonBrownianMotionwithaPotential 129
9.5 Non-IntersectingBrownianMotionsandtheKarlin–McGregorFormula 133
10 AdditionofLargeRandomMatrices 136
10.1 AddingaLargeWignerMatrixtoanArbitraryMatrix 136
10.2 GeneralizationtoNon-WignerMatrices 140
10.3 TheRank-1HCIZIntegral 142
10.4 InvertibilityoftheStieltjesTransform 145
10.5 TheFull-RankHCIZIntegral 149
11 FreeProbabilities 155
11.1 AlgebraicProbabilities:SomeDefinitions 155
11.2 AdditionofCommutingVariables 156
11.3 Non-CommutingVariables 161
11.4 FreeProduct 170
12 FreeRandomMatrices 177
12.1 RandomRotationsandFreeness 177
12.2 R-TransformsandResummedPerturbationTheory 181
12.3 TheCentralLimitTheoremforMatrices 183
12.4 FiniteFreeConvolutions 186
12.5 Freenessfor2×2Matrices 193
13 TheReplicaMethod* 199
13.1 StieltjesTransform 200
13.2 ResolventMatrix 204
13.3 Rank-1HCIZandReplicas 209
13.4 Spin-Glasses,ReplicasandLow-RankHCIZ 215
Contents vii
14 EdgeEigenvaluesandOutliers 220
14.1 TheTracy–WidomRegime 221
14.2 AdditiveLow-RankPerturbations 223
14.3 FatTails 229
14.4 MultiplicativePerturbation 231
14.5 PhaseRetrievalandOutliers 234
PartIII Applications 241
15 AdditionandMultiplication:RecipesandExamples 243
15.1 Summary 243
15.2 R-andS-TransformsandMomentsofUsefulEnsembles 245
15.3 Worked-OutExamples:Addition 249
15.4 Worked-OutExamples:Multiplication 252
16 ProductsofManyRandomMatrices 257
16.1 ProductsofManyFreeMatrices 257
16.2 TheFreeLog-Normal 261
16.3 AMultiplicativeDysonBrownianMotion 262
16.4 TheMatrixKestenProblem 264
17 SampleCovarianceMatrices 267
17.1 SpatialCorrelations 267
17.2 TemporalCorrelations 271
17.3 TimeDependentVariance 276
17.4 EmpiricalCross-CovarianceMatrices 278
18 BayesianEstimation 281
18.1 BayesianEstimation 281
18.2 EstimatingaVector:RidgeandLASSO 288
18.3 BayesianEstimationoftheTrueCovarianceMatrix 295
19 EigenvectorOverlapsandRotationallyInvariantEstimators 297
19.1 EigenvectorOverlaps 297
19.2 RotationallyInvariantEstimators 301
19.3 PropertiesoftheOptimalRIEforCovarianceMatrices 309
19.4 ConditionalAverageinFreeProbability 310
19.5 RealData 311
19.6 ValidationandRIE 317
20 ApplicationstoFinance 321
20.1 PortfolioTheory 321
20.2 TheHigh-DimensionalLimit 325
20.3 TheStatisticsofPriceChanges:AShortOverview 330
20.4 EmpiricalCovarianceMatrices 334
viii Contents
Appendix MathematicalTools 339
A.1 SaddlePointMethod 339
A.2 Tricomi’sFormula 341
A.3 ToeplitzandCirculantMatrices 343
Index 347
Preface
Physicists have always approached the world through data and models inspired by this
data. They build models from data and confront their models with the data generated by
newexperimentsorobservations.Butrealdataisbynaturenoisy;untilrecently,classical
statisticaltoolshavebeensuccessfulindealingwiththisrandomness.Therecentemergence
of very large datasets, together with the computing power to analyze them, has created a
situationwherenotonlythenumberofdatapointsislargebutalsothenumberofstudied
variables.Classicalstatisticaltoolsareinadequatetotacklethissituation,calledthelarge
dimension limit (or the Kolmogorov limit). Random matrix theory, and in particular the
studyoflargesamplecovariancematrices,canhelpmakesenseofthesebigdatasets,and
isinfactalsobecomingausefultooltounderstanddeeplearning.Randommatrixtheory
isalsolinkedtomanymodernproblemsinstatisticalphysicssuchasthespectraltheoryof
randomgraphs,interactionmatricesofspin-glasses,non-intersectingrandomwalks,many-
bodylocalization,compressedsensingandmanymore.
Thisbookcanbeconsidered asonemorebookonrandommatrixtheory.Butouraim
was to keep it purposely introductory and informal. As an analogy, high school seniors
andcollegefreshmenaretypicallytaughtbothcalculusandanalysis.Inanalysisonelearns
howtomakerigorousproofs,definealimitandaderivative.Atthesametimeincalculus
one can learn about computing complicated derivatives, multi-dimensional integrals and
solving differential equations relying only on intuitive definitions (with precise rules) of
these concepts. This book proposes a “calculus” course for random matrices, based in
particularontherelativelynewconceptof“freeness”,thatgeneralizesthestandardconcept
ofprobabilisticindependencetonon-commutingrandomvariables.
Rather than make statements about the most general case, concepts are defined with
somestronghypothesis(e.g.Gaussianentries,realsymmetricmatrices)inordertosimplify
thecomputationsandfavorunderstanding.Precisenotionsofnorm,topology,convergence,
exactdomainofapplicationareleftout,againtofavorintuitionoverrigor.Therearemany
good,mathematicallyrigorousbooksonthesubject(seereferencesbelow)andthehopeis
that our book will allow the interested reader to read them guided by his/her newly built
intuition.
ix