The Fundamentals of Heavy Tails. Properties, Emergence, and Estimation

266 Pages·2022·7.005 MB·English

by  Jayakrishnan Nair, Adam Wierman, Bert Zwart

The Fundamentals of Heavy Tails
Heavytails–extremeeventsorvaluesmorecommonthanexpected–emergeevery-
where:theeconomy,naturalevents,andsocialandinformationnetworksarejusta
fewexamples.Yetafterdecadesofprogress,theyarestilltreatedasmysterious,sur-
prising,andevencontroversial,primarilybecausethenecessarymathematicalmodels
andstatisticalmethodsarenotwidelyknown.
Thisbook,forthefirsttime,providesarigorousintroductiontoheavy-taileddistri-
butionsaccessibletoanyonewhoknowselementaryprobability.Ittacklesandtames
the zoo of terminology for models and properties, demystifying topics such as the
generalized central limit theorem and regular variation. It tracks the natural emer-
genceofheavy-taileddistributionsfromawidevarietyofgeneralprocesses,building
intuition. And it reveals the controversy surrounding heavy tails to be the result of
flawedstatistics,thenequipsreaderstoidentifyandestimatewithconfidence.Over
100exercisescompletethisengagingpackage.
(cid:3459)(cid:3450)(cid:3474)(cid:3450)(cid:3460)(cid:3467)(cid:3458)(cid:3468)(cid:3457)(cid:3463)(cid:3450)(cid:3463) (cid:3463)(cid:3450)(cid:3458)(cid:3467)isAssociateProfessorinElectricalEngineeringatIITBom-
bay. His research focuses on modeling, performance evaluation, and design issues
in online learning environments, communication networks, queueing systems, and
smartpowergrids.HeistherecipientofbestpaperawardsatIFIPPerformance(2010
and2020)andACMe-Energy(2020).
(cid:3450)(cid:3453)(cid:3450)(cid:3462) (cid:3472)(cid:3458)(cid:3454)(cid:3467)(cid:3462)(cid:3450)(cid:3463)isProfessorofComputingandMathematicalSciencesattheCal-
ifornia Institute of Technology (Caltech). His research develops tools in machine
learning,optimization,control,andeconomicswiththegoalofmakingthenetworked
systemsthatgovernourworldsustainableandresilient.Heisbestknownforhiswork
spearheadingthedesignofalgorithmsforsustainabledatacentersandheistherecip-
ientofnumerousawardsincludingtheACMSigmetricsRisingStaraward,theACM
Sigmetrics Test of Time award, the IEEE Communication Society William Bennet
Prize,andmultipleteachingandbestpaperawards.
(cid:3451)(cid:3454)(cid:3467)(cid:3469) (cid:3475)(cid:3472)(cid:3450)(cid:3467)(cid:3469) is group leader at CWI Amsterdam and Professor of Mathematics
at Eindhoven University of Technology. He has expertise in stochastic operations
research,queueingtheory,andlargedeviations,andinthecontextofheavytails,he
has focused on sample path properties, designing Monte Carlo methods and appli-
cations to computer-communication and energy networks. He was area editor of
OperationsResearch,theflagshipjournalofhisprofession,from2009to2017,and
wastherecipientoftheINFORMSAppliedProbabilitySocietyErlangprize,awarded
everytwoyearstoanoutstandingyoungappliedprobabilist.
CAMBRIDGE SERIES IN STATISTICAL AND
PROBABILISTIC MATHEMATICS
EditorialBoard
Z.Ghahramani(DepartmentofEngineering,UniversityofCambridge)
R.Gill(MathematicalInstitute,LeidenUniversity)
F.P.Kelly(DepartmentofPureMathematicsandMathematicalStatistics,
UniversityofCambridge)
B.D.Ripley(DepartmentofStatistics,UniversityofOxford)
S.Ross(DepartmentofIndustrialandSystemsEngineering,
UniversityofSouthernCalifornia)
M.Stein(DepartmentofStatistics,UniversityofChicago)
Thisseriesofhigh-qualityupper-divisiontextbooksandexpositorymonographscoversallaspects
ofstochasticapplicablemathematics.Thetopicsrangefrompureandappliedstatisticstoprobability
theory,operationsresearch,optimization,andmathematicalprogramming.Thebookscontainclear
presentationsofnewdevelopmentsinthefieldandalsoofthestateoftheartinclassicalmethods.
Whileemphasizingrigoroustreatmentoftheoreticalmethods,thebooksalsocontainapplications
anddiscussionsofnewtechniquesmadepossiblebyadvancesincomputationalpractice.
Acompletelistofbooksintheseriescanbefoundatwww.cambridge.org/statistics..
Recenttitlesincludethefollowing:
27. ModelSelectionandModelAveraging,byGerdaClaeskensandNilsLidHjort
28. BayesianNonparametrics,editedbyNilsLidHjortetal.
29. FromFiniteSampletoAsymptoticMethodsinStatistics,byPranabK.Sen,JulioM.Singer
andAntonioC.PedrosadeLima
30. BrownianMotion,byPeterMörtersandYuvalPeres
31. Probability:TheoryandExamples(FourthEdition),byRickDurrett
32. AnalysisofMultivariateandHigh-DimensionalData,byIngeKoch
33. StochasticProcesses,byRichardF.Bass
34. RegressionforCategoricalData,byGerhardTutz
35. ExercisesinProbability(SecondEdition),byLoïcChaumontandMarcYor
36. StatisticalPrinciplesfortheDesignofExperiments,byR.Mead,S.G.GilmourandA.Mead
37. QuantumStochastics,byMou-HsiungChang
38. Nonparametric Estimation under Shape Constraints, by Piet Groeneboom and
GeurtJongbloed
39. LargeSampleCovarianceMatricesandHigh-DimensionalDataAnalysis,byJianfengYao,
ShurongZhengandZhidongBai
40. Mathematical Foundations of Infinite-Dimensional Statistical Models, by Evarist Giné and
RichardNickl
41. Confidence,Likelihood,Probability,byToreSchwederandNilsLidHjort
42. ProbabilityonTreesandNetworks,byRussellLyonsandYuvalPeres
43. RandomGraphsandComplexNetworks(Volume1),byRemcovanderHofstad
44. Fundamentals of Nonparametric Bayesian Inference, by Subhashis Ghosal and
AadvanderVaart
45. Long-RangeDependenceandSelf-Similarity,byVladasPipirasandMuradS.Taqqu
46. PredictiveStatistics,byBertrandS.ClarkeandJenniferL.Clarke
47. High-DimensionalProbability,byRomanVershynin
48. High-DimensionalStatistics,byMartinJ.Wainwright
49. Probability:TheoryandExamples(FifthEdition),byRickDurrett
50. StatisticalModel-basedClusteringandClassification,byCharlesBouveyronetal.
51. SpectralAnalysisforUnivariateTimeSeries,byDonaldB.PercivalandAndrewT.Walden
52. StatisticalHypothesisTestinginContext,byMichaelP.FayandEricaBrittain

The Fundamentals of Heavy Tails
Properties, Emergence, and Estimation
Jayakrishnan Nair
IndianInstituteofTechnology,Bombay
Adam Wierman
CaliforniaInstituteofTechnology
Bert Zwart
CWIAmsterdamandEindhovenUniversityofTechnology
UniversityPrintingHouse,CambridgeCB28BS,UnitedKingdom
OneLibertyPlaza,20thFloor,NewYork,NY10006,USA
477WilliamstownRoad,PortMelbourne,VIC3207,Australia
314–321,3rdFloor,Plot3,SplendorForum,JasolaDistrictCentre,NewDelhi–110025,India
103PenangRoad,#05-06/07,VisioncrestCommercial,Singapore238467
CambridgeUniversityPressispartoftheUniversityofCambridge.
ItfurtherstheUniversity’smissionbydisseminatingknowledgeinthepursuitof
education,learning,andresearchatthehighestinternationallevelsofexcellence.
www.cambridge.org
Informationonthistitle:www.cambridge.org/9781316511732
DOI:10.1017/9781009053730
©JayakrishnanNair,AdamWierman,andBertZwart2022
Thispublicationisincopyright.Subjecttostatutoryexception
andtotheprovisionsofrelevantcollectivelicensingagreements,
noreproductionofanypartmaytakeplacewithoutthewritten
permissionofCambridgeUniversityPress.
Firstpublished2022
PrintedintheUnitedKingdombyTJBooksLimited,PadstowCornwall
AcataloguerecordforthispublicationisavailablefromtheBritishLibrary.
ISBN978-1-316-51173-2Hardback
CambridgeUniversityPresshasnoresponsibilityforthepersistenceoraccuracyof
URLsforexternalorthird-partyinternetwebsitesreferredtointhispublication
anddoesnotguaranteethatanycontentonsuchwebsitesis,orwillremain,
accurateorappropriate.
Contents
Preface pageix
Acknowledgments xiii
1 Introduction 1
1.1 DefiningHeavy-TailedDistributions 5
1.2 ExamplesofHeavy-TailedDistributions 9
1.3 What’sNext 24
1.4 Exercises 24
PartI Properties 27
2 ScaleInvariance,PowerLaws,andRegularVariation 29
2.1 ScaleInvarianceandPowerLaws 30
2.2 ApproximateScaleInvarianceandRegularVariation 32
2.3 AnalyticPropertiesofRegularlyVaryingFunctions 36
2.4 AnExample:ClosurePropertiesofRegularlyVaryingDistributions 48
2.5 AnExample:BranchingProcesses 50
2.6 AdditionalNotes 53
2.7 Exercises 54
3 Catastrophes,Conspiracies,andSubexponentialDistributions 56
3.1 ConspiraciesandCatastrophes 58
3.2 SubexponentialDistributions 62
3.3 AnExample:Randomsums 67
3.4 AnExample:ConspiraciesandCatastrophesinRandomWalks 72
3.5 AdditionalNotes 80
3.6 Exercises 81
4 ResidualLives,HazardRates,andLongTails 85
4.1 ResidualLivesandHazardRates 86
4.2 HeavyTailsandResidualLives 90
4.3 Long-TailedDistributions 93
4.4 AnExample:RandomExtrema 97
4.5 AdditionalNotes 100
4.6 Exercises 101
v
vi Contents
PartII Emergence 105
5 AdditiveProcesses 107
5.1 TheCentralLimitTheorem 108
5.2 GeneralizingtheCentralLimitTheorem 112
5.3 UnderstandingStableDistributions 114
5.4 TheGeneralizedCentralLimitTheorem 118
5.5 AVariation:TheEmergenceofHeavyTailsinRandomWalks 120
5.6 AdditionalNotes 124
5.7 Exercises 125
6 MultiplicativeProcesses 127
6.1 TheMultiplicativeCentralLimitTheorem 128
6.2 VariationsonMultiplicativeProcesses 131
6.3 AnExample:PreferentialAttachmentandYuleProcesses 138
6.4 AdditionalNotes 144
6.5 Exercises 145
7 ExtremalProcesses 148
7.1 ALimitTheoremforMaxima 150
7.2 UnderstandingMax-StableDistributions 154
7.3 TheExtremalCentralLimitTheorem 156
7.4 AnExample:ExtremesofRandomWalks 161
7.5 AVariation:TheTimebetweenRecordBreakingEvents 168
7.6 AdditionalNotes 170
7.7 Exercises 171
PartIII Estimation 175
8 EstimatingPower-LawDistributions:ListentotheBody 177
8.1 ParametricEstimationofPower-LawsUsingLinearRegression 179
8.2 MaximumLikelihoodEstimationforPower-LawDistributions 185
8.3 PropertiesoftheMaximumLikelihoodEstimator 187
8.4 VisualizingtheMLEviaRegression 189
8.5 ARecipeforParametricEstimationofPower-LawDistributions 192
8.6 AdditionalNotes 194
8.7 Exercises 195
9 EstimatingPower-LawTails:LettheTailDotheTalking 197
9.1 TheFailureofParametricEstimation 199
9.2 TheHillEstimator 203
9.3 PropertiesoftheHillEstimator 205
9.4 TheHillPlot 209
9.5 BeyondHillandRegularVariation 215
9.6 WhereDoestheTailBegin? 226
9.7 GuidelinesforEstimatingHeavy-TailedPhenomena 233
Contents vii
9.8 AdditionalNotes 235
9.9 Exercises 236
CommonlyUsedNotation 238
References 240
Index 249