Statistical Consequences of Fat Tails Real World Preasymptotics, Epistemology, and Applications

445 Pages·2020·27.277 MB·English

by  Nassim Nicholas Taleb

THE TECHNICAL INCERTO COLLECTION
STATISTICAL
CONSEQUENCES OF FAT TAILS
Real World Preasymptotics, Epistemology, and Applications
Nassim Nicholas Taleb
ThisformatisbasedonAndréMiede’sClassicThesis,withadaptationfromArs
Classica.
StatisticalConsequencesofFatTails(TechnicalIncertoCollection)
⃝c NassimNicholasTaleb,2020Allrightsreserved
iii
1
COAUTHORS
PasqualeCirillo(Chapters13,15,and16)
RaphaelDouady(Chapter14)
AndreaFontanari(Chapter13)
HélyetteGeman(Chapter25)
DonaldGeman(Chapter25)
EspenHaug(Chapter22)
TheUniversaInvestmentsteam(Chapter23)
1 Papersrelieduponhereare[45,46,47,94,104,123,161,218,221,222,223,225,226,227,228,236,237,238]
iv
GenealogyoftheIncertoprojectwithlinkstothevariousresearchtraditions.
v
(contfromleftpage).

CONTENTS
Nontechnicalchaptersareindicatedwithastar*; Discussionchaptersareindicatedwithay; adaptationfrom
published("peer-reviewed")paperswithaz.
WhilechaptersareindexedbyArabicnumerals,expositoryandverybriefmini-chapters(halfwaybetweenappendices
andfullchapters)useletterssuchasA,B,etc.
1 prologue(cid:3),† 1
2 glossary, definitions, and notations 5
2.1 GeneralNotationsandFrequentlyUsedSymbols 5
2.2 CatalogueRaisonnéofGeneral&Idiosyncraticconcepts 7
2.2.1 PowerLawClassP 7
2.2.2 LawofLargeNumbers(Weak) 8
2.2.3 TheCentralLimitTheorem(CLT) 8
2.2.4 LawofMediumNumbersorPreasymptotics 8
2.2.5 KappaMetric 8
2.2.6 Ellipticaldistribution 9
2.2.7 Statisticalindependence 9
2.2.8 Stable(Lévystable)Distribution 9
2.2.9 MultivariateStableDistribution 10
2.2.10 Karamatapoint 10
2.2.11 Subexponentiality 10
2.2.12 StudentTasproxy 11
2.2.13 Citationring 11
2.2.14 Rentseekinginacademia 12
2.2.15 Pseudo-empiricismorPinkerProblem 12
2.2.16 Preasymptotics 12
2.2.17 Stochasticizing 12
2.2.18 ValueatRisk,ConditionalVaR 13
2.2.19 Skininthegame 13
2.2.20 MSPlot 13
2.2.21 Maximumdomainofattraction,MDA 14
2.2.22 SubstitutionofIntegralinthepsychologyliterature 14
2.2.23 Inseparabilityofprobability(anothercommonerror) 14
2.2.24 Wittgenstein’sRuler 15
2.2.25 Blackswan 15
2.2.26 Theempiricaldistributionisnotempirical 15
2.2.27 Thehiddentail 16
vii
viii Contents
2.2.28 Shadowmoment 16
2.2.29 Taildependence 17
2.2.30 Metaprobability 17
2.2.31 Dynamichedging 17
i fat tails and their effects, an introduction 19
3 a non-technical overview - the darwin college lecture (cid:3),‡ 21
3.1 OntheDifferenceBetweenThinandThickTails 21
3.2 A(MoreAdvanced)CategorizationandItsConsequences 23
3.3 TheMainConsequencesandHowTheyLinktotheBook 28
3.3.1 Forecasting 38
3.3.2 TheLawofLargeNumbers 39
3.4 EpistemologyandInferentialAsymmetry 41
3.5 Naive Empiricism: Ebola Should Not Be Compared to Falls from
Ladders 46
3.6 PrimeronPowerLaws(almostwithoutmathematics) 48
3.7 WhereAretheHiddenProperties? 50
3.8 BayesianSchmayesian 54
3.9 XvsF(X),exposurestoXconfusedwithknowledgeaboutX 55
3.10 RuinandPathDependence 58
3.11 WhatToDo? 61
4 univariate fat tails, level 1, finite moments† 63
4.1 ASimpleHeuristictoCreateMildlyFatTails 63
4.1.1 AVariance-preservingheuristic 65
4.1.2 FatteningofTailsWithSkewedVariance 66
4.2 DoesStochasticVolatilityGeneratePowerLaws? 68
4.3 TheBody,TheShoulders,andTheTails 69
4.3.1 TheCrossoversandTunnelEffect. 70
4.4 FatTails,MeanDeviationandtheRisingNorms 73
4.4.1 TheCommonErrors 73
4.4.2 SomeAnalytics 74
4.4.3 EffectofFatterTailsonthe"efficiency"ofSTDvsMD 76
4.4.4 MomentsandThePowerMeanInequality 77
4.4.5 Comment:Whyweshouldretirestandarddeviation,now! 80
4.5 Visualizingtheeffectofrising poniso-norms 84
5 level 2: subexponentials and power laws 87
5.0.1 RevisitingtheRankings 87
5.0.2 Whatisaborderlineprobabilitydistribution? 89
5.0.3 LetUsInventaDistribution 90
5.1 Level3: ScalabilityandPowerLaws 91
5.1.1 ScalableandNonscalable,ADeeperViewofFatTails 91
5.1.2 GreySwans 93
5.2 SomePropertiesofPowerLaws 94
5.2.1 Sumsofvariables 94
5.2.2 Transformations 95
5.3 BellShapedvsNonBellShapedPowerLaws 96
5.4 Super-FatTails: TheLog-ParetoDistribution 97
Contents ix
5.5 Pseudo-StochasticVolatility: Aninvestigation 98
6 thick tails in higher dimensions† 101
6.1 ThickTailsinHigherDimension,FiniteMoments 102
6.2 JointFat-TailednessandEllipticalityofDistributions 104
6.3 MultivariateStudentT 106
6.3.1 EllipticalityandIndependenceunderThickTails 106
6.4 FatTailsandMutualInformation 108
6.5 FatTailsandRandomMatrices,aRapidInterlude 108
6.6 CorrelationandUndefinedVariance 109
6.7 FatTailedResidualsinLinearRegressionModels 110
a special cases of thick tails 115
a.1 MultimodalityandThickTails,ortheWarandPeaceModel 115
a.2 TransitionProbabilities: WhatCanBreakWillBreak 119
ii the law of medium numbers 121
7 limit distributions, a consolidation(cid:3),† 123
7.1 Refresher: TheWeakandStrongLLN 123
7.2 CentralLimitinAction 125
7.2.1 TheStableDistribution 125
7.2.2 TheLawofLargeNumbersfortheStableDistribution 126
7.3 SpeedofConvergenceofCLT:VisualExplorations 127
7.3.1 FastConvergence: theUniformDist. 127
7.3.2 Semi-slowconvergence: theexponential 128
7.3.3 TheslowPareto 129
7.3.4 Thehalf-cubicParetoanditsbasinofconvergence 131
7.4 CumulantsandConvergence 131
7.5 TechnicalRefresher: TraditionalVersionsofCLT 133
7.6 TheLawofLargeNumbersforHigherMoments 135
7.6.1 HigherMoments 135
7.7 MeandeviationforaStableDistributions 136
8 how much data do you need? an operational metric for fat-tailedness‡ 139
8.1 IntroductionandDefinitions 140
8.2 TheMetric 142
8.3 StableBasinofConvergenceasBenchmark 144
8.3.1 EquivalenceforStabledistributions 145
8.3.2 Practicalsignificanceforsamplesufficiency 145
8.4 TechnicalConsequences 147
8.4.1 SomeOdditiesWithAsymmetricDistributions 147
8.4.2 RateofConvergenceofaStudentTDistributiontotheGaus-
sianBasin 147
8.4.3 TheLognormalisNeitherThinNorFatTailed 148
8.4.4 CanKappaBeNegative? 148
8.5 ConclusionandConsequences 148
8.5.1 PortfolioPseudo-Stabilization 149
8.5.2 OtherAspectsofStatisticalInference 150
8.5.3 Finalcomment 150
8.6 Appendix,Derivations,andProofs 150
x Contents
8.6.1 CubicStudentT(GaussianBasin) 150
8.6.2 LognormalSums 152
8.6.3 Exponential 154
8.6.4 NegativeKappa,NegativeKurtosis 155
9 extreme values and hidden risk (cid:3),† 157
9.1 PreliminaryIntroductiontoEVT 157
9.1.1 HowAnyPowerLawTailLeadstoFréchet 159
9.1.2 GaussianCase 160
9.1.3 ThePicklands-Balkema-deHaanTheorem 162
9.2 TheInvisibleTailforaPowerLaw 163
9.2.1 ComparisonwiththeNormalDistribution 166
9.3 Appendix: TheEmpiricalDistributionisNotEmpirical 166
b the large deviation principle, in brief 169
c calibrating under paretianity 171
c.1 DistributionofthesampletailExponent 173
10 "it is what it is": diagnosing the sp500† 175
10.1 ParetianityandMoments 175
10.2 ConvergenceTests 177
10.2.1 Test1: KurtosisunderAggregation 177
10.2.2 MaximumDrawdowns 178
10.2.3 EmpiricalKappa 179
10.2.4 Test2: ExcessConditionalExpectation 180
10.2.5 Test3-Instabilityof4th moment 182
10.2.6 Test4: MSPlot 182
10.2.7 RecordsandExtrema 184
10.2.8 Asymmetryright-lefttail 187
10.3 Conclusion: Itiswhatitis 187
d the problem with econometrics 189
d.1 PerformanceofStandardParametricRiskEstimators 190
d.2 PerformanceofStandardNonParametricRiskEstimators 191
e machine learning considerations 195
e.0.1 CalibrationviaAngles 197
iii predictions, forecasting, and uncertainty 199
11 probability calibration calibration under fat tails ‡ 201
11.1 Continuousvs. DiscretePayoffs: DefinitionsandComments 202
11.1.1 AwayfromtheVerbalistic 203
11.1.2 Thereisnodefined"collapse","disaster",or"success"under
fattails 206
11.2 Spuriousoverestimationoftailprobabilityinpsychology 208
11.2.1 Thintails 208
11.2.2 Fattails 209
11.2.3 Conflations 209
11.2.4 DistributionalUncertainty 212
11.3 CalibrationandMiscalibration 213
11.4 ScoringMetrics 213
11.5 ScoringMetrics 215