Probability, Statistics and Simulation: With Application Programs Written in R

643 Pages·2022·10.372 MB·English

by  Alberto Rotondi, Paolo Pedroni, Antonio Pievatolo

UNITEXT 139
Alberto Rotondi · Paolo Pedroni
Antonio Pievatolo
Probability,
Statistics
and Simulation
With Application Programs
Written in R
UNITEXT
+
La Matematica per il 3 2
Volume 139
Editor-in-Chief
AlfioQuarteroni,PolitecnicodiMilano,Milan,Italy;ÉcolePolytechniqueFédérale
deLausanne(EPFL),Lausanne,Switzerland
SeriesEditors
LuigiAmbrosio,ScuolaNormaleSuperiore,Pisa,Italy
PaoloBiscari,PolitecnicodiMilano,Milan,Italy
CiroCiliberto,UniversitàdiRoma“TorVergata”,Rome,Italy
CamilloDeLellis,InstituteforAdvancedStudy,Princeton,NewJersey,USA
MassimilianoGubinelli,HausdorffCenterforMathematics,Rheinische
Friedrich-Wilhelms-Universität,Bonn,Germany
VictorPanaretos,InstituteofMathematics,ÉcolePolytechniqueFédéralede
Lausanne(EPFL),Lausanne,Switzerland
LorenzoRosasco,DIBRIS,UniversitàdegliStudidiGenova,Genova,Italy;
CenterforBrainsMindandMachines,MassachusettsInstituteofTechnology,
Cambridge,Massachusetts,USA;IstitutoItalianodiTecnologia,Genova,Italy
TheUNITEXT-LaMatematicaperil3+2seriesisdesignedforundergraduate
andgraduateacademiccourses,andalsoincludesadvancedtextbooksataresearch
level.
Originally released in Italian, the series now publishes textbooks in English
addressedtostudentsinmathematicsworldwide.
Some of the most successful books in the series have evolved through several
editions,adaptingtotheevolutionofteachingcurricula.
Submissions must include at least 3 sample chapters, a table of contents, and
a preface outlining the aims and scope of the book, how the book fits in with the
currentliterature,andwhichcoursesthebookissuitablefor.
For any further information, please contact the Editor at Springer:
[email protected]
THESERIESISINDEXEDINSCOPUS
***
UNITEXT is glad to announce a new series of free webinars and interviews
handledbytheBoardmembers,whowillrotateinordertointerviewtopexpertsin
theirfield.
In the first session, going live on June 9, Alfio Quarteroniwill interview Luigi
Ambrosio. The speakers will dive into the subject of Optimal Transport, and will
discuss the most challenging open problems and the future developments in the
field.
Clickheretosubscribetotheevent!
https://cassyni.com/events/TPQ2UgkCbJvvz5QbkcWXo3
Alberto Rotondi • Paolo Pedroni •
Antonio Pievatolo
Probability, Statistics
and Simulation
With Application Programs Written in R
AlbertoRotondi PaoloPedroni
DipartimentodiFisica IstitutoNazionalediFisicaNucleare
UniversitàdiPavia UniversitàdiPavia
Pavia,Italy Pavia,Italy
AntonioPievatolo
IstitutodiMatematicaApplicatae
TecnologieInformatiche
ConsiglioNazionaledelleRicerche
Milano,Italy
ISSN2038-5714 ISSN2532-3318 (electronic)
UNITEXT
ISSN2038-5722 ISSN2038-5757 (electronic)
LaMatematicaperil3+2
ISBN978-3-031-09428-6 ISBN978-3-031-09429-3 (eBook)
https://doi.org/10.1007/978-3-031-09429-3
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNatureSwitzerland
AG2022
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewhole orpart ofthematerial isconcerned, specifically therights oftranslation, reprinting, reuse
ofillustrations, recitation, broadcasting, reproductiononmicrofilmsorinanyotherphysicalway,and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
Coverillustration:“Thefacenumberthreeandtwonumbersthree,onechanceover1326”(photobythe
authors)
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Thisbook,basedonthefourthItalianedition,comesfromthecollaborationbetween
twoexperimentalphysicistsandonestatistician.Amongnon-statisticians,physicists
areperhapstheoneswhomostappreciateanduseprobabilityandstatistics,butmost
ofthetimeinapragmaticandmanualway,havinginmindthesolutionofspecific
problems or technical applications. On the other hand, in the crucial comparison
between theory and experiment, it is sometimes necessary to use sophisticated
methods which require knowledge of the fundamental logical and mathematical
principles at the basis of the study of random phenomena. More generally, even
those who are not statisticians have often to face, in any research field, problems
thatrequireparticularattentionandexpertiseforthetreatmentofrandomoraleatory
aspects. These skills are naturally mastered by the statistician, whose research
interestsarethelawsofchance.
This text has been prepared with the aim to seek a synthesis between these
different approaches, to provide the reader not only with tools useful to address
problems,butalsowithaguidetothecorrectmethodologiesneededtounderstand
thecomplicatedandfascinatingworldofrandomphenomena.
Such an objectiveobviouslyinvolvedchoices,sometimesevenpainful,bothof
content type and style. As for style, we tried not to give up the precision needed
toproperlyteachtheimportantconcepts.Whentreatingapplications,weprivileged
themethodsthatdonotrequireexcessivepreliminaryconceptualelaborations.
As an example,we have tried to use, wheneverpossible, approximatemethods
for interval estimation, with Gaussian approximations for the estimator distribu-
tions. Similarly, in the case of least squares, we have extensively adopted the
approximationbased on the χ2 distribution to verify the fitting of a model to the
data.
We also avoided insisting on the formal treatment of complicated problems in
cases where a solution using computer and simple simulation programs could be
easilyfound.
Inourbook,simulationplaysanimportantroleinthepresentationofmanytopics
andintheverificationoftheaccuracyofmanytechniquesandapproximations.This
v
vi Preface
feature, already present in the first Italian edition, is now common to many data
sciencetextsand,inouropinion,confirmsthevalidityofourinitialchoice.
Thisbookisaimedprimarilyatstudentsofscientificundergraduatecourses,such
as engineering,computerscience, and physics.However,we thinkthat it can also
be useful to all those scientific researchers who have to solve practical problems
involvingprobabilistic,statistical,andsimulationaspects.Forthisreason,wehave
given space to some topics, such as Monte Carlo methods and their applications,
minimizationtechniques,anddataanalysismethods,which,usually,areonlybriefly
mentionedinintroductorytexts.
The mathematical knowledge required by the reader is that which is normally
givenintheteachingofthebasiccalculuscourseinthescientificdegrees,withthe
additionof minimumnotionsof linear algebraand advancedcalculus,such asthe
elementaryconceptsofthederivationandintegrationofmultidimensionalfunctions.
Thestructureofthetextallowsdifferentlearningpathsandreadinglevels.The
first seven chapters deal with all the topics usually developedin a standard, basic
statisticalcourse.Atthechoiceoftheteacher,thisprogramcanbeintegratedwith
somemoreadvancedtopicsfromthe otherchapters.Forexample,Chap.8 should
certainlybeincludedinasimulation-orientedcourse.
The notions of probability and statistics usually taught to physics students in
undergraduatelaboratorycoursesareenclosedinthefirstthreechapters,inChaps.6
and7(basicstatistics) andinChap.12,writtenexplicitlyforphysicistsandforall
thosewhoneedtoprocessdatafromlaboratoryexperiments.
Manypagesaredevotedtothecompleteresolutionofseveralexercisesinserted
directlyinsidethechapterstobetterexplainthecoveredtopics.Wealsorecommend
tothereadertheproblems(allwithsolutions)reportedattheendofeachchapter.
This book makes use of the statistical software R, which has now become the
world standard for solving statistical problems. The 2019 ranking of the Institute
ofAmericanElectricalandElectronicEngineers(IEEE)placesRinfourthposition
amongthemostpopularprogramminglanguages,afterPython,Java,andC.Many
Rroutineshavebeenwrittenbyus,toguidethereaderwhilegoingthroughthetext.
Theseroutinescanbeeasilydownloadedfromthelinkspecifiedbelow.Wetherefore
recommendaninteractivereading,inwhichthestudyofatopicisfollowedbythe
useofRroutinesinthewayshowedbothinthetextandinthetechnicalinstructions
includedintheindicatedWebpages.
We thank again the readers who reported errors or inaccuracies present in the
previousItalianeditions,andthepublisher,Springer,forthecontinuedtrustinour
work.
Pavia,Italy AlbertoRotondi
Pavia,Italy PaoloPedroni
Milano,Italy AntonioPievatolo
March2022
How to Use the Text
Figures, equations, definitions, theorems, Tables, and exercises are numbered
progressively.
The abbreviationsof quotations(e.g.,[57])refer to the bibliographiclist at the
endofthebook.
Solutions of the problems are given in Appendix D. The table of symbols
reportedinAppendixAmayalsobeuseful.
Calculation codes as hist are marked with a different text style. Routines
starting with a lowercase letter are (with some exceptions) the original R codes,
whichcanbe freelycopiedfromtheCRAN (ComprehensiveRArchiveNetwork)
website,whilethosestartingwithanuppercaseletterarewrittenbytheauthorsand
arein:
https://tinyurl.com/ProbStatSimul
Inthissite,youwillalsofindalltheinformationfortheinstallationandtheuseof
R,aguidetotheuseofroutineswrittenbytheauthorsandcomplementaryteaching
materials.
vii
Contents
1 Probability................................................................... 1
1.1 Chance,ChaosandDeterminism .................................. 1
1.2 SomeBasicTerms .................................................. 8
1.3 TheConceptofProbability ........................................ 10
1.4 AxiomaticProbability .............................................. 13
1.5 RepeatedTrials ..................................................... 19
1.6 ElementsofCombinatorialAnalysis .............................. 23
1.7 Bayes’Theorem .................................................... 26
1.8 LearningAlgorithms ............................................... 33
1.9 Problems ............................................................ 36
2 RepresentationofRandomPhenomena.................................. 39
2.1 Introduction......................................................... 39
2.2 RandomVariables .................................................. 40
2.3 CumulativeorDistributionFunction .............................. 44
2.4 DataRepresentation ................................................ 46
2.5 DiscreteRandomVariables ........................................ 49
2.6 BinomialDistribution .............................................. 51
2.7 ContinuousRandomVariables .................................... 54
2.8 Mean,SumofSquares,Variance,StandardDeviation
andQuantiles ....................................................... 57
2.9 Operators ........................................................... 64
2.10 SimpleRandomSample ........................................... 67
2.11 ConvergenceCriteria ............................................... 69
2.12 Problems ............................................................ 73
3 BasicProbabilityTheory .................................................. 75
3.1 Introduction......................................................... 75
3.2 PropertiesoftheBinomialDistribution ........................... 75
3.3 PoissonDistribution ................................................ 79
3.4 NormalorGaussianDensity ....................................... 81
ix
x Contents
3.5 TheThree-SigmaLaw and the StandardGaussian
Density .............................................................. 87
3.6 CentralLimitTheoremandUniversalityoftheGaussian
Curve ................................................................ 91
3.7 PoissonStochasticProcesses ...................................... 94
3.8 χ2Density .......................................................... 101
3.9 UniformDensity ................................................... 108
3.10 Chebyshev’sInequality ............................................ 112
3.11 HowtoUseProbabilityCalculus .................................. 113
3.12 Problems ............................................................ 122
4 MultivariateProbabilityTheory.......................................... 125
4.1 Introduction......................................................... 125
4.2 MultivariateStatisticalDistributions .............................. 126
4.3 CovarianceandCorrelation ........................................ 134
4.4 Two-DimensionalGaussianDistribution ......................... 139
4.5 TheGeneralMultidimensionalCase .............................. 148
4.6 MultivariateProbabilityRegions .................................. 155
4.7 MultinomialDistribution .......................................... 159
4.8 Problems ............................................................ 161
5 FunctionsofRandomVariables........................................... 163
5.1 Introduction......................................................... 163
5.2 FunctionsofaRandomVariable .................................. 165
5.3 FunctionsofSeveralRandomVariables .......................... 168
5.4 MeanandVarianceTransformation ............................... 184
5.5 MeansandVariancesfornVariables ............................. 190
5.6 Problems ............................................................ 197
6 BasicStatistics:ParameterEstimation .................................. 199
6.1 Introduction......................................................... 199
6.2 ConfidenceIntervals ............................................... 201
6.3 ConfidenceIntervalswithPivotalVariables ...................... 204
6.4 MentionoftheBayesianApproach ............................... 207
6.5 SomeNotations .................................................... 208
6.6 ProbabilityEstimation ............................................. 209
6.7 ProbabilityEstimationfromLargeSamples ...................... 212
6.8 PoissonianIntervalEstimation .................................... 218
6.9 MeanEstimationfromLargeSamples ............................ 222
6.10 VarianceEstimationfromLargeSamples ......................... 224
6.11 MeanandVarianceEstimationforGaussianSamples ........... 229
6.12 HowtoUsetheEstimationTheory ................................ 232
6.13 EstimatesfromaFinitePopulation ................................ 239
6.14 HistogramAnalysis ................................................ 242
6.15 EstimationoftheCorrelation ...................................... 248
6.16 Problems ............................................................ 257