Uncertainty Quantification using R

768 Pages·2023·37.357 MB·English

by  Eduardo Souza de Cursi

International Series in
Operations Research & Management Science
Eduardo Souza de Cursi
Uncertainty
Quantification
using R
International Series in Operations Research &
Management Science
FoundingEditor
FrederickS.Hillier,StanfordUniversity,Stanford,CA,USA
Volume 335
SeriesEditor
Camille C. Price, Department of Computer Science, Stephen F. Austin State
University,Nacogdoches,TX,USA
AssociateEditor
Joe Zhu, Foisie Business School, Worcester Polytechnic Institute, Worcester, MA,
USA
EditorialBoardMembers
Emanuele Borgonovo, Department of Decision Sciences, Bocconi University,
Milan,Italy
BarryL.Nelson,DepartmentofIndustrialEngineering&ManagementSciences,
NorthwesternUniversity,Evanston,IL,USA
BruceW.Patty,VeritecSolutions,MillValley,CA,USA
MichaelPinedo,SternSchoolofBusiness,NewYorkUniversity,NewYork,NY,
USA
RobertJ.Vanderbei,PrincetonUniversity,Princeton,NJ,USA
The book series International Series in Operations Researchand Management
Science encompasses the various areas of operations research and management
science. Both theoretical and applied books are included. It describes current
advances anywhere intheworld thatareatthecuttingedgeofthefield.Theseries
is aimed especially at researchers, advanced graduate students, and sophisticated
practitioners.
Theseriesfeaturesthreetypesofbooks:
(cid:129)Advancedexpositorybooksthatextendandunifyourunderstandingofpartic-
ularareas.
(cid:129)Researchmonographsthatmakesubstantialcontributionstoknowledge.
(cid:129) Handbooks that define the new state of the art in particular areas. Each
handbook will be edited by a leading authority in the area who will organize
ateamofexpertsonvariousaspectsofthetopictowriteindividualchapters.A
handbook may emphasize expository surveys or completely new advances
(eitherresearchorapplications)oracombinationofboth.
Theseriesemphasizesthefollowingfourareas:
MathematicalProgramming: Including linearprogramming, integerprogram-
ming, nonlinear programming, interior point methods, game theory, network opti-
mization models, combinatorics, equilibrium programming, complementarity
theory, multiobjective optimization, dynamic programming, stochastic program-
ming,complexitytheory,etc.
Applied Probability: Including queuing theory, simulation, renewal theory,
Brownian motion and diffusion processes, decision analysis, Markov decision
processes, reliability theory, forecasting, other stochastic processes motivated by
applications,etc.
ProductionandOperationsManagement:Includinginventorytheory,produc-
tion scheduling, capacity planning, facility location, supply chain management,
distributionsystems,materialsrequirementsplanning,just-in-timesystems,flexible
manufacturing systems, design of production lines, logistical planning, strategic
issues,etc.
Applications of Operations Research and Management Science: Including
telecommunications,healthcare,capitalbudgetingandfinance,economics,market-
ing, public policy, military operations research, humanitarian relief and disaster
mitigation,serviceoperations,transportationsystems,etc.
ThisbookseriesisindexedinScopus.
Eduardo Souza de Cursi
fi
Uncertainty Quanti cation
using R
EduardoSouzadeCursi
DepartmentMechanics/CivilEngineering
INSARouenNormandie
Saint-EtienneduRouvray,France
ISSN0884-8289 ISSN2214-7934 (electronic)
InternationalSeriesinOperationsResearch&ManagementScience
ISBN978-3-031-17784-2 ISBN978-3-031-17785-9 (eBook)
https://doi.org/10.1007/978-3-031-17785-9
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNatureSwitzerland
AG2023
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseof
illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by
similarordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors, and the editorsare safeto assume that the adviceand informationin this
bookarebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Introduction
This book presentsa collection of methods of uncertainty quantification (UQ) – id
est, a collection of methods for the analysis of numerical data, namely when
uncertaintyorvariabilityisinvolved.
The general aim of UQ is to characterize the observed variability in a quantity
XbyusingarandomvariableU.Intheidealsituation,theconnectionbetweenXand
U is perfectly known and the random variable U has a known distribution. Unfor-
tunately, such a situation may be unrealistic in practice and we must also consider
situations where this knowledge is imperfect or even non-existent – for example,
situations where U is simply unknown: variability is observed without precise
knowledgeofthecause.
UQ tries touse all the available information about (X,U) toconstruct an expla-
nationofXbyU,intoaformwhichwillbeusefulforuseinnumericalcalculations
involvingX.Theinformationmaybe,forinstance,anequation,anumericalproblem
involvingboththevariables,orsamples.
ThemethodsofUQaregeneralandmaybeappliedtoawiderangeofsituations.
They generally belong to the large and well-supplied family of methods based on
functional representations, id est, on expansions of the unknowns in series of
functions – we find these approaches in particular in Fourier analysis, spectral
methods, finite elements, Bayesian optimization, quantum algorithms, etc. It is a
largefamilywithnumerousandverydiversifiedapplications.
Our objective is to present the practical use of UQ techniques under R. We
assumethatyouareameanuserofthiskindofsoftware:ifyouareauserofScilab,
®
Octave,orMatlab ,youwillrecognizetheinstructionsandthecodespresentedwill
appearasfamiliar.Evidently,ifyouareanexpertinR,youwillfindalargenumber
of improvements in our codes and programs: do not hesitate in making your own
enhancementsand,eventually,insharingthem.
RisaGNUprojecttodevelopatoolforlanguageandenvironmentforstatistical
computingandgraphics.AnIDEisproposedbyRStudio.ThepopularityofRand
RStudio is no more to be demonstrated: you will find on the web many sites and
v
vi Introduction
information about R. A wide literature can be found about this software. The
community of the users of R proposes a wide choice of packages to extend the
possibilities of R. You will find repositories containing them, for instance, https://
search.r-project.org/R/doc/html/packages.html#lib-1
Contents
1 SomeTipstoUseRandRStudio. . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 HowtoInstallRandRStudio. . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 HowtoIncludeaThird-PartAdd-In. . . . . . . . . . . . . . . . . . . . . . 2
1.3 HowtoCreateaDocumentwithRStudio. . . . . . . . . . . . . . . . . . 4
1.4 HowtoCreateaScriptwithRStudio. . . . . . . . . . . . . . . . . . . . . 5
1.5 HowtoManipulateNumericVariables,Vectors,andFactors. . . . 9
1.6 HowtoManipulateMatricesandArrays. . . . . . . . . . . . . . . . . . 19
1.7 HowtoUseLists. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
1.8 Usingdata.frames. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.9 PlottingwithRStudio. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
1.10 ProgrammingwithR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
1.11 ClassesinR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
1.12 HowtoSolveDifferentialEquationswithR. . . . . . . . . . . . . . . . 61
1.12.1 InitialValueProblemsforOrdinary
DifferentialEquations.. . . . .. . . . .. . . .. . . . .. . . . .. 62
1.12.2 BoundaryValueProblemsforOrdinary
DifferentialEquations.. . . . .. . . . .. . . .. . . . .. . . . .. 64
1.13 OptimizationwithR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
1.13.1 LinearProgramming. . . . . . . . . . . . . . . . . . . . . . . . . . 69
1.13.2 NonlinearProgramming. . . . . . . . . . . . . . . . . . . . . . . . 71
1.13.3 DualityMethods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
1.13.4 MultiobjectiveOptimization. . . . . . . . . . . . . . . . . . . . . 79
1.14 SolvingEquationswithR. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
1.14.1 SystemsofLinearEquations. . . . . . . . . . . . . . . . . . . . . 83
1.14.2 SystemsofNonlinearEquations. . . . . . . . . . . . . . . . . . 85
1.14.3 OptimizationandSystemsofEquations. . . . . . . . . . . . . 86
1.15 InterpolationandApproximationwithR. . . . . . . . . . . . . . . . . . . 92
1.15.1 VariationalApproximation. . . . . . . . . . . . . . . . . . . . . . 94
1.15.2 SmoothedParticleApproximation. . . . . . . . . . . . . . . . . 96
vii
viii Contents
1.16 IntegralsandDerivativeswithR. . . . . . . . . . . . . . . . . . . . . . . . 100
1.16.1 VariationalApproximationoftheDerivatives. . . . . . .. 104
1.16.2 SmoothedParticleApproximationoftheDerivative. . . . 106
2 ProbabilitiesandRandomVariables. . . . . . . . . . . . . . . . . . . . . . . . . 109
2.1 Notation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
2.2 Probability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
2.2.1 MassFunctionsandMassDensities. . . . . . . . . . . . . . . 114
2.2.2 CombinatorialProbabilities. . . . . . . . . . . . . . . . . . . . . . 119
2.3 IndependentEvents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122
2.4 NumericalVariablesonFinitePopulations. . . . . . . . . . . . . . . . . 126
2.4.1 CouplesofNumericalVariables. . . . . . . . . . . . . . . . . . 137
2.4.2 IndependentNumericalVariables. . . . . . . . . . . . . . . . . 144
2.5 NumericalVariablesasElementsofHilbertSpaces. . . . . . . . . . . 146
2.5.1 ConditionalProbabilitiesasOrthogonalProjections. . . . 149
2.5.2 MeansasOrthogonalProjections. . . . . . . . . . . . . . . . . 150
2.5.3 AffineApproximationsandCorrelations. . . . . . . . . . . . 150
2.5.4 ConditionalMean. . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
2.6.1 NumericalEvaluationofStatistics. . . . . . . . . . . . . . . . . 167
2.7 RandomVectors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 170
2.8 DiscreteandContinuousRandomVariables. . . . . . . . . . . . . . . . 180
2.8.1 DiscreteVariables. . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
2.8.2 ContinuousVariablesHavingaPDF. . . . . . . . . . . . . . . 185
2.9 SequencesofRandomVariables. . . . . . . . . . . . . . . . . . . . . . . . 194
2.10 Samples. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
2.10.1 Maximum-LikelihoodEstimators. . . . . . . . . . . . . . . . . 208
2.10.2 SamplesfromRandomVectors. . . . . . . . . . . . . . . . . . . 212
2.10.3 EmpiricalCDFandEmpiricalPDF. . . . . . . . . . . . . . . . 214
2.10.4 TestingAdequacyofaSampletoaDistribution. . . . . . . 217
2.10.5 TestingtheIndependenceaCoupleofVariables. . . . .. 226
2.12 GeneratingRandomNumbersbyInversion. . . . . .. . . . . . . . . . . 230
2.13 GeneratingRandomVectorswithaGivenCovarianceMatrix. . . 232
2.14 GeneratingRegularRandomFunctions. . . . . . . . . . . . . . . . . . . 235
2.15 GeneratingRegularRandomCurves. . . . . . . . . . . . . . . . . . . . . 243
3 RepresentationofRandomVariables. . . . . . . . . . . . . . . . . . . . . . . . 251
3.1 TheUQApproachfortheRepresentationofRandomVariables. . 251
3.2 Collocation. . . . . .. . . . . .. . . . . . .. . . . . . .. . . . . .. . . . . . .. 257
3.3 VariationalApproximation. . . . . . . . . . . . . . . . . . . . . . . . . . . . 290
3.4.1 TheStandardFormulationofM3. . . . . . . . . . . . . . . . . 300
3.4.2 AlternativeFormulationsofM3. . . . . . . . . . . . . . . . . . 308
3.5 MultidimensionalExpansions. . . . . . . . . . . . . . . . . . . . . . . . . . 317
3.5.1 CaseWhereUIsMultidimensional. . . . . . . . . . . . . . . . 317
3.5.2 CaseWhereXIsMultidimensional. . . . . . . . . . . . . . . . 320
3.6 RandomFunctions. . .... .... .... .... .... ... .... .... . 323
Contents ix
3.7 RandomCurves. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 332
3.8 Mean,Variance,andConfidenceIntervalsforRandom
FunctionsorRandomCurves. . . . . . . . . . . . .. . . . . . . . . . . .. . 342
4 StochasticProcesses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
4.1 Ergodicity. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 361
4.2 DeterminationoftheDistributionofaStationaryProcess. . . . . . 387
4.3 WhiteNoise. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 393
4.4 MovingAverageProcesses. . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
4.5 AutoregressiveProcesses. . . . . . . . . . . . . .. . . . . . . . . . . . .. . . 411
4.6 ARMAProcesses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426
4.7 MarkovProcesses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 437
4.8 DiffusionProcesses.. . . . . . . . . .. . . . . . . . . . .. . . . . . . . . .. . 449
4.8.1 TimeIntegralandDerivativeofaProcess. . . . . . . . . . . 449
4.8.2 SimulationoftheTimeIntegralofaWhiteNoise. . . . . . 455
4.8.3 BrownianMotion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 460
4.8.4 RandomWalks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 467
4.8.5 Itô’sIntegrals. . . .. . . . . .. . . . .. . . . .. . . . .. . . . . .. 470
4.8.6 Itô’sCalculus. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 477
4.8.7 NumericalSimulationofStochastic
Differentialequations. . . . . . . . . . . . . . . . . . . . . . . . . . 484
5 UncertainAlgebraicEquations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503
5.1 UncertainLinearSystems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511
5.1.1 VerySmallLinearSystems. . . . . . . . . . . . . . . . . . . . . . 518
5.2 NonlinearEquationsandAdaptationofanIterativeCode. . . . .. 523
5.3 IterativeEvaluationofEigenvalues. . . . . . . . . . . . . . . . . . . . . . 541
5.3.1 VerySmallMatrices. .. . . .. . . .. . . .. . . .. . . .. . . .. 548
5.4 TheVariationalApproachforUncertainAlgebraicEquations. . . 550
6 RandomDifferentialEquations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 559
6.1 LinearDifferentialEquations. . . . . . . . . . . . . . . . . . . . . . . . . . . 563
6.2 NonlinearDifferentialEquations. . . . . . . . . . . . . . . . . . . . . . . . 575
6.3 AdaptationofODESolvers. . . . . . . . . . . . . . . . . . . . . . . . . . . . 581
6.4 UncertaintiesonCurvesConnectedtoDifferentialEquations. . . . 583
7 UQinGameTheory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 591
7.1 TheLanguagefromGameTheory. . . . . . . . . . . . . . . . . . . . . . . 591
7.2 ASimplifiedOddsandEvensGame. . . . . . . . . . . . . . . . . . . . . 593
7.2.1 GTStrategiesWhenp=(p ,p )IsKnown. . . . . . . . . . 595
1 2
7.2.2 StrategiesWhenpIsUnknown. . . . . . . . . . . . . . . . . . . 598
7.2.3 StrategiesfortheStochasticGame. . . . . . . . . . . . . . . . 603
7.2.4 ReplicatorDynamics. . . . . . . . . . . . . . . . . . . . . . . . . . 605
7.3 ThePrisoner’sDilemma. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612
7.3.1 ReplicatorDynamics. . . . . . . . . . . . . . . . . . . . . . . . . . 614
7.4 TheGoalie’sAnxietyatthePenaltyKick. . . . . . . . . . . . . . . . . . 619
7.5 HawksandDoves. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626