Probability with Applications in Engineering, Science, and Technology

664 Pages·2017·13.127 MB·English

by  Matthew A. Carlton, Jay L. Devore (auth.)

Springer Texts in Statistics
Matthew A. Carlton
Jay L. Devore
Probability with
Applications in
Engineering,
Science, and
Technology
Second Edition
Springer Texts in Statistics
SeriesEditors
R.DeVeaux
S.E.Fienberg
I.Olkin
Moreinformationabout thisseries athttp://www.springer.com/series/417
Matthew A. Carlton (cid:129) Jay L. Devore
Probability with Applications
in Engineering, Science,
and Technology
Second Edition
Editors
MatthewA.Carlton JayL.Devore
DepartmentofStatistics DepartmentofStatistics
CaliforniaPolytechnicStateUniversity CaliforniaPolytechnicStateUniversity
SanLuisObispo,CA,USA SanLuisObispo,CA,USA
Additionalmaterialtothisbookcanbedownloadedfromhttp://extras.springer.com.
ISSN1431-875X ISSN2197-4136 (eBook)
SpringerTextsinStatistics
ISBN978-3-319-52400-9 ISBN978-3-319-52401-6(eBook)
DOI10.1007/978-3-319-52401-6
LibraryofCongressControlNumber:2017932278
#SpringerInternationalPublishingAG2014, 2017
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeor
part of the material is concerned, specifically the rights of translation, reprinting, reuse of
illustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,
andtransmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,
orbysimilarordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthis
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesare
exemptfromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationin
thisbookarebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernor
the authors or the editors give a warranty, express or implied, with respect to the material
containedhereinorforanyerrorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
ThisSpringerimprintispublishedbySpringerNature
TheregisteredcompanyisSpringerInternationalPublishingAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
Purpose
Ourobjectiveistoprovideapost-calculusintroductiontothesubjectofprobabilitythat
• Hasmathematicalintegrityandcontainssomeunderlyingtheory
• Showsstudentsabroadrangeofapplicationsinvolvingrealproblemscenarios
• Iscurrentinitsselectionoftopics
• Isaccessibletoawideaudience,includingmathematicsandstatisticsmajors(yes,thereareafew
of the latter, and their numbers are growing), prospective engineers and scientists, and business
andsocialsciencemajorsinterestedinthequantitativeaspectsoftheirdisciplines
• Illustrates the importance of software for carrying out simulations when answers to questions
cannotbeobtainedanalytically
A number of currently available probability texts are heavily oriented toward a rigorous mathe-
maticaldevelopmentofprobability,withmuchemphasisontheorems,proofs,andderivations.Even
when applied material is included, the scenarios are often contrived (many examples and exercises
involving dice, coins, cards, and widgets). So in our exposition we have tried to achieve a balance
betweenmathematicalfoundationsandtheapplicationofprobabilitytoreal-worldproblems.Itisour
beliefthatthetheoryofprobabilitybyitselfisoftennotenoughofa“hook”togetstudentsinterested
in further work in the subject. We think that the best way to persuade students to continue their
probabilisticeducationbeyondafirstcourseistoshowthemhowthemethodologyisusedinpractice.
Let’s first seduce them (figuratively speaking, of course) with intriguing problem scenarios and
applications.Opportunitiesforexposuretomathematicalrigorwillfollowinduecourse.
Content
ThebookbeginswithanIntroduction,whichcontainsourattempttoaddressthefollowingquestion:
“Why study probability?” Here we are trying to tantalize students with a number of intriguing
problem scenarios—coupon collection, birth and death processes, reliability engineering, finance,
queuingmodels,andvariousconundrumsinvolvingthemisinterpretationofprobabilisticinformation
(e.g., Benford’s Law and the detection of fraudulent data, birthday problems, and the likelihood of
having a rare disease when a diagnostic test result is positive). Most of the exposition contains
references torecentlypublishedresults.It isnotnecessaryoreven desirable tocover verymuchof
thismotivationalmaterialintheclassroom.Instead,wesuggestthatinstructorsasktheirstudentsto
readselectivelyoutsideclass(abitofpleasurereadingattheverybeginningofthetermshouldnotbe
v
vi Preface
anundueburden!).Subsequentchaptersmakelittlereferencetotheexamplesherein,andseparating
outour“peptalk”shouldmakeiteasiertocoveraslittleormuchasaninstructordeemsappropriate.
Chapter 1 covers sample spaces and events, the axioms of probability and derived properties,
counting,conditionalprobability,andindependence.Discreterandomvariablesanddistributionsare
the subject of Chap.2, and Chap.3 introduces continuous random variablesand their distributions.
Joint probability distributions are the focus of Chap. 4, including marginal and conditional
distributions, expectation of a function of several variables, correlation, modes of convergence, the
CentralLimitTheorem,reliabilityofsystemsofcomponents,thedistributionofalinearcombination,
andsomeresultsonorderstatistics.Thesefourchaptersconstitutethecoreofthebook.
The remaining chapters build on the core in various ways. Chapter 5 introduces methods of
statistical inference—point estimation, the use of statistical intervals, and hypothesis testing. In
Chap.6wecoverbasicpropertiesofdiscrete-timeMarkovchains. Various otherrandomprocesses
and their properties, including stationarity and its consequences, Poisson processes, Brownian
motion, and continuous-time Markov chains, are discussed in Chap. 7. The final chapter presents
someelementaryconceptsandmethodsintheareaofsignalprocessing.
Onefeatureofourbookthatdistinguishesitfromthecompetitionisasectionattheendofalmost
everychapterthatconsiderssimulationmethodsforgettingapproximateanswerswhenexactresults
aredifficultorimpossibletoobtain.BoththeRsoftwareandMatlabareemployedforthispurpose.
Another noteworthy aspect of the book is the inclusion of roughly 1100 exercises; the first four
core chapters together have about 700 exercises. There are numerous exercises at the end of each
section and also supplementary exercises at the end of every chapter. Probability at its heart is
concernedwithproblemsolving.Astudentcannothopetoreallylearnthematerialsimplybysitting
passively in the classroom and listening to the instructor. He/she must get actively involved in
working problems. To this end, we have provided a wide spectrum of exercises, ranging from
straightforwardtoreasonablychallenging.Itshouldbeeasyforaninstructortofindenoughproblems
atvariouslevelsofdifficultytokeepstudentsgainfullyoccupied.
Mathematical Level
Thechallengeforstudentsatthislevelshouldbetomastertheconceptsandmethodstoasufficient
degree that problems encountered inthereal worldcan besolved.Mostofourexercises areofthis
type,andrelativelyfewaskforproofsorderivations.Consequently,themathematicalprerequisites
anddemandsarereasonablymodest.Mathematicalsophisticationandquantitativereasoningability
are,ofcourse,crucialtotheenterprise.Univariatecalculusisemployedinthecontinuousdistribution
calculations of Chap. 3 as well as in obtaining maximum likelihood estimators in the inference
chapter. But even here the functions we ask students to work with are straightforward—generally
polynomials, exponentials, and logs. A stronger background is required for the signal processing
material at the end of the book (we have included a brief mathematical appendix as a refresher for
relevantproperties).MultivariatecalculusisusedinthesectiononjointdistributionsinChap.4and
thereafterappearsratherrarely.ExposuretomatrixalgebraisneededfortheMarkovchainmaterial.
Recommended Coverage
Our book contains enough material for a year-long course, though we expect that many instructors
will use it for a single term (one semester or one quarter). To give a sense of what might be
reasonable,wenowbrieflydescribethreecoursesatourhomeinstitution,CalPolyStateUniversity
(inSanLuisObispo,CA),forwhichthisbookisappropriate.Syllabiwithexpandedcourseoutlines
areavailablefordownloadonthebook’swebsiteatSpringer.com.
Preface vii
Title: Introductionto IntroductiontoProbability ProbabilityandRandomProcesses
Probabilityand Models forEngineers
Simulation
Main Statisticsandmath Statisticsandmathmajors Electricalandcomputer
audience: majors engineeringmajors
Prerequisites: Univariatecalculus, Univariatecalculus,computer Multivariatecalculus,continuous-
computer programming,matrixalgebra timesignalsincl.Fourieranalysis
programming
Sections 1.1–1.6 1.1–1.6 1.1–1.5
covered:
2.1–2.6,2.8 2.1–2.5,2.8 2.1–2.5
3.1–3.4,3.8 3.1–3.4,3.8 3.1–3.5
4.1–4.3,4.5 4.1–4.3,4.5,4.8 4.1–4.3,4.5,4.7
6.1–6.5 7.1–7.3,7.5–7.6
7.5 8.1–8.2
Bothofthefirsttwocoursesplaceheavyemphasisoncomputersimulationofrandomphenomena;
instructors typically have students work in R. As is evident from the lists of sections covered,
IntroductiontoProbabilityModelstakestheearliermaterialatafasterpaceinordertoleaveafew
weeksattheendforMarkovchainsandsomeotherapplications(typicallyreliabilitytheoryandabit
aboutPoissonprocesses).Inourexperience,thecomputerprogrammingprerequisiteisessentialfor
students’successinthosetwocourses.
The third course listed, Probability and Random Processes for Engineers, is our university’s
versionofthetraditional“randomsignalsandnoise”courseofferedbymanyelectricalengineering
departments. Again, the first four chapters are covered at a somewhat accelerated pace, with about
30–40%ofthecoursededicatedtotimeandfrequencyrepresentationsofrandomprocesses(Chaps.7
and 8).Simulation of random phenomena is notemphasized inour course, though we make liberal
useofMatlabfordemonstrations.
Weareabletocoverasmuchmaterialasindicatedontheforegoingsyllabiwiththeaidofanot-
so-secretweapon:weprepareandrequirethatstudentsbringtoclassacoursebooklet.Thebooklet
containsmostoftheexampleswepresentaswellassomesurroundingmaterial.Atypicalexample
beginswithaproblemstatementandthenposesseveralquestions(asintheexercisesinthisbook).
Aftereachposedquestionthereissomeblankspacesothestudentcaneithertakenotesasthesolution
isdevelopedinclassorelseworktheproblemonhis/herownifaskedtodoso.Becausestudentshave
abooklet,theinstructordoesnothavetowriteasmuchontheboardaswouldotherwisebenecessary
andthestudentdoesnothavetodoasmuchwritingtotakenotes.Boththeinstructorandthestudents
benefit.
Wealsoliketothinkthatstudentscanbeaskedtoreadanoccasionalsubsectionorevensectionon
theirownandthenworkexercisestodemonstrateunderstanding,sothatnoteverythingneedstobe
presented in class. For example, we have found that assigning a take-home exam problem that
requiresreadingabouttheWeibulland/orlognormaldistributionsisagoodwaytoacquaintstudents
withthem.Butinstructorsshouldalwayskeepinmindthatthereisneverenoughtimeinacourseof
anydurationtoteachstudentsallthatwe’dlikethemtoknow.Hopefullystudentswilllikethebook
enough to keep it after the course is over and use it as a basis for extending their knowledge of
probability!
viii Preface
Acknowledgments
We gratefully acknowledge the plentiful feedback provided by the following reviewers: Allan Gut,
MuradTaqqu,MarkSchillingandRobertHeiny.
WeverymuchappreciatetheproductionservicesprovidedbythefolksatSPiTechnologies.Our
productioneditors,Sasireka.KandMariaDaviddidafirst-ratejobofmovingthebookthroughthe
productionprocess andwere always promptandconsiderateincommunicationswithus.Thanksto
our copyeditors at SPi for employing a light touch and not taking us too much to task for our
occasionalgrammaticalandstylisticlapses.ThestaffatSpringerU.S.hasbeenespeciallysupportive
duringboththedevelopmentalandproductionstages;specialkudosgotoMichaelPennandRebekah
McClure.
A Final Thought
Itisourhopethatstudentscompletingacoursetaughtfromthisbookwillfeelaspassionatelyabout
thesubjectofprobabilityaswestilldoaftersomanyyearsoflivingwithit.Onlyteacherscanreally
appreciate how gratifying it is to hear from a student after he/she has completed a course that the
experiencehadapositiveimpactandmaybeevenaffectedacareerchoice.
SanLuisObispo,CA MatthewA.Carlton
SanLuisObispo,CA JayL.Devore
Contents
1 Probability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 SampleSpacesandEvents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 TheSampleSpaceofanExperiment. . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1.2 Events. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1.3 SomeRelationsfromSetTheory. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.4 Exercises:Section1.1(1–12). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 Axioms,Interpretations,andPropertiesofProbability. .. . . . . . . . . . . . . .. . . . . 7
1.2.1 InterpretingProbability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.2 MoreProbabilityProperties. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
1.2.3 DeterminingProbabilitiesSystematically. . . . . . . . . . . . . . . . . . . . . . . 13
1.2.4 EquallyLikelyOutcomes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.2.5 Exercises:Section1.2(13–30). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
1.3 CountingMethods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.3.1 TheFundamentalCountingPrinciple. . . . . . . . . . . . . . . . . . . . . . . . . . 18
1.3.2 TreeDiagrams. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.3.3 Permutations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.3.4 Combinations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22
1.3.5 Exercises:Section1.3(31–49). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
1.4 ConditionalProbability. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.4.1 TheDefinitionofConditionalProbability. . . . . . . . . . . . . . . . . . . . . . . 30
1.4.2 TheMultiplicationRuleforP(A \ B). . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.4.3 TheLawofTotalProbabilityandBayes’Theorem. . . . . . . . . . . . . . . . 34
1.4.4 Exercises:Section1.4(50–78). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
1.5 Independence. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
1.5.1 P(A \ B)WhenEventsAreIndependent. . . . . . . . . . . . . . . . . . . . . . . . 44
1.5.2 IndependenceofMorethanTwoEvents. . . . . . . . . . . . . . . . . . . . . . . . 45
1.5.3 Exercises:Section1.5(79–100). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
1.6 SimulationofRandomEvents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
1.6.1 TheBackboneofSimulation:RandomNumberGenerators. . . . . . . . . . 51
1.6.2 PrecisionofSimulation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
1.6.3 Exercises:Section1.6(101–120). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
1.7 SupplementaryExercises(121–150). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2 DiscreteRandomVariablesandProbabilityDistributions. . . . . . . . . . . . . . . . . . . . 67
2.1 RandomVariables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.1.1 TwoTypesofRandomVariables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69
2.1.2 Exercises:Section2.1(1–10). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
ix
x Contents
2.2 ProbabilityDistributionsforDiscreteRandomVariables. .. . . . .. . . . .. . . . .. . 71
2.2.1 AParameterofaProbabilityDistribution. . . . . . . . . . . . . . . . . . . . . . . 74
2.2.2 TheCumulativeDistributionFunction.. . . .. . .. . . .. . . .. . . .. . . .. . 75
2.2.3 AnotherViewofProbabilityMassFunctions. . . . . . . . . . . . . . . . . . . . . 78
2.2.4 Exercises:Section2.2(11–28). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
2.3 ExpectedValueandStandardDeviation.. . . . . . . . .. . . . . . . . .. . . . . . . . .. . . 83
2.3.1 TheExpectedValueofX. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
2.3.2 TheExpectedValueofaFunction. . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
2.3.3 TheVarianceandStandardDeviationofX. . . . . . . . . . . . . . . . . . . . . . 88
2.3.4 PropertiesofVariance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
2.3.5 Exercises:Section2.3(29–48). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
2.4 TheBinomialDistribution. . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . .. . . . . . . . . 95
2.4.1 TheBinomialRandomVariableandDistribution. . . . . . . . . . . . . . . . . . 97
2.4.2 ComputingBinomialProbabilities. . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
2.4.3 TheMeanandVarianceofaBinomialRandomVariable. . . . . . . . . . . . 101
2.4.4 BinomialCalculationswithSoftware. . . . . . . . . . . . . . . . . . . . . . . . . . 102
2.4.5 Exercises:Section2.4(49–74). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102
2.5 ThePoissonDistribution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
2.5.1 ThePoissonDistributionasaLimit. . . . . . . . . . . . . . . . . . . . . . . . . . . 107
2.5.2 TheMeanandVarianceofaPoissonRandomVariable. . . . . . . . . . . . . 110
2.5.3 ThePoissonProcess. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
2.5.4 PoissonCalculationswithSoftware. . . . . . . .. . . . . . . . . . . . . . . . . . . . 111
2.5.5 Exercises:Section2.5(75–89). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
2.6 OtherDiscreteDistributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
2.6.1 TheHypergeometricDistribution. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 114
2.6.2 TheNegativeBinomialandGeometricDistributions. . . . . . . . . . . . . . . 117
2.6.3 AlternativeDefinitionoftheNegativeBinomialDistribution. . . . . . . . . 120
2.6.4 Exercises:Section2.6(90–106). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
2.7 MomentsandMomentGeneratingFunctions. . . . . . . . . . . . . . . . . . . . . . . . . . . 123
2.7.1 TheMomentGeneratingFunction. . . . . . . . . .. . . . . . . . . . . . . .. . . . . 125
2.7.2 ObtainingMomentsfromtheMGF. . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
2.7.3 MGFsofCommonDistributions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
2.7.4 Exercises:Section2.7(107–128). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
2.8 SimulationofDiscreteRandomVariables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131
2.8.1 SimulationsImplementedinRandMatlab. . . . . . . . . . . . . . . . . . . . . . 134
2.8.2 SimulationMean,StandardDeviation,andPrecision. . . . . . . . . . . . . . . 135
2.8.3 Exercises:Section2.8(129–141). . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
2.9 SupplementaryExercises(142–170). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
3 ContinuousRandomVariablesandProbabilityDistributions. . . . . . . . . . . . . . . . . 147
3.1 ProbabilityDensityFunctionsandCumulativeDistributionFunctions. . . . . . . . . 147
3.1.1 ProbabilityDistributionsforContinuousVariables.. . . . . . .. . . . . . .. . 148
3.1.2 TheCumulativeDistributionFunction.. . . .. . .. . . .. . . .. . . .. . . .. . 152
3.1.3 UsingF(x)toComputeProbabilities. . . . . . . . . . . . . . . . . . . . . . . . . . . 154
3.1.4 Obtainingf(x)fromF(x). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
3.1.5 PercentilesofaContinuousDistribution. . . . . . . . . . . . . . . . . . . . . . . . 156
3.1.6 Exercises:Section3.1(1–18). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
3.2 ExpectedValuesandMomentGeneratingFunctions. . . . . . . . . . . . . . . .. . . . . . 162
3.2.1 ExpectedValues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
3.2.2 MomentGeneratingFunctions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
3.2.3 Exercises:Section3.2(19–38). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168