Table Of ContentSpringer Undergraduate Texts
in Mathematics and Technology
Series Editors
Jonathan M. Borwein
Helge Holden
Editorial Board
Lisa Goldberg
Armin Iske
Palle E.T. Jorgensen
Stephen M. Robinson
Mario Lefebvre
Basic Probability Theory with
Applications
Mario Lefebvre
Département de mathématiques et de génie industriel
École Polytechnique de Montréal, Québec
C.P. 6079, succ. Centre-ville
Montréal H3C 3A7
Canada
mlefebvre@polymtl.ca
SeriesEditors
JonathanM.Borwein HelgeHolden
FacultyofComputerScience DepartmentofMathematicalSciences
DalhousieUniversity NorwegianUniversityofScienceand
Halifax,NovaScotiaB3H1W5 Technology
Canada AlfredGetzvei1
jborwein@cs.dal.ca NO-7491Trondheim
Norway
holden@math.ntnu.no
ISBN 978-0-387-74994-5 e-ISBN 978-0-387-74995-2
DOI 10.1007/978-0-387-74995-2
Springer Dordrecht Heidelberg London New York
Library of Congress Control Number: 2009928845
Mathematics Subject Classification (2000): 60-01
© Springer Science+Business Media, LLC 2009
All rights reserved. This work may not be translated or copied in whole or in part without the written permission of
the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York, NY 10013, USA), except for
brief excerpts in connection with reviews or scholarly analysis. Use in connection with any form of information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known
or hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not
identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary
rights.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)
To the memory of my father
I will never believe that God plays dice with the universe.
Albert Einstein
Then they gave lots to them, and the lot fell upon Matthias,
and he was counted with the eleven apostles.
Acts 1: 26
Preface
The main intended audience for this book is undergraduate students in pure and
applied sciences, especially those in engineering. Chapters 2 to 4 cover the probability
theory they generally need in their training. Although the treatment of the subject is
surely sufficient for non-mathematicians, I intentionally avoided getting too much into
detail. For instance, topics such as mixed type random variables and the Dirac delta
function are only briefly mentioned.
Courses on probability theory are often considered difficult. However, after having
taughtthissubjectformanyyears,Ihavecometotheconclusionthatoneofthebiggest
problems that the students face when they try to learn probability theory, particularly
nowadays, is their deficiencies in basic differential and integral calculus. Integration by
parts, for example, is often already forgotten by the students when they take a course
on probability. For this reason, I have decided to write a chapter reviewing the basic
elementsofdifferentialcalculus.Eventhoughthischaptermightnotbecoveredinclass,
thestudentscanrefertoitwhenneeded.Inthischapter,aneffortwasmadetogivethe
readers a good idea of the use in probability theory of the concepts they should already
know.
Chapter 2 presents the main results of what is known as elementary probability,
including Bayes’ rule and elements of combinatorial analysis. Although these notions
are not mathematically complicated, it is often a chapter that the students find hard
to master. There is no trick other than doing a lot of exercises to become comfortable
with this material.
Chapter 3 is devoted to the more technical subject of random variables. All the
important models for the applications, such as the binomial and normal distributions,
are introduced. In general, the students do better when examined on this subject and
feel that their work is more rewarded than in the case of combinatorial analysis, in
particular.
Random vectors, including the all-important central limit theorem, constitute the
subject of Chapter 4. I have endeavored to present the material as simply as possible.
Nevertheless,itisobviousthatdoubleintegralscannotbesimplerthansingleintegrals.
ApplicationsofChapters2to4arepresentedinChapters5to7.First,Chapter5is
devotedtotheimportantsubjectofreliabilitytheory,whichisusedinmostengineering
disciplines,inparticularinmechanicalengineering.Next,thebasicqueueingmodelsare
studiedinChapter6.Queueingtheoryisneededformanycomputerscienceengineering
students, as well as for those in industrial engineering. Finally, the last application
considered, in Chapter 7, is the concept of time series. Civil engineers, notably those
specialized in hydrology, make use of stochastic processes of this type when they want
to model various phenomena and forecast the future values of a given variable, such as
theflowofariver.Timeseriesarealsowidelyusedineconomyandfinancetorepresent
the variations of certain indices.
Preface VII
Nomatterthelevelandthebackgroundofthestudentstakingacourseonprobability
theory, one thing is always true: as mentioned above, they must try to solve many
exercisesbeforetheycanfeelthattheyhavemasteredthetheory.Tothisend,thebook
contains more than 400 exercises, many of which are multiple part questions. At the
end of each chapter, the reader will find some solved exercises, whose solutions can be
found in Appendix C, followed by a large number of unsolved exercises. Answers to the
even-numberedquestionsareprovidedinAppendixDattheendofthebook.Thereare
also many multiple choice questions, whose answers are given in Appendix E.
It is my pleasure to thank all the people I worked with over the years at the E´cole
Polytechnique de Montr´eal and who provided me with interesting exercises that were
included in this work.
Finally, I wish to express my gratitude to Vaishali Damle, and the entire publishing
team at Springer, for their excellent support throughout this book project.
Mario Lefebvre
Montr´eal, July 2008
Contents
Preface................................................................ vi
ListofTables ......................................................... xiii
ListofFigures ........................................................ xv
1 Review of differential calculus ..................................... 1
1.1 Limits and continuity ............................................ 1
1.2 Derivatives ..................................................... 3
1.3 Integrals ....................................................... 7
1.3.1 Particular integration techniques ............................ 9
1.3.2 Double integrals........................................... 12
1.4 Infinite series ................................................... 14
1.4.1 Geometric series........................................... 15
1.5 Exercises for Chapter 1 .......................................... 18
2 Elementary probability ............................................ 27
2.1 Random experiments ............................................ 27
2.2 Events ......................................................... 28
2.3 Probability ..................................................... 29
2.4 Conditional probability........................................... 32
2.5 Total probability ................................................ 35
2.6 Combinatorial analysis ........................................... 36
2.7 Exercises for Chapter 2 .......................................... 39
3 Random variables ................................................. 55
3.1 Introduction .................................................... 55
3.1.1 Discrete case.............................................. 55
3.1.2 Continuous case........................................... 57
3.2 Important discrete random variables ............................... 61
3.2.1 Binomial distribution ...................................... 61
3.2.2 Geometric and negative binomial distributions ................ 64
3.2.3 Hypergeometric distribution ................................ 66
3.2.4 Poisson distribution and process............................. 68
3.3 Important continuous random variables ............................ 70
X Contents
3.3.1 Normal distribution ....................................... 70
3.3.2 Gamma distribution ....................................... 74
3.3.3 Weibull distribution ....................................... 77
3.3.4 Beta distribution .......................................... 78
3.3.5 Lognormal distribution..................................... 80
3.4 Functions of random variables..................................... 81
3.4.1 Discrete case.............................................. 81
3.4.2 Continuous case........................................... 82
3.5 Characteristics of random variables ................................ 83
3.6 Exercises for Chapter 3 .......................................... 94
4 Random vectors ................................................... 115
4.1 Discrete random vectors.......................................... 115
4.2 Continuous random vectors ....................................... 118
4.3 Functions of random vectors ...................................... 124
4.3.1 Discrete case.............................................. 125
4.3.2 Continuous case........................................... 127
4.3.3 Convolutions ............................................. 128
4.4 Covariance and correlation coefficient.............................. 131
4.5 Limit theorems.................................................. 135
4.6 Exercises for Chapter 4 .......................................... 137
5 Reliability ......................................................... 161
5.1 Basic notions ................................................... 161
5.2 Reliability of systems ............................................ 170
5.2.1 Systems in series .......................................... 170
5.2.2 Systems in parallel ........................................ 172
5.2.3 Other cases............................................... 176
5.3 Paths and cuts .................................................. 178
5.4 Exercises for Chapter 5 .......................................... 183
6 Queueing .......................................................... 191
6.1 Continuous-time Markov chains ................................... 191
6.2 Queueing systems with a single server.............................. 197
6.2.1 The M/M/1 model........................................ 199
6.2.2 The M/M/1 model with finite capacity ...................... 207
6.3 Queueing systems with two or more servers ......................... 212
6.3.1 The M/M/s model ........................................ 212
6.3.2 The M/M/s/c model ...................................... 218
6.4 Exercises for Chapter 6 .......................................... 220
Description:This book presents elementary probability theory with interesting and well-chosen applications that illustrate the theory. An introductory chapter reviews the basic elements of differential calculus which are used in the material to follow. The theory is presented systematically, beginning with the