Probability, statistics, and random processes for electrical engineering

833 Pages·4.559 MB·English

by  Leon-Garcia, Alberto

Probability, Statistics,
and Random Processes
for Electrical Engineering
Third Edition
Alberto Leon-Garcia
University of Toronto
Upper Saddle River,NJ 07458
Library of Congress Cataloging-in-Publication Data
Leon-Garcia,Alberto.
Probability,statistics,and random processes for electrical engineering / Alberto Leon-Garcia.-- 3rd ed.
p.cm.
Includes bibliographical references and index.
ISBN-13:978-0-13-147122-1 (alk.paper)
1. Electric engineering--Mathematics.2. Probabilities.3. Stochastic processes. I.Leon-Garcia,Alberto.Probability
and random processes for electrical engineering.II.Title.
TK153.L425 2007
519.202'46213--dc22
2007046492
Vice President and Editorial Director,ECS:Marcia J.Horton
Associate Editor:Alice Dworkin
Editorial Assistant:William Opaluch
Senior Managing Editor:Scott Disanno
Production Editor:Craig Little
Art Director:Jayen Conte
Cover Designer:Bruce Kenselaar
Art Editor:Greg Dulles
Manufacturing Manager:Alan Fischer
Manufacturing Buyer:Lisa McDowell
Marketing Manager:Tim Galligan
© 2008 Pearson Education,Inc.
Pearson Prentice Hall
Pearson Education,Inc.
Upper Saddle River,NJ 07458
All rights reserved.No part of this book may be reproduced,in any form or by any means,without permission in
writing from the publisher.
Pearson Prentice HallTMis a trademark of Pearson Education,Inc.MATLABis a registered trademark of The Math
Works,Inc.All other product or brand names are trademarks or registered trademarks of their respective holders.
The author and publisher of this book have used their best efforts in preparing this book.These efforts include the
development,research,and testing of the theories and programs to determine their effectiveness.The author and
publisher make no warranty of any kind,expressed or implied,with regard to the material contained in this book.The
author and publisher shall not be liable in any event for incidental or consequential damages in connection with,or
arising out of,the furnishing,performance,or use of this material.
Printed in the United States of America
10 9 8 7 6 5 4 3 2 1
ISBN 0-13-147122-8
978-0-13-147122-1
Pearson Education Ltd.,London
Pearson Education Australia Pty.Ltd.,Sydney
Pearson Education Singapore,Pte.Ltd.
Pearson Education North Asia Ltd.,Hong Kong
Pearson Education Canada,Inc.,Toronto
Pearson Educación de Mexico,S.A.de C.V.
Pearson Education—Japan,Tokyo
Pearson Education Malaysia,Pte.Ltd.
Pearson Education,Upper Saddle River,New Jersey
TOKAREN,CARLOS,MARISA,ANDMICHAEL.
This page intentionally left blank
Contents
Preface ix
CHAPTER 1 Probability Models in Electrical
and Computer Engineering 1
1.1 Mathematical Models as Tools in Analysis and Design 2
1.2 Deterministic Models 4
1.3 Probability Models 4
1.4 A Detailed Example:A Packet Voice Transmission System 9
1.5 Other Examples 11
1.6 Overview of Book 16
Summary 17
Problems 18
CHAPTER 2 Basic Concepts of Probability Theory 21
2.1 Specifying Random Experiments 21
2.2 The Axioms of Probability 30
*2.3 Computing Probabilities Using Counting Methods 41
2.4 Conditional Probability 47
2.5 Independence of Events 53
2.6 Sequential Experiments 59
*2.7 Synthesizing Randomness:Random Number Generators 67
*2.8 Fine Points:Event Classes 70
*2.9 Fine Points:Probabilities of Sequences of Events 75
Summary 79
Problems 80
CHAPTER 3 Discrete Random Variables 96
3.1 The Notion of a Random Variable 96
3.2 Discrete Random Variables and Probability Mass Function 99
3.3 Expected Value and Moments of Discrete Random Variable 104
3.4 Conditional Probability Mass Function 111
3.5 Important Discrete Random Variables 115
3.6 Generation of Discrete Random Variables 127
Summary 129
Problems 130
v
vi Contents
CHAPTER 4 One Random Variable 141
4.1 The Cumulative Distribution Function 141
4.2 The Probability Density Function 148
4.3 The Expected Value of X 155
4.4 Important Continuous Random Variables 163
4.5 Functions of a Random Variable 174
4.6 The Markov and Chebyshev Inequalities 181
4.7 Transform Methods 184
4.8 Basic Reliability Calculations 189
4.9 Computer Methods for Generating Random Variables 194
*4.10 Entropy 202
Summary 213
Problems 215
CHAPTER 5 Pairs of Random Variables 233
5.1 Two Random Variables 233
5.2 Pairs of Discrete Random Variables 236
5.3 The Joint cdf of XandY 242
5.4 The Joint pdf of Two Continuous Random Variables 248
5.5 Independence of Two Random Variables 254
5.6 Joint Moments and Expected Values of a Function of Two Random
Variables 257
5.7 Conditional Probability and Conditional Expectation 261
5.8 Functions of Two Random Variables 271
5.9 Pairs of Jointly Gaussian Random Variables 278
5.10 Generating Independent Gaussian Random Variables 284
Summary 286
Problems 288
CHAPTER 6 Vector Random Variables 303
6.1 Vector Random Variables 303
6.2 Functions of Several Random Variables 309
6.3 Expected Values of Vector Random Variables 318
6.4 Jointly Gaussian Random Vectors 325
6.5 Estimation of Random Variables 332
6.6 Generating Correlated Vector Random Variables 342
Summary 346
Problems 348
Contents vii
CHAPTER 7 Sums of Random Variables and Long-Term Averages 359
7.1 Sums of Random Variables 360
7.2 The Sample Mean and the Laws of Large Numbers 365
Weak Law of Large Numbers 367
Strong Law of Large Numbers 368
7.3 The Central Limit Theorem 369
Central Limit Theorem 370
*7.4 Convergence of Sequences of Random Variables 378
*7.5 Long-Term Arrival Rates and Associated Averages 387
7.6 Calculating Distribution’s Using the Discrete Fourier
Transform 392
Summary 400
Problems 402
CHAPTER 8 Statistics 411
8.1 Samples and Sampling Distributions 411
8.2 Parameter Estimation 415
8.3 Maximum Likelihood Estimation 419
8.4 Confidence Intervals 430
8.5 Hypothesis Testing 441
8.6 Bayesian Decision Methods 455
8.7 Testing the Fit of a Distribution to Data 462
Summary 469
Problems 471
CHAPTER 9 Random Processes 487
9.1 Definition of a Random Process 488
9.2 Specifying a Random Process 491
9.3 Discrete-Time Processes:Sum Process,Binomial Counting Process,
and Random Walk 498
9.4 Poisson and Associated Random Processes 507
9.5 Gaussian Random Processes,Wiener Process
and Brownian Motion 514
9.6 Stationary Random Processes 518
9.7 Continuity,Derivatives,and Integrals of Random Processes 529
9.8 Time Averages of Random Processes and Ergodic Theorems 540
*9.9 Fourier Series and Karhunen-Loeve Expansion 544
9.10 Generating Random Processes 550
Summary 554
Problems 557
viii Contents
CHAPTER 10 Analysis and Processing of Random Signals 577
10.1 Power Spectral Density 577
10.2 Response of Linear Systems to Random Signals 587
10.3 Bandlimited Random Processes 597
10.4 Optimum Linear Systems 605
*10.5 The Kalman Filter 617
*10.6 Estimating the Power Spectral Density 622
10.7 Numerical Techniques for Processing Random Signals 628
Summary 633
Problems 635
CHAPTER 11 Markov Chains 647
11.1 Markov Processes 647
11.2 Discrete-Time Markov Chains 650
11.3 Classes of States,Recurrence Properties,and Limiting
Probabilities 660
11.4 Continuous-Time Markov Chains 673
*11.5 Time-Reversed Markov Chains 686
11.6 Numerical Techniques for Markov Chains 692
Summary 700
Problems 702
CHAPTER 12 Introduction to Queueing Theory 713
12.1 The Elements of a Queueing System 714
12.2 Little’s Formula 715
12.3 The M/M/1 Queue 718
12.4 Multi-Server Systems:M/M/c,M/M/c/c,And M>M>ˆ 727
12.5 Finite-Source Queueing Systems 734
12.6 M/G/1 Queueing Systems 738
12.7 M/G/1 Analysis Using Embedded Markov Chains 745
12.8 Burke’s Theorem:Departures From M/M/cSystems 754
12.9 Networks of Queues:Jackson’s Theorem 758
12.10 Simulation and Data Analysis of Queueing Systems 771
Summary 782
Problems 784
Appendices
A. Mathematical Tables 797
B. Tables of Fourier Transforms 800
C. Matrices and Linear Algebra 802
Index 805
Preface
This book provides a carefully motivated, accessible, and interesting introduction to
probability,statistics,and random processes for electrical and computer engineers.The
complexity of the systems encountered in engineering practice calls for an understand-
ing of probability concepts and a facility in the use of probability tools.The goal of the
introductory course should therefore be to teach both the basic theoretical concepts
and techniques for solving problems that arise in practice.The third edition of this
book achieves this goal by retaining the proven features of previous editions:
• Relevance to engineering practice
• Clear and accessible introduction to probability
• Computer exercises to develop intuition for randomness
• Large number and variety of problems
• Curriculum flexibility through rich choice of topics
• Careful development of random process concepts.
This edition also introduces two major new features:
• Introduction to statistics
• Extensive use of MATLAB©/Octave.
RELEVANCE TO ENGINEERING PRACTICE
Motivating students is a major challenge in introductory probability courses.Instructors
need to respond by showing students the relevance of probability theory to engineering
practice.Chapter 1 addresses this challenge by discussing the role of probability models
in engineering design.Practical current applications from various areas of electrical and
computer engineering are used to show how averages and relative frequencies provide
the proper tools for handling the design of systems that involve randomness.These ap-
plication areas include wireless and digital communications, digital media and signal
processing, system reliability, computer networks, and Web systems.These areas are
used in examples and problems throughout the text.
ACCESSIBLE INTRODUCTION TO PROBABILITY THEORY
Probability theory is an inherently mathematical subject so concepts must be presented
carefully,simply,and gradually.The axioms of probability and their corollaries are devel-
oped in a clear and deliberate manner.The model-building aspect is introduced through
the assignment of probability laws to discrete and continuous sample spaces.The notion
of a single discrete random variable is developed in its entirety,allowing the student to
ix