ebook img

Applied Stochastic Processes PDF

205 Pages·2013·7.844 MB·\205
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Applied Stochastic Processes

Applied Stochastic Processes K20401_FM.indd 1 6/19/13 12:20 PM K20401_FM.indd 2 6/19/13 12:20 PM Applied Stochastic Processes Ming Liao Auburn University Alabama, USA K20401_FM.indd 3 6/19/13 12:20 PM MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MAT- LAB® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software. CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2014 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20130614 International Standard Book Number-13: 978-1-4665-8934-6 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the validity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying, microfilming, and recording, or in any information stor- age or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copy- right.com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that pro- vides licenses and registration for a variety of users. For organizations that have been granted a pho- tocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com i i i i Contents 1 Probability and stochastic processes 1 1.1 Probability . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Random variables and their distributions . . . . . . . . 3 1.3 Mathematical expectation . . . . . . . . . . . . . . . . . 6 1.4 Joint distribution and independence . . . . . . . . . . . 8 1.5 Convergence of random variables . . . . . . . . . . . . . 11 1.6 Laplace transform and generating functions . . . . . . . 12 1.7 Examples of discrete distributions . . . . . . . . . . . . 14 1.8 Examples of continuous distributions . . . . . . . . . . 15 1.9 Stochastic processes . . . . . . . . . . . . . . . . . . . . 17 1.10 Stopping times . . . . . . . . . . . . . . . . . . . . . . . 19 1.11 Conditional expectation . . . . . . . . . . . . . . . . . . 21 2 Poisson processes 29 2.1 Introduction to Poisson processes . . . . . . . . . . . . 29 2.2 Arrival and inter-arrival times of Poisson processes . . . 33 2.3 Conditional distribution of arrival times . . . . . . . . . 36 2.4 Poisson processes with different types of events . . . . . 39 2.5 Compound Poisson processes . . . . . . . . . . . . . . . 41 2.6 Nonhomogeneous Poisson processes . . . . . . . . . . . 43 3 Renewal processes 47 3.1 An introduction to renewal processes . . . . . . . . . . 47 3.2 Renewal reward processes . . . . . . . . . . . . . . . . . 51 3.3 Queuing systems . . . . . . . . . . . . . . . . . . . . . . 55 3.4 Queue lengths, waiting times, and busy periods . . . . 60 3.5 Renewal equation . . . . . . . . . . . . . . . . . . . . . 66 3.6 Key renewal theorem . . . . . . . . . . . . . . . . . . . 71 3.7 Regenerative processes . . . . . . . . . . . . . . . . . . 77 3.8 Queue length distribution and PASTA . . . . . . . . . . 79 v i i i i i i i i vi 4 Discrete time Markov chains 87 4.1 Markov property and transition probabilities . . . . . . 87 4.2 Examples of discrete time Markov chains . . . . . . . . 91 4.3 Multi-step transition and reaching probabilities . . . . . 94 4.4 Classes, recurrence, and transience . . . . . . . . . . . . 97 4.5 Periodicity, class property, and positive recurrence . . . 103 4.6 Expected hitting time and hitting probability . . . . . . 106 4.7 Stationary distribution . . . . . . . . . . . . . . . . . . 110 4.8 Limiting properties . . . . . . . . . . . . . . . . . . . . 117 5 Continuous time Markov chain 123 5.1 Markov property and transition probability . . . . . . . 123 5.2 Transition rates . . . . . . . . . . . . . . . . . . . . . . 125 5.3 Stationary distribution and limiting properties . . . . . 130 5.4 Birth and death processes . . . . . . . . . . . . . . . . . 138 5.5 Exponential queuing systems . . . . . . . . . . . . . . . 141 5.6 Time reversibility . . . . . . . . . . . . . . . . . . . . . 149 5.7 Hitting time and phase-type distributions . . . . . . . . 155 5.8 Queuing systems with time-varying rates . . . . . . . . 159 6 Brownian motion and beyond 165 6.1 Brownian motion . . . . . . . . . . . . . . . . . . . . . 165 6.2 Standard Brownian motion and its maximum . . . . . . 169 6.3 Conditional expectation and martingales . . . . . . . . 172 6.4 Brownian motion with drift . . . . . . . . . . . . . . . . 175 6.5 Stochastic integrals . . . . . . . . . . . . . . . . . . . . 178 6.6 Itˆo’s formula and stochastic differential equations . . . 182 6.7 A single stock market model . . . . . . . . . . . . . . . 186 Bibliography 195 Index 197 i i i i i i i i Preface The purpose of these notes is to present a concise account of applied stochastic processes as usually covered in a first-year graduate course, emphasizing applications and practical computation, but also devel- oping an essentially complete mathematical theory. The topics, after a quick review of basic probability, include Poisson processes, renewal processes, discrete time and continuous time Markov chains, Brownian motion, and an introduction to stochastic differential equations. The main applications are queues, but other examples are also considered, including the mathematical model of a single stock market. We have tried to present the materials in a quick to-the-point fashion, possibly with some motivation and short examples, but without much digres- sion. The mathematical theory is developed with strong emphasis on probability intuition, often informally, to be easily accessible, but al- ways based on sound reasoning. Most sections end with a few closely related exercises. The solutions to all exercises are included in a sepa- rate solutions manual. The short bibliography contains the books and papers used in the preparation of the notes. It is not intended to be comprehensive, and so does not include many other good books on applied stochastic pro- cesses. Studentsofmystochasticprocessclassesenduredtheearlyversions ofthenotes,andprovidedusefulinputs.Iwishtothanktheanonymous reviewers for the helpful comments and my colleague Erkan Nane for providing a list of errors. A lot of effort has been put into these notes, and my wife’s support and understanding are always important for me to complete such a project. MATLAB and Simulink are registered trademarks of The Math- Works, Inc. For product information, please contact: The MathWorks, Inc. 3 Apple Hill Drive Natick, MA 01760-2098 USA Tel: 508 647 7000 vii i i i i i i i i viii Fax: 508-647-7001 E-mail: [email protected] Web: www.mathworks.com i i i i i i i i Chapter 1 Probability and stochastic processes The first eight sections summarize basic concepts and facts in proba- bility theory with little explanation and very few proofs. The reader may consult a standard graduate text on probability, such as [2], for more details. The stochastic processes, stopping times, and conditional expectation are introduced in the last three sections. 1.1 Probability Sample space and events: The set of all possible outcomes is called the sample space and is denoted by Ω. A collection of subsets of Ω, including Ω itself and the empty set ∅, are called events. This will be defined more precisely in the note on the measure theory later, but a reader who is not familiar with the measure theory may just assume any subset of Ω is an event. We will assume the reader is familiar with the basic set notation and operations such as the membership ∈, the inclusion ⊂, the union ∪, the intersection ∩, and the complement Ec of an event E. Probability: The probability P(E) is a function defined for every event E, which satisfies the following three basic axioms (or rules): (a) 0 ≤ P(E) ≤ 1. (b) P(Ω) = 1 and P(∅) = 0. ∑ (c) P(∪∞ E ) = ∞ P(E ) for any sequence of disjoint events n=1 n n=1 n E ;E ;E ;:::, 1 2 3 Property (c) is called the countable additivity of the probability as a sequence of objects is called a countable set in mathematics. The sample space Ω together with a probability P is called a prob- 1 i i i i

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.