Table Of Content

Computational Probability INTERNATIONAL SERIES IN OPERATIONS RESEARCH & MANAGEMENT SCIENCE Frederick S. Hillier, Series Editor Stanford University Saigal, R / LINEAR PROGRAMMING: A Modern Integrated Analysis Nagurney, A. & Zhang, D. / PROJECTED DYNAMICAL SYSTEMS AND VARIATIONAL INEQUALITIES WITH APPLICATIONS Padberg, M. & Rijal, M. / LOCATION. SCHEDULING, DESIGN AND INTEGER PROGRAMMING Vanderbei, R. / LINEAR PROGRAMMING: Foundations and Extensions Jaiswal, N.K.' MILITARY OPERATIONS RESEARCH: Quantitative Decision Making Gal, T. & Greenberg, H. , ADVANCES IN SENSITIVITY ANALYSIS AND PARAMETRIC PROGRAMMING Prabhu, N.V.' FOUNDATIONS OF QUEUEING THEORY Fang, S.-C., Rajasekera, J.R & Tsao, H.-SJ. / ENTROPY OPTIMIZATION AND MATHEMATICAL PROGRAMMING Yu, G. , OPERATIONS RESEARCH IN THE AIRLINE INDUSTRY Ho, T.-H. & Tang, C. S. I PRODUCT VARIETY MANAGEMENT El-Taba, M. & Stidham, S. , SAMPLE-PATH ANALYSIS OF QUEUEING SYSTEMS Miettinen, K. M.' NONLINEAR MULTIOBJECTIVE OPTIMIZATION Chao, H. & Huntington, H. G. I DESIGNING COMPETITIVE ELECTRICITY MARKETS Weglarz, J. , PROJECT SCHEDULING: Recent Models, Algorithms & Applications Sabin, I. & Polatoglu, H. , QUALITY, WARRANTY AND PREVENTIVE MAINTENANCE Tavares, L. V.I ADVANCED MODELS FOR PROJECT MANAGEMENT Tayur, S., Ganeshan, R & Magazine, M. I QUANTITATIVE MODELING FOR SUPPLY CHAIN MANAGEMENT Weyant, J.! ENERGY AND ENVIRONMENTAL POLICY MODELING Shanthikumar, J.G. & Sumita, V.lAPPLIED PROBABILITY AND STOCHASTIC PROCESSES Liu, B. & Esogbue, A.O. I DECISION CRITERIA AND OPTIMAL INVENTORY PROCESSES Gal, Stewart & Hannel MULTICRITERIA DECISION MAKING: Advances in MCDM Models, Algorithms, Theory, and Applications Fox, B. L.I STRATEGIES FOR QUASI-MONTE CARLO Hall, RW. I HANDBOOKOFTRANSPORTATIONSCIENCE Computational Probability Edited by Winfried K. Grassmann University of Saskatchewan ~. " Springer Science+Business Media, LLC Library of Congress Cataloging-in-Publication Data A C.I.P. Catalogue record for this book is available from the Library of Congress. ISBN 978-1-4419-5100-7 ISBN 978-1-4757-4828-4 (eBook) DOI 10.1007/978-1-4757-4828-4 Copyright © 2000 by Springer Science+Business Media New York Originally published by Kluwer Academic Publishers in 2000 Softcover reprint of the hardcover 1s t edition 2000 AII rights reserved. No part of this publication may be reproduced, stored m a retrieval system or transmitted in any form or by any means, mechanical, photo copying, record ing, or otherwise, without the prior written permission of the publisher, Springer Science+Business Media, LLC. Printed on acid-free paper. Contents Preface vii 1 Computational Probability: Challenges and Limitations 1 Winfried K. Grassmann 2 Tools for Formulating Markov Models 11 Gianfranco Ciardo 3 Transient Solutions for Markov Chains 43 Edmundo de Souza e Silva and H. Richard Gail 4 Numerical Methods for Computing Stationary Distributions of Finite Irreducible 81 Markov Chains William J. Stewart 5 Stochastic Automata Networks 113 Brigitte Plateau and William J. Stewart 6 Matrix Analytic Methods 153 Winf ried K. Grassmann and David A. Stanford 7 Use of Characteristic Roots for Solving Infinite State Markov Chains 205 H. Richard Gail, Sidney L. Hantler and B. Alan Taylor 8 An Introduction to Numerical Transform Inversion and Its Application to Prob- 257 ability Models Joseph Abate, Gagan L. Choudhury and Ward Whitt vi COMPUTATIONAL PROBABILITY 9 Optimal Control of Markov Chains 325 Shaler Stidham, Jr. 10 On Numerical Computations of Some Discrete-Time Queues 365 Afohan L. Chaudhry 11 The Product Form Tool for Queueing Networks 409 Nico Af. van Dijk, Win/Tied K. Grassmann 12 Techniques for System Dependability Evaluation 445 Jogesh K. Afuppala, Ricardo Af. Fricks, and Kishor S. Trivedi Index 481 Preface In recent years, great advances have been in the field of computational proba bility, in particular in areas related to queueing systems, stochastic Petri-nets and systems dealing with reliability. The objective of this book is to make these topics accessible to researchers, graduate students, and, hopefully, practition ers. Great care was taken to make the exposition as clear as possible. Every line in this text has been evaluated, and changes have been made whenever it was felt that the initial exposition was not clear enough for the intended readership. The topics were selected with great care to cover as wide a range as possible, with particular emphasis on Markov modeling and queueing applications. I feel that I was very privileged to obtain the contributions of so many outstanding researchers in this field, and I thank all the contributors for their great effort. It is not easy to explain and summarize the sophisticated techniques used in Markov modeling and queueing, but I am happy to say that all contributers did an outstanding job. The outline of this book is as follows. The first chapter describes, in non mathematical terms, the challenges in computational probability. Most of the models used in this book are based on Markov chains, and Chapter 2 describes the methodologies available to obtain the transition matrices for these Markov chains, with particular emphasis on stochastic Petri-nets. Chapter 3 discusses how to find transient probabilities and transient rewards for theses Markov chains. The next two chapters indicate how to find steady-state probabilities for Markov chains with a finite number of states. In this case, the equilibrium probabilities can be found by solving a system of linear equations, with one equation for each state. If the number of states is not too large, one can use methods based on Gaussian elimination, that is, direct methods, to solve these systems. However, as the number of states increases, iterative methods become advantageous. Both direct and iterative methods are described in Chapter 4. In many situations, the transition matrices are similar to Kronecker products, and this can be exploited for reducing the storage requirements and/or the number of operations needed to find equilibrium solutions. Details for these methods viii COMPUTATIONAL PROBABILITY are given in Chapter 5. The next two chapters deal with infinite-state Markov chains. Infinite-state Markov chains occur frequently in queueing, because one typically does not want to set a bound for all queues. There are two main methods to analyze problems with infinite queues: matrix analytical methods and methods based on spectral expansion. These are discussed in Chapters 6 and 7, respectively. Chapter 8 deals with transforms, in particular Laplace transforms. Trans form methods have been used extensively in theoretical work, but their practi cal value was questioned. This has changed, mainly due to the effort of Whitt and his collaborators, who have developed a number of numerical methods for transform inversions. They describe their work in Chapter 8. Often one is not satisfied with improving the system, but one wants to configure the system in the best possible way, that is, one wants to optimize the system. One way to do the optimization is through Markov decision making, a topic described in Chapter 9. Markov modeling has found applications in many areas, three of which are described in some detail: Chapters 10 analyses discrete-time queues, Chapter 11 describes networks of queues, and Chapter 12 deals with reliability theory. I would like to thank all the people who have helped me to make the project a success. I would also like to thank the Natural Science and Engineering Council of Canada (NSERC) who provided me with a generous operating grant, part of which was used for this project, The Department of Computer Science of the University of Saskatchewan and the Department of Statistical and Actuarial Sciences of the University of Western Ontario have provided me with computer facilities and technical advice, which was very much appreciated. Last but not least, I would like to thank all the authors for their excellent contributions. 1 COMPUTATIONAL PROBABILITY: CHALLENGES AND LIMITATIONS Winfried K. Grassmann Department of Computer Science University of Saskatchewan Saskatoon, Sask S7N 5A9 Canada [email protected] 1 INTRODUCTION Computational probability is the science of calculating probabilities and expec tations. As this book demonstrates, there are many mathematical challenges in the area of computational probability. To set the stage, we discuss the objectives of computational probability in more detail, and we point out the difficulties one encounters in this area. We also contrast computational proba bility with other approaches. 2 STOCHASTIC SYSTEMS AND THEIR ANALYSIS Probability theory deals with uncertainty, that is, with situations admitting different possible outcomes. The set of all outcomes forms the sample space. In contrast to other methods, such as fuzzy set theory [Klir and Yuan, 1995], prob ability theory associates probabilities with outcomes. If the number of outcomes is countable, a probability can be associated with each outcome. However, to deal with continuous sample spaces, one typically associates probabilities with sets of outcomes. A system is called stochastic if it can behave in different ways, and if one can associate a probability with each set of possible behaviors. Most stochastic systems are also dynamic, that is, they change with time. This book deals predominantly with stochastic dynamic systems, including queueing systems, communication systems, and systems dealing with reliability. W. K. Grassmann (ed.), Computational Probability © Springer Science+Business Media New York 2000 2 COMPUTATIONAL PROBABILITY To analyze a stochastic system mathematically, it must be formulated in mathematical terms which can then be manipulated. In other words, a math ematical model must be built. Ideally, the model should behave like the real system, that is, it should predict the system behavior accurately. This may not be possible for two reasons: firstly, most real systems are too complicated to be dealt with mathematically, and secondly, the working of the real system may be unknown. The realism of the model can be assessed by comparing its behavior with that of the real system, at least as long as the system is de terministic. However, in stochastic systems, different realizations are possible, and the comparison with the real system must be based on probabilities and expectations. There are three approaches to analyze stochastic systems and predicting their behavior: analytical methods, numerical methods, and methods based on Monte Carlo simulation. All of these methods are now described, and their strengths and weaknesses are outlined. Analytical methods express dependencies between the variables of a model explicitly, usually by means of a formula. For instance, Little's theorem pro vides a simple formula relating the average number in a system, the arrival rate to the system, and the time spent in the system. Simple explicit formu las such as this are definitely the ideal. Unfortunately, many explicit formulas are far from simple. In particular, using symbol manipulation programs, such as Maple or Mathematica, one can easily derive analytical results of amazing complexity. It is questionable whether or not complex formulas have any direct benefit because the human mind cannot deal with more than three to four vari ables at a time. However, complex formulas may have indirect benefits if they allow one to derive simple results, or if they form the basis of an algorithm. This brings us to the second approach, the numerical approach. In the numerical approach, one fixes all input parameters of the model to cer tain values, and one explicitly calculates the probabilities and/or expectations of interest. To do this, an algorithm is created, and a program is written and executed on a computer, using the prescribed input parameters. For instance, to find the waiting time in a GI/G/1 queue, one must use numerical methods, such as the ones described in Chapters 8 and 10. However, once these meth ods are implemented, the user may specify an appropriate distribution, and set appropriate parameters, and the computer delivers the results. This allows the users to experiment without having to know the underlying mathematical theory. The methods underlying a numerical algorithm need not be simple. Since the user no longer needs to know about the underlying mathematical details, even extremely complex analytical results may became useful once programmed. In fact, some analytical results, which, at the time of their discovery, seemed to be only of theoretical interest, turned out to be very useful as the basis of numerical algorithms. As a case in point, consider Chapter 8, which describes a number of methods for the numerical inversion of Laplace transforms. These methods make it practical to actually use theoretical results involving transforms derived