Table Of Content

Statistics and Computing Series Editors: J. Chambers w. Eddy W. W l e S. Sheather L. Tiemey Springer Y~rk Berlin Heidelberg Hang Kong London Milan Perfs Tokyo Statistics and Computing Dalgomd: lnboductory Statistics with R. Gentle.. Elemak ofComputational Stptistics. OenfIe: Numerical Linear Algebra for Applications m Statistics. Oentle Random N& omaation and Monte &lo Mahods, 2nd Editim. Hcr*dwMWwlach: XploRe: An Intnactive Statistical Computing Bnvirommt me RiOUFPN)Iron: Basics of S and S-Pws,3 rd Edition Lmge: NNllmrkal Analysis for Statisticians. b&r: Local Regrnsion and Lilihcd bRurmcrldh/Fibgemld. Numrical Baycsisn Mcmads Applied to Signal Roassing. Pluvrallw: VARIOWIN: Softwan for Spatial Data Analysis in 2D. PinheirOlBau1: Mixed-Effcds Models in S and S-hus venabk.dRiy,l~: Modem ~ppliedS tatistics with S, 4th ~tlon. venabler/Riprey: s ProgmEhg. . me WWnmn: Ibe of Graphics. James E. Gentle Random Number Generation and Monte Carlo Methods Second Edition With 54 Illustrati~ns Springer James H. Gentle School of Computational Sciences George Mason University Fairfax. VA 22030-4444 USA j gen [email protected] Series Editors: J. Chambers W. Eddy W, Hardle Bell Labs, Lucent Techonologies Department of Statistics Institut fiir Slatistik und Okonnmetrie 600 Mountain Avenue Carnegie Mellon University Humboldt-University Murray Hill. NJ 07974 Pittsburgh, PA Spandaucr .Str. I USA USA D-10178 Berlin Germany S. Sheather L. Tiemey Australian Graduate School Sclool of Statistics and Actuarial Science of Management Universily of Iowa University of New South Wales lowa City. IA 52242-1414 Sydney, NSW 2052 USA Australia Library of Congress Cataloging-in-Puhlication Data Gentle, James E.. 1943-- Random number generation and Monte Carlo methods / James H. Gentle. p. cm. — (Statistics and computing) Includes bibliographical references and index ISBN 0-3S7-OOI78-6 (alk, paper) 1. Monte (,Carlo method. 2. Random number generators. I. Title. [I. Series. (QA2298 ,(G46 2003 519 .2'82—dc21 2003042437 ISBN 0-387-0017-6 e-ISBN 0-387-21610 Printed on acid-free paper CO 2003, 3l')'1998 Springer Science Business Media, Inc. All rights reserved. This work may not be translated or copied in whole or in pan without the written permission of the publisher (Springer Science Business Media, Inc., 233 Spring Strcoi, New York, NY 10013, USA), except for brief excerpts in connection wish reviews or scholarly analysis. Use in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed is for- bidden. The use in (his publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject In proprietary rights. Printed in the United States of America. (HID 9 S 7 6 5 4 3 2 Corrected second printing, 2005. SPIN 11016038 springeronline.cnm To Maria This page intentionally left blank Preface The role of Monte Carlo methods and simulation in all of the sciences has increasedinimportanceduringthepastseveralyears. Thiseditionincorporates discussion of many advances in the field of random number generation and Monte Carlo methods since the appearance of the first edition of this book in 1998. These methods play a central role in the rapidly developing subdisciplines of the computational physical sciences, the computational life sciences, and the other computational sciences. The growing power of computers and the evolving simulation methodology have led to the recognition of computation as a third approach for advancing the natural sciences, together with theory and traditional experimentation. At the kernel of Monte Carlo simulation is random number generation. Generation of random numbers is also at the heart of many standard statistical methods. The random sampling required in most analyses is usually done by the computer. The computations required in Bayesian analysis have become viable because of Monte Carlo methods. This has led to much wider applications of Bayesian statistics, which, in turn, has led to development of newMonteCarlomethods andtorefinementofexistingproceduresforrandom number generation. Various methods for generationof random numbers havebeen used. Some- times, processes that are considered random are used, but for Monte Carlo methods,whichdependonmillionsofrandomnumbers,aphysicalprocessasa sourceofrandomnumbersisgenerallycumbersome. Insteadof“random”numbers, most applications use “pseudorandom” numbers, which are deterministic but “look like” they were generated randomly. Chapter 1 discusses methods for generation of sequences of pseudorandom numbers that simulate a uniform distribution over the unit interval (0,1). These are the basic sequences from whicharederivedpseudorandomnumbersfromotherdistributions, pseudorandom samples, and pseudostochastic processes. In Chapter 1, as elsewhere in this book, the emphasis is on methods that work. Development of these methods often requires close attention to details. For example, whereas many texts on random number generation use the fact that the uniform distribution over (0,1) is the same as the uniform distribution over (0,1] or [0,1], I emphasize the fact that we are simulating this dis- vii viii PREFACE tribution with a discrete set of “computer numbers”. In this case whether 0 and/or 1 is included does make a difference. A uniform random number gen- erator should not yield a 0 or 1. Many authors ignore this fact. I learned it over twenty years ago, shortly after beginning to design industrial-strength software. The Monte Carlo methods raise questions about the quality of the pseudo- randomnumbersthatsimulatephysicalprocessesandabouttheabilityofthose numbers to cover the range of a random variable adequately. In Chapter 2, I address some issues of the quality of pseudorandom generators. Chapter 3 describes some of the basic issues in quasirandom sequences. These sequences are designed to be very regular in covering the support of the random process simulated. Chapter4discussesgeneralmethodsfortransformingauniformrandomde- viate or a sequence of uniform random deviates into a deviate from a different distribution. Chapter 5 describes methods for some common specific distributions. The intent is not to provide a compendium in the manner of Devroye (1986a)but, for many standard distributions, to give at least a simple method or two, which may be the best method, but, if the better methods are quite complicated, to give references to those methods. Chapter 6 continues the de- velopments of Chapters 4 and 5 to apply them to generation of samples and nonindependent sequences. Chapter 7 considers some applications of random numbers. Some of these applications are to solve deterministic problems. This type of method is called Monte Carlo. Chapter8providesinformationoncomputersoftwareforgenerationofran- domvariates. ThediscussionconcentratesontheS-Plus,R,andIMSLsoftware systems. Monte Carlo methods are widely used in the researchliterature to evaluate properties of statistical methods. Chapter 9 addresses some of the considera- tions that apply to this kind of study. I emphasize that a Monte Carlo study uses an experiment, and the principles of scientific experimentation should be observed. The literature on random number generation and Monte Carlo methods is vast and ever-growing. There is a rather extensive list of references beginning onpage336;however,Idonotattempttoprovideacomprehensivebibliography or to distinguish the highly-varyingquality of the literature. The main prerequisite for this text is some backgroundin what is generally called“mathematicalstatistics”. Inthediscussionsandexercisesinvolvingmul- tivariate distributions, some knowledgeof matrices is assumed. Some scientific computerliteracyisalsonecessary. Idonotuseanyparticularsoftwaresystem in the book, but I do assume the ability to program in either Fortran or C and the availability of either S-Plus, R, Matlab, or Maple. For some exercises, the required software can be obtained from either statlib or netlib (see the bibliography). The book is intended to be both a reference and a textbook. It can be PREFACE ix usedastheprimarytext orasupplementarytext foravarietyofcoursesatthe graduate or advanced undergraduate level. A course in Monte Carlo methods could proceed quickly through Chapter 1, skip Chapter 2, cover Chapters 3 through 6 rather carefully, and then, in Chapter7,dependingonthe backgroundsofthe students,discussMonteCarlo applicationsinspecificfieldsofinterest. Alternatively,acourseinMonteCarlo methodscouldbeginwithdiscussionsofsoftwaretogeneraterandomnumbers, as in Chapter 8, and then go on to cover Chapters 7 and 9. Although the material in Chapters 1 through 6 provides the background for understanding the methods, in this casethe details of the algorithms arenot covered,andthe material in the first six chapters would only be used for reference as necessary. General courses in statistical computing or computational statistics could use the book as a supplemental text, emphasizing either the algorithms or the Monte Carlo applications as appropriate. The sections that address computer implementations,suchasSection1.2,cangenerallybeskippedwithoutaffecting the students’ preparation for later sections. (In any event, when computer implementations are discussed, note should be taken of my warnings about use of software for random number generation that has not been developed by software development professionals.) In most classes that I teach in computational statistics, I give Exercise 9.3 inChapter9(page311)asatermproject. ItistoreplicateandextendaMonte Carlostudyreportedinsomerecentjournalarticle. Inworkingonthisexercise, the students learn the sad facts that many authors are irresponsibleand many articles have been published without adequate review. Acknowledgments IthankJohnKimmelofSpringerforhisencouragementandadviceonthisbook and other books on which he has worked with me. I thank Bruce McCullough for comments that corrected some errors and improved clarity in a number of spots. I thank the anonymousreviewersof this edition for their comments and suggestions. Ialsothankthemanyreadersofthefirsteditionwhoinformedme of errors and who otherwise provided comments or suggestions for improving the exposition. I thank my wife Mar´ıa, to whom this book is dedicated, for everything. I did all of the typing, programming, etc., myself, so all mistakes are mine. I would appreciate receiving suggestions for improvement and notice of errors. Notes on this book, including errata, are available at http://www.science.gmu.edu/~jgentle/rngbk/ Fairfax County, Virginia James E. Gentle April 10, 2003