Table Of Content

An Introduction to Stochastic Processes and Their Applications An Introduction to Stochastic Processes and Their Applications CHIN LONG CHIANG Professor of Biostatistics University of California, Berkeley ROBERT E. KRIEGER PUBLISHING COMPANY HUNTINGTON, NEW YORK Original Edition 1980, based upon previous title Introduction to Stochastic Processes in Biostatistics Printed and Published by ROBERT E. KRIEGER PUBLISHING CO., INC. 645 NEW YORK AVENUE HUNTINGTON, NEW YORK 11743 Basic edition © Copyright 1968 by JOHN WILEY & SONS, INC. transferred to Chin Long Chiang 1978 New Edition, © Copyright 1980 by Robert E. Krieger Pub. Co., Inc. All rights reserved. No reproduction in any form of this book, in whole or in part (exceptfor brief quotation in critical articles or reviews), may be made without written authorization from the publisher. Printed in the United States of America Library of Congress Cataloging in Publication Data Chiang, Chin Long An introduction to stochastic processes and their applications. Bibliography: p. 1. Applied mathematics 2. Probability 3. Stochastic processes 4. Mathematical statistics I. Title. QH323.5.C5 1975 574'.01'82 74-14821 ISBN 0-88275-200-6 To Fu Chen and William, Robert, Harriet and Carol Preface This book is intended to be a textbook in stochastic processes or in applied probability. It may be used also as a reference book for courses in mathematical statistics, engineering (reliability theory), biostatistics (survival analysis), and demography (migration processes). The volume consists of an extensive revision of eight chapters of Part I of Introduction to Stochastic Processes in Biostatistics (Wiley, 1968) and a collection of nine new chapters on various topics on the subject. The biostatistics component of the 1968 book is eliminated. The 1968 book went out of print within five years of the date of publication; I only just managed to complete the revision, almost six years after the last copy was sold. The evolution of stochastic processes is envisioned here in a particular sequence. The material is arranged in the order of discrete processes, one-state continuous processes, two-state processes, and multiple-state processes. The preliminaries now include a new chapter (Chapter 3) on exponential type distributions because of their impor tance and their utility in stochastic processes, in survival analysis, and in reliability theory. Discrete processes and renewal processes in Chapters 4 through 7 have been added in this volume as they are an essential part of stochastic processes. The algebraic treatment in Chapter 6 is in variation of conventional methods that one may find, for example, in Feller [1968]. The simple and more explicit formulas for the higher order transition probabilities presented in this chapter may help the reader achieve a better understanding of Markov chains and facilitate their application. Chapters 8 and 10 contain well-known continuous processes describ ing population growth and queueing processes. The general birth process and the equality in stochastic processes in Chapter 9 provide two additional means for deriving explicit formulas for any increasing process or decreasing process. The epidemic model serves as an example to demonstrate the potential of the equality in resolving difficulties. vii viii PREFACE The first chapter (Chapter 11) on the simple illness-death process is basically unaltered except for the addition of a section on generating functions and a section on survival and stages of diseases. The material on multiple transition probability and multiple transition time in Chapters 12 and 13 is essentially new. The corresponding formulas in these chapters are now of closed form and thus the process can more easily be subject to practical application. When there are no absorbing states, the two-state model generates an alternating renewal process. Parts of the general renewal theory have been extended to this case. Explicit solution for the Kolmogorov differential equations for the case where the intensity function matrix (infinitesimal generator) V has distinct eigenvalues has been presented in the previous edition and reproduced in Chapter 14. The solution has been extended in Chapter 15 for the general case where matrix V has multiple and complex eigenvalues. Chapters 16 and 17 are reproductions of the corresponding chapters in the old book but with some minor changes. I place the significance of stochastic processes on their potential as an analytic tool for scientific research rather than on the theoretical development of the subject. I believe that this volume as well as the 1968 book reflect this point of view. I have used the material in this volume in courses that I have taught at the University of California, Berkeley and at Harvard University. I have once again benefitted from comments and en couragement from several friends who have read selected chapters. They include B. J. van den Berg, J. Deming, J. Emerson, J. P. Hsu, E. Peritz, P. Rust, S. Selvin, R. Wong and G. L. Yang. Solutions to the problems at the end of each chapter are in preparation but will not be published together with the text in order to meet the publication date of the book. Finally, my deep appreciation is due to Ms. Bonnie Hutchings who provided secretarial assistance during the course of the revision and expert typing from my handwritten pages to the final version with peerless skill and patience. Chin Long Chiang University of California, Berkeley September, 1979 Preface to The 1968 Book Time, life, and risks are three basic elements of stochastic processes in biostatistics. Risks of death, risks of illness, risks of birth, and other risks act continuously on man with varying degrees of intensity. Long before the development of modem probability and statistics, men were concerned with the chance of dying and the length of life, and they constructed tables to measure longevity. But it was not until the advances in the theory of stochastic process made in recent years that empirical processes in the human population have been systematically studied from a probabilistic point of view. The purpose of this book is to present stochastic models describing these processes. Emphasis is placed on specific results and explicit solutions rather than on the general theory of stochastic processes. Those readers who have a greater curiosity about the theoretical arguments are advised to consult the rich literature on the subject. A basic knowledge in probability and statistics is required for a profitable reading of the test. Calculus is the only mathematics presupposed, although some familiarity with differential equations and matrix algebra is needed for a thorough understanding of the material. The text is divided into two parts. Part I begins with one chapter on random variables and one on probability generating functions for use in succeeding chapters. Chapter 3 is devoted to basic models of population growth, ranging from the Poisson process to the time dependent birth-death process. Some other models of practical interest that are not included elsewhere are given in the problems at the end of the chapter. Birth and death are undoubtedly the most important events in the human population, but what is statistically more complex is the illness process. Illnesses are potentially concurrent, repetitive, and reversible and consequently analysis is more challenging. In this book illnesses are treated as discrete entities, and a population is visualized as consisting of discrete states of illnesses. An individual is said to be ix X PREFACE TO THE I96K BOOK in a particular state of illness if he is affected with the corresponding diseases. Since he may leave one illness state for another or enter a death state, consideration of illness opens up a new domain of interest in multiple transition probability and multiple transition time. A basic and important case is that in which there are two illness states. Two chapters (Chapters 4 and 5) are devoted to this simple illness-death process. In dealing with a general illness-death process that considers any finite number of illness states, I found myself confronted with a finite Markov process. To avoid repetition and to maintain a reasonable graduation of mathematical involvement, I have interrupted the devel opment of illness processes to discuss the Kolmogorov differential equations for a general situation in Chapter 6. This chapter is concerned almost entirely with the derivation of explicit solutions of these equations. For easy reference a section (Section 3) on matrix algebra is included. Once the Kolmogorov differential equations are solved in Chapter 6, the discussion on the general illness-death process in Chapter 7 becomes straightforward; however, the model contains sufficient points of interest to require a separate chapter. The general illness-death process has been extended in Chapter 8 to account for the population increase through immigration and birth. These two possibilities lead to the emigration-immigration process and the birth-illness-death process, respectively. But my effort failed to provide an explicit solution for the probability distribution function in the latter case. Part II is devoted to special problems in survival and mortality. The life table and competing risks are classical and central topics in biostatistics, while the follow-up study dealing with truncated information is of considerable practical importance. I have endeavored to integrate these topics as thoroughly as possible with probabilistic and statistical principles. I hope that I have done justice to these topics and to modern probability and statistics. It should be emphasized that although the concept of illness processes has arisen from studies in biostatistics, it has a wide application to other fields. Intensity of risk of death (force of mortality) is synony mous with “failure rate” in reliability theory; illness states may be alternatively interpreted as geographic locations (in demography), compartments (in compartment analysis), occupations, or other defined conditions. Instead of the illness of a person, we may consider whether a person is unemployed, or whether a gene is a mutant gene, a telephone PREFACE TO THE 196« BOOK xi line is busy, an elevator is in use, a mechanical object is out of order, etc. The book was written originally for students in biostatistics, but it may be used for courses in other fields as well. The following are some suggestions for teaching plans: 1. As a year course in biostatistics: Chapters 1 and 2 followed by Chapters 10 through 12, and then by Chapters 3 through 8. In this arrangement, a formal introduction of the pure death process is necessary at the beginning of Chapter 10. 2. For a year course in demography: Plan 1 above may be followed, except that the term “illness process” might be more appro priately interpreted as “internal migration process.” 3. As a supplementary text for courses in biostatistics or demo graphy: Chapters 9 through 12. 4. For a one-semester course in stochastic processes: Chapters 2 through 8. As a general reference book, Chapter 9 may be omitted. The book is an outgrowth partly of my own research, some of which appears here for the first time (e.g., Chapter 5 and parts of Chapter 6), and partly of lecture notes for courses in stochastic processes for which I am grateful to the many contributors to the subject. I have used the material in my teaching at the Universities of California (Berkeley), Michigan, Minnesota, North Carolina; Yale and Emory Universities, and at the London School of Hygiene, University of London. The work could not have been completed without incurring indebt edness to a number of friends. It is my pleasure to acknowledge the generous assistance of Mrs. Myra Jordan Samuels and Miss Helen E. Supplee, who have read early versions and made numerous constructive criticisms and valuable suggestions. Their help has tre mendously improved the quality of the book. I am indebted to the School of Public Health, University of California, Berkeley, and the National Institutes of Health, Public Health Service, for financial aid under Grant No. 5-S01-FR-0544-06 to facilitate the work. An invitation from Peter Armitage to lecture in a seminar course at the London School of Hygiene gave me an opportunity to work almost exclusively on research projects associated with this book. I also wish to express my appreciation to Richard J. Brand and Geoffrey S. Watson who read some of the chapters and provided useful suggestions. My thanks are also due to Mrs. Shirley A. Hinegardner