Stochastic Monotonicity and Queueing Applications of Birth-Death Processes

124 Pages·1981·2.036 MB·English

by  E. A. van Doorn (auth.)

Lecture Notes in
Statistics
Edited by D. Brillinger, S. Fienberg, J. Gani,
J. Hartigan, J. Kiefer, and K. Krickeberg
4
Erik van Doorn
Stochastic Monotonicity
and Queueing Applications
of Birth-Death Processes
Springer-Verlag
New York Heidelberg Berlin
E. A. van Doorn
Netherlands Postal and Telecommunications Services
Dr. Neher - Laboratories
Post Office Box 421
2260 AK Leldschendam
The Netherlands
This monograph is a polished version of the author's dissertation entitled:
"Stochastic Monotonicity of the Birth-Death Processes," which was written
while he was affiliated with the Department of Applied Mathematics,
Twente University of Technology, Enschede.
AMS Subject Classifications (1980): primary 6OJ80; secondary 6OK25, 92A15
Library of Congress Cataloging In Publication Data
Doorn, Erik van.
Stochastic monotonicity and queueing applica
tions of birth-death processes.
(Lecture notes in statistics; 4)
Based on the author's thesis, Twente University
of Technology, Enschede.
Bibliography: p.
Includes Indexes.
1. Birth and death processes (Stochastic
processes) 2. Monotone operators. 3. Queueing
theory. I. rltle. II. Series.
QA274.76.D66 519.2'34 80-25183
ISBN-13: 978-0-387-90547-1 e-ISBN-13: 978-1-4612-5883-4
001: 10.1007/978-1-4612-5883-4
All rights reserved.
No part of this book may be translated or reproduced In any form
without written permission from Springer-Verlag.
© 1981 by Springer-Verlag New York Inc.
9 8 7 8 5 4 3 2 1
PREFACE
A stochastic process {X(t): 0 S t < =} with discrete state space S c ~ is said to be
stochastically increasing (decreasing) on an interval T if the probabilities
Pr{X(t) > i}, i E S, are increasing (decreasing) with t on T. Stochastic monotonicity
is a basic structural property for process behaviour. It gives rise to meaningful
bounds for various quantities such as the moments of the process, and provides the
mathematical groundwork for approximation algorithms.
Obviously, stochastic monotonicity becomes a more tractable subject for analysis if
the processes under consideration are such that stochastic mono tonicity on an inter
val 0 < t < E implies stochastic monotonicity on the entire time axis. DALEY (1968)
was the first to discuss a similar property in the context of discrete time Markov
chains. Unfortunately, he called this property "stochastic monotonicity", it is more
appropriate, however, to speak of processes with monotone transition operators.
KEILSON and KESTER (1977) have demonstrated the prevalence of this phenomenon in
discrete and continuous time Markov processes. They (and others) have also given a
necessary and sufficient condition for a (temporally homogeneous) Markov process to
have monotone transition operators. Whether or not such processes will be stochas
tically monotone as defined above, now depends on the initial state distribution.
Conditions on this distribution for stochastic mono tonicity on the entire time axis
to prevail were given too by KEILSON and KESTER (1977).
It is very well conceivable that a process with monotone transition operators is not
stochastically monotone on the entire positive time axis but on an interval of the
form (t 1 ,co), with tl > O. Clearly, i"t is of some interest to know under which cir
cumstances this phenomenon occurs. The study of these circumstances is the main sub
ject of this monograph. The analysis is restricted to birth-death processes, which
form the most important class of temporally homogeneous Markov processes in conti
nuous time with monotone transition operators. In some proofs explicit use is made
of specific properties of birth-death processes, so that it is probably not possible
to extend the results to other classes of Markov processes.
The main results of this monograph are obtained in chapter 5 where necessary and
sufficient conditions are given for a birth-death process to be stochastically mono
tone in the long run when the state space is a semi-infinite lattice of integers and
the initial state distribution is supported by finitely many points. The theory
needed to arrive at these results is developed in the chapters 3 and 4. It appears
that the concept of dual processes, which is only touched upon in the existing
literature, is very fruitful and of intrinsic interest.
In chapter 1 and chapter 2 many known facts about birth-death processes are collect
ed. Also in chapter 2 some preliminary analysis is done with a view to the chapters
6, 7 and 8, where the results are applied to specific processes. To do this one needs
iv
at least partial knowledge of the so-called spectral representation of the transition
probabilities. As for the linear growth, birth-death processes of chapter 8 (including
the M/M/m queue length process) this knowledge is available and application of the
results of chapter 5 to these processes is straightforward. This is not the case,
however, with the MIMls queue length process of chapter 6 and the queue length process
of chapter 7 which models a system where potential customers are discouraged by queue
length. A substantial part of this monograph, in fact the main part of the chapters 6
and 7 is therefore concerned with obtaining these representations, which have an
interest of their own. Our findings in this respect extend the results previously
obtained by KARLIN and McGREGOR (1958a) and NATVIG (1974), respectively.
In chapter 9 various aspects of the first moment of birth-death processes are dis
cussed. It appears to behave very regularly in a number of important cases.
Finally, birth-death processes with a finite state space are considered in chapter 10.
Although the analysis of the phenomenon of stochastic monotonicity may be performed
through the concept of dual processes as in the infinite case, an entirely different
approach is chosen.
I take pleasure in closing this preface by acknowledging the support of Professor
Jos H.A. de Smit of Twente University of Technology who provided the key references.
and by thanking Miss Bea Bhola of the Dr. Neher - Laboratories for the careful typing
of the manuscript.
Leidschendam, August 1980 Erik van Doorn
TABLE OF CONTENTS
Chapter I : PRELIMINARIES
1.1 Markov processes I
1.2 Stochastic monotonicity 3
1.3 Birth-death processes 6
1.4 Some notation and terminology 8
Chapter 2: NATURAL BIRTH-DEATH PROCESSES
2. I Some basic properties II
2.2 The spectral representation 12
2.3 Exponential ergodicity 17
2.4 The moment problem and related topics 18
Chapter 3: DUAL BIRTH-DEATH PROCESSES
3. I Introduction • 22
3.2 Duality relations 23
3.3 Ergodic properties 26
Chapter 4: STOCHASTIC KlNOTONICITY: GENERAL RESULTS
4. I The case llO - 0 28
4.2 The case llO > 0 32
4.3 Properties of E(t) 35
Chapter 5 : STOCHASTIC KlNOTONICITY: DEPENDENCE ON THE INITIAL STATE DISTRIBUTION
5. I Introduction to the case of a fixed initial state 38
5.2 The transient and null recurrent process • • • 40
5.3 The positive recurrent process • • • • • • • • 41
5.4 The case of an initial state distribution with finite support 41
Chapter 6 : THE MIMls QUEUE LENGTH PROCESS
.
6. I Introduction 44
6.2 The spectral function 46
6.3 Stochastic mono tonicity 60
6.4 Exponential ergodicity • 65
Chapter 7: A QUEUEING MODEL WHERE POTENTIAL CUSTOMERS ARE DISCOURAGED BY QUEUE
LENGTH
7. I Introduction 66
7.2 The spectral representation 67
vi
7.3 Stochastic monotonicity and exponential ergodicity • • • • • . . 71
Chapter 8: LINEAR GROWTH BIRTH-DEATH PROCESSES
8. I Introduction • • . • • • 72
8.2 Stochastic monotonicity 74
Chapter 9: THE MEAN OF BIRTH-DEATH PROCESSES
9. I Introduction • • 76
9.2 Representations 76
9.3 Sufficient conditions for finiteness 80
9.4 Behaviour of the mean in special cases 82
Chapter 10 : THE TRUNCATED BIRTH-DEATH PROCESS
10.1 Introduction........ 87
10.2 Preliminaries • . . • • • • 88
10.3 The sign structure of P'(t) 93
10.4 Stochastic monotonicity 97
Appendix I: PROOF OF THE SIGN VARIATION DIMINISHING PROPERTY OF STRICTLY TOTALLY
POSITIVE MATRICES . • • • • • • • • • • • • • • • • • • • • • • . • 100
Appendix 2: ON PRODUCTS OF INFINITE MATRICES ••.•••••••••••..• 103
Appendix 3: ON THE SIGN OF CERTAIN QUANTITIES • • • • • • . • • • • •• . . •• 104
Appendix 4: PROOF OF THEOREM 10.2.8 •••••••.•••••.•••••••• 107
REFERENCES • • • • • • • • • • • • • • • • • • • • • • • • . • • • • • • • • • • I I I
NOTATION INDEX • • • • • • • . • . • • • • . • • • . • • • . • • • • • . • . . . I 13
AUTHOR INDEX . • • • • . • • • • • • • • • • • • • • • • • • • . • • • • • . • • I 16
SUBJECT INDEX . • • . • • • • • • • • • • • • • • . • • • • • • • • . . • • • • I I 7
I • PRELIMINARIES
1.1 tfarkov processes (CHUNG (1967). FREEDMAN (1971). REUTER (1957»
By a Map/'Ov pPOae88 we shall understand a continuous time stochastic process
{X(t): 0 S t < co} which has a denumerable state space S and which possesses the
M=kov pPOpepty. i.e .• for every n;" 2. 0 s tl < ..... < tn and any i l ...... in in
S one has
(I. I. I) pdX(tn )-i n I X(tl)-il •••••• X(tP I)-iP I}-
Pr{X(tn ) - i n I X(t n-I) - i n-I}'
The process is supposed to be temporaLLy homogeneous. i.e •• for every i. j in S
the conditional probability Pr{X(t+s) - j I Xes) - i} does not depend on s. In this
case we may put
(I. I. 2) p l.J. (t) - Pr{X(t+s) a j IX(s) - i} t ;" O.
The function p .. (.) is the transition ppobabiLity jUnation from the 8tate i to
lJ
the 8tate j. The ab80Lute di8tPibution at time t;" 0 is defined to be
{Pi (t): i € S}; where
(1.1.3) Pi (t) - pr{X(t) - i}. l.: p. (t)· I.
i 1
The absolute distribUtion at time t -0 is called the initiaL di8tPibution. We have
the obvious relation
(1.1.4) t ;" O.
(.».
The denumerable array of functions (p l.J. i. j € S. is the tronsition mat1'i:x:
of the Markov process. It satisfies for every i. j and s. t the conditions
(1.1.5) p .. (t);"O
lJ
(1.1.6) l.: p ..( t) - I
j lJ
(1.1.7) ~ Pik (s) Pkj (t) a Pij (t+s) •
Conversely. any array of functions (p .. (.» satisfying (1.1.5) - (1.1.7) and
lJ
the initial condition p .. (0) - cs •• (cs •• is Kroneckers's delta) for every i.j € S
lJ lJ lJ
and s. t;"O. is the transition matrix of a Markov process {X(t): OSt<oo}
2
for which (I. 1.2) holds.
A transition matrix is called standard iff
(I.1.8) H!!! p •• (t) • p .• (O) ;; 6 •••
t+o 1J 1J 1J
A standard transition matrix has the property that each Pij('} is uniformly
continuous. Furthermore, the right-hand derivatives
(I.1.9) q 1•J• • p!1 J• (O) • lUimO (P1'J' (t) - 61·J·}/t
(p! . (t) will denote the right-hand derivative when t. 0, and the two-sided
1J
derivative when t > O) exist, and they are finite except possibly when i· j.
Always
(I. L JO) i I' j,
(I.I.II) 1: q .. s -q .. s ....,
jl'i 1J 11
and inmost cases of practical interest
(I. I .12) q 1••1 > -... i € S,
(I.1.13) 1: q ••• 0 i € S.
j 1J
The matrix (q .. ) • (p! .(O}) is the q-matrire of the transition matrix (p .. (.});
1J 1J 1J
it is stable iff (1.1.12) holds and conservative iff it is stable and (1.1.13)
holds. The elements p .. (.) of a stable transition matrix, i.e., a transition
1J
matrix having a stable q-matrix, have continuous derivatives. The functions
p .. (.) satisfy the bacTaJard equa-tione
1J
(I.1.14)
iff the q-matrix is conservative.
Given a conservative q-matrix, i.e., a matrix (qij) with non-negative elements
off the main diagonal and satisfying both (1.1.12) and (1.1.13), then there exists
at least one standard transition matrix (p •• (.}) for which p! .(O) • q .. , but in
1J 1J 1J
general this transition matrix is not unique. When it is unique the conservative
q-matrix will be called noPmal.
For the elements p .. (.) of a stable transition \ll8trix to satisfy the fOThJal'd
1J
equations
(I.I.IS)
it is sufficient (but not necessary) that the q-matrix is normal.
3
Given a transition matrix (p •• (.». i.j € S. the state i is called absorbing iff
1J
Pii(t) • I for all t > O. A necessary and sufficient condition for this is
(I. I. 16)
The state i is .r.e OU1'1'ent iff
....
(1.1.17) !p .. (t)
o
u
otherwise it is called transient. A recurrent state i is called positive or null
according as Pii > 0 or Pii • O. where
(1.1.18) p ••• lim p •• (t).
1J t-+oo 1J
The latter limit exists for every i and j. A Markov process with transition matrix
(Pij('» is called transient (null PeOU1'1'ent. positive PeOU1'1'ent) iff every state
i € S is transient (null recurrent. positive recurrent). The process is said to be
i~duaible on S' c S when p •• (t) > 0 for all i.j € S' and t> O. It was shown by
1J
KENDALL (1959) that a Markov process with normal q-matrix has p •. (t) > 0 for all
1J
t> 0 iff there exists a finite sequence (i.kl ••••• kr.j) with r > 0 and satisfying
(1.1.19)
1.2 StOChastic monotonicity
We define R as the set of probability distribution vectors on E _ {O.I.2 •••• }.i.e ••
(1.2.1) r .• J}
1
(superscript T will denote transpose).
DEFINITION 1.2.1. Let ~(I) .~(2) € R. Then r(l) dominates r(2) (~(I) o~ ~(2» iff
for i • 1.2 ....
(I) (2)
(1.2.2) .r..r. ~ .r..r.
Jl!1 J Jl!l. J
The vector ~(I) stPiotly dominates ~(2) (~(I) 0> ~(2» iff strict inequality holds
in (l.i.2) for i • 1.2 •••••
DEFINITION 1.2.2. An operator L mapping R into R is monotone iff for every pair
~(I) .~(2) € R with ~(I) 0> ~(2)
(1.2.3) L(~(I» 0> L(~(2».
4
Now let {X(t): O:s; t < oo} be II Markov process with state space
=
S = E {O, I, }, say. The transition matrix (Pij('» defines a set (in fact,
a semigroup) of operators P t' t ~ 0, mapping R into R by means of
T
(\.2.4) (Pt(-r»J. - Ei r.1.p 1...J (t), -r - (rO' rl, •••• ) € R.
{Pt: O:S;t<oo} is the set of transition operutors of {X(t)}. If one denotes the
probability distribution vector of {X(t)} at time t ~ 0 by
(1.2.5)
the relation (1.1.4) may be written as
(1.2.6)
More generally one has for s, t ~ 0
(1.2.7)
as can easily be verified.
In view of definition 1.2.1 it is natural to introduce the concept of stochastic
monotonicity in terms of {X(t)} as follows.
DEFINITION 1.2.3. The process {X(t)} :i.sstoahaatiooZZy inareasing (deareaaing)
on the interval (tl, t2) iff for every pair TI ' T2 with o:s; tl:S; TI < T2 < t2 :s; ..
(1.2.8)
The process is striatZy stoahastiaaZZy inareasing (deareasing) iff strict
domination prevails throughout.
It appears from the next theorem that in the context of stochastic monotonicity,
.monotone transition operators are of particular interest.
THEOREM 1.2.4 (STOYAN (1977), Satz 4.2.4b). The Markov proaess {X(t)} is
stoahastiaaZZy inareasing (deareasing) on the intervaZ (tl, .. ) if the transition
operutors P t' t ~ 0, of {X(t)} are monotone and there e:x:ists a number T > 0
suah that {X(t)} is stoahastiaaZZy inareasing (deareasing) on the intervaZ
(tl, tl + T).
Several authors have given necessary and suffieient conditions for the transition
operator P t to be monotone for all t ~ 0 (KEItsON and KESTER (\977), KIRSTEIN (\976),
STOYAN (1977». The following is STOYAN's result.