Max-Plus Linear Stochastic Systems and Perturbation Analysis

323 Pages·2007·3.33 MB·English

by  Bernd Heidergott (eds.)

Max-Plus Linear Stochastic Systems
and
Perturbation Analysis
THE INTERNATIONAL SERIES
ON DISCRETE EVENT DYNAMIC SYSTEMS
Series Editor
Yu-Chi Ho
Harvard University
SUPERVISION OF PETRI NETS
Geert Stremersch
ISBN: 0-7923-7486-X
ANALYSIS OF MANUFACTURING ENTERPRISES: An Approach to Leveraging Value Delivery
Processes for Competitive Advantage
N. Viswanadham
ISBN: 0-7923-8671-X
INTRODUCTION TO DISCRETE EVENT SYSTEMS
Christos G. Cassandras, St6phane Lafortune
ISBN: 0-7923-86094
OBJECT-ORIENTED COMPUTER SIMULATION OF DISCRETE-EVENT SYSTEMS
Jerzy Tyszer
ISBN: 0-7923-8506-3
TIMED PETRI NETS: Theory and Application
Jiacun Wang
ISBN: 0-7923-8270-6
SUPERVISORY CONTROL OF DISCRETE EVENT SYSTEMS USING PETRI NETS
John O. Moody and Panos J. Antsaklis
ISBN: 0-7923-8199-8
GRADIENT ESTIMATION VIA PERTURBATION ANALYSIS
P. Glasserman
ISBN: 0-7923-90954
PERTURBATION ANALYSIS OF DISCRETE EVENT DYNAMIC SYSTEMS
Yu-Chi Ho and Xi-Ren Cao
ISBN: 0-7923-9174-8
PETRI NET SYNTHESIS FOR DISCRETE EVENT CONTROL OF MANUFACTURING
SYSTEMS
MengChu Zhou and Frank DiCesare
ISBN: 0-7923-9289-2
MODELING AND CONTROL OF LOGICAL DISCRETE EVENT SYSTEMS
Ratnesh Kumar and Vijay K. Garg
ISBN: 0-7923-9538-7
UNIFORM RANDOM NUMBERS: THEORY AND PRACTICE
Shu Tezuka
ISBN: 0-7923-9572-7
OPTIMIZATION OF STOCHASTIC MODELS: THE INTERFACE BETWEEN SIMULATION
AND OPTIMIZATION
Georg Ch. Pflug
ISBN: 0-7923-9780-0
CONDITIONAL MONTE CARLO: GRADIENT ESTIMATION AND OPTIMIZATION
APPLICATIONS
Michael FU and Jian-Qiang HU
ISBN: 0-7923-98734
MAX-PLUS LINEAR
STOCHASTIC SYSTEMS
AND
PERTURBATION
ANALYSIS
by
Bernd Heidergott
Vrije Universiteit, Amsterdam, The Netherlands
^ Springer
Bernd Heidergott
Vrije Universiteit
Faculty of Economics and Business Administration
Department of Econometrics and Operations Research
DeBoelelaanllOS
1081 HV Amsterdam
Email: [email protected]
Library of Congress Control Number: 2006931638
Max-Plus Linear Stochastic Systems and Perturbation Analysis
by Bernd Heidergott
ISBN-13: 978-0-387-35206-0
ISBN-10: 0-387-35206-6
e-ISBN-13: 978-0-387-38995-0
e-ISBN-10: 0-387-38995-4
Printed on acid-free paper.
Parts of this manuscript [in Chapter 3] were reprinted by permission,
[Heidergott, B. 2001. A differential calculus for random matrices
with applications to (max, +)-linear stochastic systems. Math. Oper.
Res. 26 679-699]. Copyright 2006, the Institute for Operations
Research and the Management Sciences, 7240 Parkway Drive, Suite
310, Hanover, MD 21076 USA.
© 2006 Springer Science+Business Media, LLC
All rights reserved. This work may not be translated or copied in
whole or in part without the written permission of the publisher
(Springer Science+Business Media, LLC, 233 Spring Street, New
York, NY 10013, USA), except for brief excerpts in connection with
reviews or scholarly analysis. Use in connection with any form of
information storage and retrieval, electronic adaptation, computer
software, or by similar or dissimilar methodology now know or
hereafter developed is forbidden.
The use in this publication of trade names, trademarks, service marks
and similar terms, even if the are not identified as such, is not to be
taken as an expression of opinion as to whether or not they are
subject to proprietary rights.
9 8 7 6 5 4 3 21
springer.com
Preface
This monograph presents perturbation analysis for max-plus linear stochastic
systems. Max-Plus algebra has been successfully applied to many areas of sto
chastic networks. For example, applying Kingman's subadditive ergodic theorem
to max-plus linear queuing networks, one can establish ergodicity of the inverse
throughput. More generally, applying backward coupling arguments, stability
results for max-plus linear queuing systems follow. In addition to that, stability
results for waiting times in open queuing networks can be obtained.
Part I of this book is a self-contained introduction to stochastic max-plus
linear systems. Chapter 1 provides an introduction to the max-plus algebra.
More specifically, we introduce the basic algebraic concepts and properties of
max-plus algebra. The emphasis of the chapter is on modeling issues, that is,
we will discuss what kind of discrete event systems, such as queueing networks,
can be modeled by max-plus algebra. Chapter 2 deals with the ergodic theory
for stochastic max-plus linear systems. The common approaches are discussed
and the chapter may serve as a reference to max-plus ergodic theory.
Max-Plus algebra is an area of intensive research and a complete treatment
of the theory of max-plus linear stochastic systems is beyond the scope of this
book. An area of applications of max-plus linearity to queuing systems not
covered in this monograph is the generalization of Lindley-type results for the
GI/G/1 queue to max-plus linear queuing networks. For example, in [1, 2] Alt-
man, Gaujal and Hordijk extend a result of Hajek [59] on admission control to
a GI/G/1 queue to max-plus linear queuing networks. FYirthermore, the focus
of this monograph is on stochastic systems and we only briefly present the main
results of the theory of deterministic max-plus systems. Readers particularly
interested in deterministic theory are referred to [10] and the more recent book
[65]. For this reason, network calculus, a min-plus based mathematical theory
for analyzing the flow in deterministic queueing networks, is not covered either
and readers interested in this approach are referred to [79]. Various approaches
that are extensions of, or, closely related to max-plus algebra are not addressed
in this monograph. Readers interested in min-max-plus systems are referred to
[37, 72, 87, 98]. References on the theory of non-expansive maps are [43, 49, 58],
and for MM functions we refer to [38, 39]. For applications of max-plus methods
to control theory, we refer to [85].
Part II studies perturbation analysis of max-plus linear systems. Our ap
proach to perturbation analysis of max-plus linear systems mirrors the hierar-
vi Preface
chical structure inherited by the structure of the problem. More precisely, the
individual chapters will have the following internal structure:
Random variable level: We set off with carefully developing a concept of
differentiation for random variables and distributions, respectively.
Matrix level: For the kind of applications we have in mind, the dynamic of a
system is modeled by random matrices, the elements of which are (sums
of) simple random variables. Our theory will provide sufficient conditions
such that (higher-order) differentiability or analyticity of the elements of
a matrix in the max-plus algebra implies (higher-order) differentiability
or analyticity of the matrix itself
System level: For (higher-order) differentiability or analyticity we then pro
vide product rules, that is, we will establish conditions under which the
(random) product (or sum) of differentiable (respectively, analytic) ma
trices is again differentiable (respectively, analytic). In other words, we
establish sufficient conditions for (higher-order) differentiability or ana
lyticity of the state-vector of max-plus linear systems.
Performance level: The concept of differentiability is such that it allows
statements about (higher-order) derivatives or Taylor series expansions
for a predefined class of performance functions applied to max-plus linear
systems. We will work with a particular class of performance functions
that covers many functions that are of interest in applications and that is
most suitable to work with in a max-plus environment.
The reason for choosing this hierarchical approach to perturbation analysis is
that we want to provide conditions for differentiability that are easy to check.
One of the highlights of this approach is that we will show that if a particular ser
vice time in a max-plus linear queuing network is differentiable [random variable
level], then the matrix modeling the network dynamic is differentiable [matrix
level] and by virtue of our product rule of differentiation the state-vector of
the system is differentiable [system level]. This fact can then be translated into
expressions for the derivative of the expected value of the performance of the
system measured by performance functions out of a predefined class [perform
ance level]. We conclude our analysis with a study of Taylor series expansions
of stationary characteristics of max-plus linear systems.
Part II is organized as follows. Chapter 3 introduces our concept of weak
differentiation of measures, called ^-differentiation of measures. Using the al
gebraic properties of max-plus, we extend this concept to max-plus matrices
and vectors and thereby establish a calculus of unbiased gradient estimators. In
Chapter 4, we extend the ^-differentiation approach of Chapter 3 to higher-
order derivatives. In Chapter 5 we turn our attention to Taylor series expansions
of max-plus systems. This area of application of max-plus linearity has been
initiated by Baccelli and Schmidt who showed in their pioneering paper [17]
that waiting times in max-plus linear queuing networks with Poisson-A-arrival
stream can be obtained via Taylor expansions w.r.t. A, see [15]. For certain
Preface vii
classes of open queuing networks this yields a feasible way of calculating the
waiting time distribution, see [71]. Concerning analyticity of closed networks,
there are promising first results, see [7], but a general theory has still to be de
veloped. We provide a unified approach to the aforementioned results on Taylor
series expansions and new results will be established as well.
A reader interested in an introduction to stochastic max-plus linear systems
will benefit from Part I of this book, whereas the reader interested in pertur
bation analysis, will benefit from Chapter 3 and Chapter 4, where the theory
of ^-differentiation is developed. The full power of this method can be appre
ciated when studying Taylor series expansions, and we consider Chapter 5 the
highlight of the book.
Notation and Conventions
This monograph covers two areas in applied probability that have been disjoint
until now. Both areas (that of max-plus linear stochastic systems and that of per
turbation analysis) have developed their own terminology independently. This
has led to notational conventions that are sometimes not compatible. Through
out this monograph we stick to the established notation as much as possible.
In two prominent cases, we even choose for ambiguity of notation in order to
honor notational conventions. The first instance of ambiguity will be the symbol
9. More specifically, in ergodic theory of max-plus linear stochastic systems (in
the first part of this monograph) the shift operator on the sample space fi, tra
ditionally denoted by 6, is the standard means for analysis. On the other hand,
the parameter of interest in perturbation analysis is typically denoted by 9 too,
and we will follow this standard notation in the second part of the monograph.
Fortunately, the shift operator is only used in the first part of the monograph
and from the context it will always be clear which interpretation of 9 is meant.
The second instance of ambiguity will be the symbol A. More specifically, for
ergodic theory of max-plus linear stochastic systems we will denote by A the
Lyapunov exponent of the system and A will also be used to denote the inten
sity of a given Poisson process. Both notations are classical and it will always
be clear from the context which interpretation of A is meant.
Throughout this monograph, we assume that an underlying probability
space [Q,,A,P) is given and that any random variable introduced is defined
on (n,yl, P). Furthermore, we will use the standard abbreviation 'i.i.d.' for 'in
dependent and identically distributed,' and 'a.s.' for 'almost surely.' To avoid
an inflation of subscripts, we will suppress in Part II the subscript 9 when this
causes no confusion. In addition to that, we will write E^ in order to denote the
expected value of a random variable evaluated at 9. Furthermore, let a <b, for
a, 6 6 K, and let / : (a, &) —* E be n times differentiable with respect to 9 on
(a, 6), then we write ^|g^g f{9) for the n*'' derivative of / evaluated at 9o.
We will frequently work with the set R U {—oo} and we introduce the following
convention: for any a; 6 R we set a; -|- (—oo) = —oo 4- a; = —oo = —oo — x,
X — (—oo) = oo, and —oo + (—oo) = —oo and —oo — (—oo) = 0.
viii Preface
Acknowledgements
This work was supported in parts by EC-TMR project ALAPEDES under
Grant ERBFMRXCT960074 and in parts by Deutsche Forschungsgemeinschaft
Grant He 3139/1-1.
The author wants to thank Frangois Baccelh and Dohy Hong for many fruit
ful discussions. Furthermore, we are grateful to Bruno Gaujal for bringing the
problem addressed in Remark 1.5.2 and Remark 1.5.3 in Section 1.5 to our
attention. The presentation of max-plus ergodic theory benefitted from our di
scussions with Stephane Gaubert and Jean Mairesse. We are also grateful to
Arie Hordijk and Haralambie Leahu for many insightful discussions on the topic
of measure-valued differentiation. Finally, the author wants to thank TU Delft,
EURANDOM, TU Eindhoven, and Vrije Universiteit Amsterdam for giving him
the opportunity to work and continue working on this project during the various
stages of his professional career.
The author Amsterdam, The Netherlands, May 2006
Contents
Preface
I Max-Plus Algebra 1
1 Max-Plus Linear Stochastic Systems 3
1.1 The Max-Plus Algebra 3
1.2 Heap of Pieces 7
1.3 The Projective Space 10
1.4 Petri Nets 10
1.4.1 Basic Definitions 10
1.4.2 The Max-Plus Recursion for Firing Times 12
1.4.3 Autonomous Systems and Irreducible Matrices 15
1.4.4 The Max-Plus Recursion for Waiting Times in Non-
Autonomous Event Graphs 17
1.5 Queueing Systems 20
1.5.1 Queueing Networks 21
1.5.2 Examples of Max-Plus Linear Systems 24
1.5.3 Sample Path Dynamics 30
1.5.4 Invariant Queueing Networks 44
1.5.5 Condition (A) Revisited 46
1.5.6 Beyond Fixed Support: Patterns 48
1.6 Bounds and Metrics 49
1.6.1 Real-Valued Upper Bounds for Semirings 49
1.6.2 General Upper Bounds over the Max-Plus Semiring . .. 51
1.6.3 The Max-Plus Semiring as a Metric Space 54
2 Ergodic Theory 59
2.1 Deterministic Limit Theory 60
2.2 Subadditive Ergodic Theory 66
2.2.1 The Irreducible Case 71
2.2.2 The Reducible Case 75
2.2.3 Variations and Extensions 80
2.3 Stability Analysis of Waiting Times 86
CONTENTS
2.4 Harris Recurrence 93
2.5 Limits in the Projective Space 99
2.5.1 Countable Models 100
2.5.2 General Models 102
2.5.3 Periodic Regimes of Deterministic Max-Plus DES 105
2.5.4 The Cycle Formula 106
2.6 Lyapunov Exponents via Second Order Limits 107
2.6.1 The Projective Space 108
2.6.2 Backward Coupling Ill
II Perturbation Analysis 117
3 A Max-Plus Differential Calculus 119
3.1 Measure-Valued DifTerentiation 120
3.2 The Space Cp 128
3.3 'D-Derivatives of Random Matrices 132
3.4 An Algebraic Tool for 25-Derivatives 136
3.5 Rules for Cp-Differentiation of Random Matrices 141
3.6 Max-Plus Gradient Estimators 146
3.6.1 Homogeneous Recurrence Relations 147
3.6.2 Inhomogeneous Recurrence Relations 149
4 Higher-Order I>-Derivatives 151
4.1 Higher-Order 2?-Derivatives 151
4.2 Higher-Order Differentiation in Cp-Spaces 159
4.3 Higher-Order P-Differentiation on Al^^-^ 162
4.4 Rules of Cp-Differentiation 165
4.5 P-Analyticity 170
4.6 P-Analyticity on M^^'^ 174
5 Taylor Series Expansions 179
5.1 Finite Horizon Experiments 180
5.1.1 The General Result 180
5.1.2 Analyticity of Waiting Times 183
5.1.3 Variability Expansion 186
5.2 Random Horizon Experiments 201
5.2.1 The 'Halted' Max-Plus System 202
5.2.2 The Time Until Two Successive Breakdowns 222
5.3 The Lyapunov Exponent 229
5.3.1 Analytic Expansion of the Lyapunov Exponent 230
5.3.2 The BernoulU Scheme 236
5.3.3 A Note on the Elements of the Taylor Series for the
Bernoulli System 240
5.4 Stationary Waiting Times 243
5.4.1 Cycles of Waiting Times 245