Dynamic programming and inventory control

378 Pages·2011·2.871 MB·English

by  ebrary Inc., Bensoussan, Alain

DYNAMIC PROGRAMMING AND
INVENTORY CONTROL
Studies in
Probability, Optimization and Statistics
Volume 3
Published previously in this series
Vol. 2. J.W. Cohen, Analysis of Random Walks
Vol. 1. R. Syski, Passage Times for Markov Chains
ISSN 0928-3986 (print)
ISSN 2211-4602 (online)
Dynamic Programming and
Inventory Control

Alain Bensoussan
International Center for Risk and Decision Analysis, School of Management,
University of Texas at Dallas
Faculty of Business, The Hong Kong Polytechnic University and Graduate
Department of Financial Engineering, Ajou University

Amsterdam • Berlin • Tokyo • Washington, DC
© 2011 Alain Bensoussan and IOS Press.
All rights reserved. No part of this book may be reproduced, stored in a retrieval system,
or transmitted, in any form or by any means, without prior written permission from the publisher.
ISBN 978-1-60750-769-7 (print)
ISBN 978-1-60750-770-3 (online)
doi:10.3233/978-1-60750-770-3-i
Library of Congress Control Number: 2011933745
Publisher
IOS Press BV
Nieuwe Hemweg 6B
1013 BG Amsterdam
Netherlands
fax: [phone]
e-mail: [email protected]
Distributor in the USA and Canada
IOS Press, Inc.
4502 Rachael Manor Drive
Fairfax, VA 22032
USA
fax: [phone]
e-mail: [email protected]
LEGAL NOTICE
The publisher is not responsible for the use which might be made of the following information.
PRINTED IN THE NETHERLANDS
Contents
Chapter 1. INTRODUCTION 1
Chapter 2. STATIC PROBLEMS 3
2.1. NEWSVENDOR PROBLEM 3
2.2. EOQ MODEL 4
2.3. PRICE CONSIDERATIONS 5
2.4. SEVERAL PRODUCTS WITH SCARCE RESOURCE 7
2.5. CONTINUOUS PRODUCTION OF SEVERAL PRODUCTS 7
2.6. LEAD TIME 8
2.7. RANDOM DEMAND RATE: UNSATISFIED DEMAND LOST 11
Chapter 3. MARKOV CHAINS 13
3.1. NOTATION 13
3.2. CHAPMAN-KOLMOGOROV EQUATIONS 14
3.3. STOPPING TIMES 15
3.4. SOLUTION OF ANALYTIC PROBLEMS 15
3.5. ERGODIC THEORY 18
3.6. EXAMPLES 21
Chapter 4. OPTIMAL CONTROL IN DISCRETE TIME 25
4.1. DETERMINISTIC CASE 25
4.2. STOCHASTIC CASE: GENERAL FORMULATION 26
4.3. FUNCTIONAL EQUATION 30
4.4. PROBABILISTIC INTERPRETATION 33
4.5. UNIQUENESS 37
Chapter 5. INVENTORY CONTROL WITHOUT SET UP COST 41
5.1. NO SHORTAGE ALLOWED. 41
5.2. BACKLOG ALLOWED 46
5.3. DETERMINISTIC CASE 51
Chapter 6. ERGODIC CONTROL IN DISCRETE TIME 53
6.1. FINITE NUMBER OF STATES 53
6.2. ERGODIC CONTROL OF INVENTORIES WITH NO SHORTAGE 59
6.3. ERGODIC CONTROL OF INVENTORIES WITH BACKLOG 63
6.4. DETERMINISTIC CASE 66
Chapter 7. OPTIMAL STOPPING PROBLEMS 67
7.1. DYNAMIC PROGRAMMING 67
7.2. INTERPRETATION 69
7.3. PENALTY APPROXIMATION 72
v
vi CONTENTS
7.4. ERGODIC CASE 74
Chapter 8. IMPULSE CONTROL 79
8.1. DESCRIPTION OF THE MODEL 79
8.2. STUDY OF THE FUNCTIONAL EQUATION 81
8.3. ANOTHER FORMULATION 81
8.4. PROBABILISTIC INTERPRETATION 86
Chapter 9. INVENTORY CONTROL WITH SET UP COST 89
9.1. DETERMINISTIC MODEL 89
9.2. INVENTORY CONTROL WITH FIXED COST AND NO
SHORTAGE 94
9.3. INVENTORY CONTROL WITH FIXED COST AND BACKLOG 114
Chapter 10. ERGODIC CONTROL OF INVENTORIES WITH SET UP
COST 133
10.1. DETERMINISTIC CASE 133
10.2. ERGODIC INVENTORY CONTROL WITH FIXED COST AND
NO SHORTAGE 135
10.3. ERGODIC INVENTORY CONTROL WITH FIXED COST AND
BACKLOG 144
Chapter 11. DYNAMIC INVENTORY MODELS WITH EXTENSIONS 155
11.1. CAPACITATED INVENTORY MANAGEMENT 155
11.2. MULTI SUPPLIER PROBLEM 159
Chapter 12. INVENTORY CONTROL WITH MARKOV DEMAND 169
12.1. INTRODUCTION 169
12.2. NO BACKLOG AND NO SET-UP COST 169
12.3. BACKLOG AND NO SET UP COST 183
12.4. NO BACKLOG AND SET UP COST 194
12.5. BACKLOG AND SET UP COST 200
12.6. LEARNING PROCESS 207
Chapter 13. LEAD TIMES AND DELAYS 211
13.1. INTRODUCTION 211
13.2. MODELS WITH INVENTORY POSITION 211
13.3. MODELS WITHOUT INVENTORY POSITION 218
13.4. INFORMATION DELAYS 227
13.5. ERGODIC CONTROL WITH INFORMATION DELAYS 232
Chapter 14. CONTINUOUS TIME INVENTORY CONTROL 247
14.1. DETERMINISTIC MODEL 247
14.2. ERGODIC PROBLEM 251
14.3. CONTINUOUS RATE DELIVERY 252
14.4. LEAD TIME 253
14.5. NEWSVENDOR PROBLEM 254
14.6. POISSON DEMAND 255
14.7. ERGODIC CASE FOR THE POISSON DEMAND 263
14.8. POISSON DEMAND WITH LEAD TIME 265
CONTENTS vii
14.9. ERGODIC APPROACH FOR POISSON DEMAND WITH LEAD
TIME 272
14.10. POISSON DEMAND WITH LEAD TIME: USE OF INVENTORY
POSITION 279
14.11. ERGODIC THEORY FOR LEAD TIME WITH INVENTORY
POSITION 281
Chapter 15. INVENTORY CONTROL WITH DIFFUSION DEMAND 287
15.1. INTRODUCTION 287
15.2. PROBLEM FORMULATION 287
15.3. s,S POLICY 289
15.4. SOLVING THE Q.V.I 296
15.5. ERGODIC THEORY 298
15.6. PROBABILISTIC INTERPRETATION 303
Chapter 16. MEAN-REVERTING INVENTORY CONTROL 325
16.1. INTRODUCTION 325
16.2. DESCRIPTION OF THE PROBLEM 325
16.3. s,S POLICY 327
16.4. SOLUTION OF THE Q.V.I 339
Chapter 17. TWO BAND IMPULSE CONTROL PROBLEMS 341
17.1. INTRODUCTION 341
17.2. THE PROBLEM 341
17.3. a,A,b,B POLICY 343
17.4. SOLUTION OF THE Q.V.I. 351
17.5. COMPUTATIONAL ASPECTS 353
Bibliography 361
Appendix A. 363
A.1. PROOF OF LEMMAS 363
A.2. PROOF OF MEASURABLE SELECTION 364
A.3. EXTENSION TO U NON COMPACT 368
A.4. COMPACTNESS PROPERTIES 369
This page intentionally left blank
CHAPTER 1
INTRODUCTION
The objective of this book is to present a unified theory of Dynamic Pro-
gramming and Markov Decision Processes with its application to a major field of
Operations Research and Operations Management, Inventory Control. We will de-
velopmodelsindiscretetimeaswellasincontinuoustime. Incontinuoustime,the
diversity of situations is huge. We will not cover all of them and concentrate on
the models of interest to Inventory Control. In discrete time we will focus mostly
on infinite horizon models. This is also the situation when the Bellman equation
of Dynamic Programming is really a functional equation, the solution of which is
called the value function. With a finite horizon, Dynamic Programming leads to a
recursive relation, which is an easier situation. Of course, finite horizon problems
canalsobeconsideredasapproximationstoinfinitehorizonproblems,andthiswill
be used in our presentation. When the horizon is infinite, the discount plays an
essential role. An important problem concerns the behavior of the value function
when the discount factor tends to 1. The value function also tends to infinity, but
an average cost function becomes meaningful. This development is called ergodic
theory, an interesting aspect of which is that the solution, when it exists, can be
simplerthaninthecasewhenthediscountissmallerthan1. However,thetheoryis
morecomplex. Thesimplicityislinkedtothefactthattheproblembecomesstatic,
insteadofdynamic. Thecomplexitystemsfromthefactthatthisstaticproblemis
an averaging. The averaging procedure is not trivial and may be not intuitive.
Another important question concerns the difference between impulse control
andcontinuouscontrol. Thedifferenceisparticularlyunderstandableincontinuous
time. An impulse control will elicit an instantaneous jump for the state, whereas a
continuouscontrolcanonlyleadtoacontinuousevolutionofthestate. Inpractice,
this occurrence is linked to fixed costs, namely costs which arise just because a
decision is taken, whatever the impact of the decision may be. In discrete time,
the difference disappears in principle, since time is not continuous by construction.
However,fixedcostsremain. Theconsequenceisthatanappropriateformulationof
impulse control remains meaningful and useful in discrete time. Indeed, in discrete
time, the usual assumption is that the result of a decision materializes at the end
oftheperiod,whereasthedecisionistakenatthebeginningoftheperiod. Impulse
control in discrete time means that the result also materializes at the beginning of
the period, so instantaneously. We will also consider ergodic control in the context
ofimpulsecontrolandjustifysomesimpleruleswhicharecurrentlyusedinpractice.
In chapter 2, we shall introduce some of the classical static problems, which
are preliminary to the dynamic models of interest in inventory control. By static,
we mean that we have to solve an ordinary optimization problem. The decision
does not depend on time. Such models occur when one considers one period only,
1