Markov Chains: Models, Algorithms and Applications PDF
Preview Markov Chains: Models, Algorithms and Applications
International Series in Operations Research & Management Science Wai-Ki Ching Ximin Huang Michael K. Ng Tak-Kuen Siu Markov Chains Models, Algorithms and Applications Second Edition International Series in Operations Research & Management Science Volume 189 SeriesEditor FrederickS.Hillier StanfordUniversity,CA,USA SpecialEditorialConsultant CamilleC.Price StephenF.AustinStateUniversity,TX,USA Forfurthervolumes: http://www.springer.com/series/6161 Wai-Ki Ching • Ximin Huang Michael K. Ng (cid:129) Tak-Kuen Siu Markov Chains Models, Algorithms and Applications Second Edition 123 Wai-KiChing XiminHuang DepartmentofMathematics CollegeofManagement TheUniversityofHongKong GeorgiaInstituteofTechnology HongKong,SAR Atlanta,Georgia,USA MichaelK.Ng Tak-KuenSiu DepartmentofMathematics CassBusinessSchool HongKongBaptistUniversity CityUniversityLondon KowloonTong London HongKongSAR UnitedKingdom ISSN0884-8289 ISBN978-1-4614-6311-5 ISBN978-1-4614-6312-2(eBook) DOI10.1007/978-1-4614-6312-2 SpringerNewYorkHeidelbergDordrechtLondon LibraryofCongressControlNumber:2013931264 ©SpringerScience+BusinessMediaNewYork2006,2013 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation, broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer. PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations areliabletoprosecutionundertherespectiveCopyrightLaw. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. While the advice and information in this book are believed to be true and accurate at the date of publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with respecttothematerialcontainedherein. Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) To Mandyand myParents Wai-KiChing To myParents XiminHuang To Anna,Ceciliaand myParents MichaelK. Ng To CandyandmyParents Tak-Kuen Siu Preface TheaimofthisbookistooutlinetherecentdevelopmentofMarkovchainmodels andtheir applicationsin queueingsystems, manufacturingsystems, remanufactur- ing systems, inventory systems, ranking the importance of a web site, and also financialriskmanagement. Thisbookconsistsofeightchapters.InChapter1, we givea briefintroduction to the classical theory on both discrete and continuous time Markov chains. The relationship between Markov chains of finite states and matrix theory will also behighlighted.Someclassicaliterativemethodsforsolvinglinearsystemswillbe introducedforfindingthe stationarydistributionof a Markovchain.We then give thebasictheoriesandalgorithmsforhiddenMarkovmodels(HMMs)andMarkov decisionprocesses(MDPs). Chapter2discussestheapplicationsofcontinuoustimeMarkovchainstomodel queueingsystemsanddiscretetimeMarkovchainsforcomputingthePageRank,a rankingoftheimportanceofawebsiteintheInternet.Chapter3studiesMarkovian models for manufacturing and remanufacturing systems. We present closed form solutions and fast numerical algorithms for solving the captured systems. In Chapter4,wepresentasimplehiddenMarkovmodel(HMM)withfastnumerical algorithmsforestimatingthemodelparameters.Wethenpresentanapplicationof theHMMforcustomerclassification. Chapter 5 discusses Markov decision processes for customer lifetime values. Customerlifetimevalues(CLV)isanimportantconceptandquantityinmarketing management.WepresentanapproachbasedonMarkovdecisionprocessesforthe calculationofCLVusingrealdata. In Chapter6,we considerhigher-orderMarkovchainmodels.Inparticular,we discuss a class of parsimonioushigher-orderMarkov chain models. Efficient esti- mationmethodsformodelparametersbasedonlinearprogrammingarepresented. Contemporary research results on applications to demand predictions, inventory control, and financial risk measurement are presented. In Chapter 7, a class of parsimoniousmultivariateMarkovmodelsisintroduced.Again,efficientestimation methods based on linear programming are presented. Applications to demand predictions,inventorycontrolpolicy,andmodelingcreditratingsdataarediscussed. vii viii Preface InChapter8,werevisithiddenMarkovmodels.Weproposeanewclassofhidden Markov models with efficient algorithms for estimating the model parameters. Applicationstomodelinginterestrate,creditratings,anddefaultdataarediscussed. TheauthorswouldliketothankOperationalResearchSociety,OxfordUniversity Press, Palgrave,Taylor& Francis’, Wiley & Sons, JournalofCredit Risk Incisive Media, Incisive Financial Publishing Limited, and Yokohama Publishers for their permission to reproduce the material in this book. The authors would also like to thank Werner Fortmann, Gretel Fortmann, and Mimi Lui for their help in the preparationofthisbook. HongKongSAR Wai-KiChing Atlanta,Georgia XiminHuang Kowloon,HongKongSAR MichaelK.Ng Sydney,Australia Tak-KuenSiu Contents 1 Introduction .................................................................. 1 1.1 MarkovChains.......................................................... 1 1.1.1 ExamplesofMarkovChains................................... 2 1.1.2 Thenth-StepTransitionMatrix................................ 5 1.1.3 IrreducibleMarkovChainandClassificationsofStates...... 7 1.1.4 AnAnalysisoftheRandomWalk ............................. 8 1.1.5 SimulationofMarkovChainswithEXCEL................... 10 1.1.6 BuildingaMarkovChainModel............................... 11 1.1.7 StationaryDistributionofaFiniteMarkovChain............. 13 1.1.8 ApplicationsoftheStationaryDistribution ................... 18 1.2 ContinuousTimeMarkovChainProcess.............................. 19 1.2.1 AContinuousTwo-StateMarkovChain....................... 21 1.3 IterativeMethodsforSolvingLinearSystems......................... 22 1.3.1 SomeResultsonMatrixTheory ............................... 23 1.3.2 SplittingofaMatrix............................................ 24 1.3.3 ClassicalIterativeMethods..................................... 26 1.3.4 SpectralRadius ................................................. 28 1.3.5 SuccessiveOver-Relaxation(SOR)Method .................. 29 1.3.6 ConjugateGradientMethod.................................... 30 1.3.7 ToeplitzMatrices ............................................... 34 1.4 HiddenMarkovModels................................................. 35 1.5 MarkovDecisionProcess............................................... 37 1.5.1 StationaryPolicy................................................ 41 1.6 Exercises................................................................. 42 2 QueueingSystemsandtheWeb............................................ 47 2.1 MarkovianQueueingSystems.......................................... 47 2.1.1 AnM/M/1=n(cid:2)2QueueingSystem ........................... 48 2.1.2 AnM/M/s=n(cid:2)s(cid:2)1QueueingSystem....................... 49 2.1.3 AllocationoftheArrivalsinaSystem ofM/M/1/1Queues ........................................... 51 ix
The list of books you might like
