Semi-Markov processes : applications in system reliability and maintenance

252 Pages·2015·2.925 MB·English

by  Franciszek Grabski

Semi-Markov Processes: Applications in System
Reliability and Maintenance
Semi-Markov Processes:
Applications in System
Reliability and
Maintenance
Franciszek Grabski
Polish Naval University
Gdynia, Poland
AMSTERDAM (cid:129) BOSTON (cid:129) HEIDELBERG (cid:129) LONDON (cid:129) NEW YORK (cid:129) OXFORD
PARIS (cid:129) SAN DIEGO (cid:129) SAN FRANCISCO (cid:129) SYDNEY (cid:129) TOKYO
Elsevier
Radarweg29,POBox211,1000AEAmsterdam,Netherlands
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UK
225WymanStreet,Waltham,MA02451,USA
Copyright(cid:2)c 2015ElsevierInc.Allrightsreserved.
Nopartofthispublicationmaybereproduced,storedinaretrievalsystemortransmittedin
anyformorbyanymeanselectronic,mechanical,photocopying,recordingorotherwise
withoutthepriorwrittenpermissionofthepublisher.
PermissionsmaybesoughtdirectlyfromElsevier’sScience&TechnologyRights
DepartmentinOxford,UK:phone[phone];fax[phone];
email:[e-mail]tyourrequestonlineby
visitingtheElsevierwebsiteathttp://elsevier.com/locate/permissions,andselectingObtaining
permissiontouseElseviermaterial.
Notice
Noresponsibilityisassumedbythepublisherforanyinjuryand/ordamagetopersonsor
propertyasamatterofproductsliability,negligenceorotherwise,orfromanyuseoroperation
ofanymethods,products,instructionsorideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-12-800518-7
ForinformationonallElsevierpublications
visitourwebsiteatstore.elsevier.com
ThisbookhasbeenmanufacturedusingPrintonDemandtechnology.Eachcopyisproduced
toorderandislimitedtoblackink.Theonlineversionofthisbookwillshowcolorfigures
whereappropriate.
Dedication
To my wife Marcelina
Preface
The semi-Markov processes were introduced independently and almost simultane-
ouslybyLevy[70],Smith[92],andTakacs[94]in1954-1955.Theessentialdevelop-
mentsofsemi-MarkovprocessestheorywereproposedbyPyke[85,86],Cinlar[15],
KorolukandTurbin[60–62],Limnios[72],Takacs[95].Herewepresentonlysemi-
Markovprocesseswithadiscretestatespace.Asemi-Markovprocessisconstructed
bytheMarkovrenewalprocess,whichisdefinedbytherenewalkernelandtheinitial
distributionorbyothercharacteristicsthatareequivalenttotherenewalkernel.
Semi-Markov Processes: Applications in System Reliability and Maintenance
consistsofapreface,15relativelyshortchapters,asummary,andabibliography.
Chapter 1 is devoted to the discrete state space Markov processes, especially
continuous-time Markov processes and homogeneous Markov chains. The Markov
processes are an important class of the stochastic processes. This chapter covers
some basic concepts, properties, and theorems on homogeneousMarkov chains and
continuous-timehomogeneousMarkovprocesseswithadiscretesetofstates.
Chapter 2 provides the definitions and basic properties related to a discrete state
spacesemi-Markovprocess.Thesemi-Markovprocessisconstructedbytheso-called
Markov renewal process. The Markov renewal process is defined by the transition
probabilities matrix, called the renewal kernel, and by an initial distribution or by
othercharacteristicsthatareequivalenttotherenewalkernel.Theconceptspresented
are illustrated by some examples. Elements of the semi-Markov process statistical
estimationarealsopresentedinthechapter.Here,theestimationoftherenewalkernel
elements is considered by observing oneor many sample paths in the time interval,
or givennumberof thestate changes.Basicconceptsof thenonhomogeneoussemi-
Markovprocessestheoryarealsointroducedinthechapter.
Chapter 3 is devoted to some characteristics and parameters of the semi-Markov
process.Arenewalkernelandaninitialdistributioncontainfullinformationaboutthe
processandtheyallowustofindmanycharacteristicsandparametersoftheprocess,
whichwecantranslateonthereliabilitycharacteristicsinthesemi-Markovreliability
model.Thecumulativedistributionfunctionsofthefirstpassagetimefromthegiven
statestoasubsetofstates,andexpectedvaluesandsecondmomentscorrespondingto
them,areconsideredinthischapter.Theequationsforthesequantitiesarepresented
here. Moreover, the chapter discusses a concept of interval transition probabilities
and the Feller equations are also derived. Karolyuk and Turbin theorems of the
limiting probabilities are also presented here. Furthermore, the reliability and main-
tainabilitycharacteristicsandparametersinsemi-Markovmodelsareconsideredinthe
chapter.
xii Preface
Chapter 4 is concerned with the application of the perturbed semi-Markov pro-
cesses in reliability problems. The results coming from the theory of semi-Markov
processes perturbations allow us to find the approximate reliability function. The
perturbedsemi-Markovprocesses are defined in differentways by differentauthors.
Thistheoryhasarichliterature.Inthischapterwepresentonlyafewofthesimplest
types of perturbed SM processes. All concepts of the perturbed SM processes are
explainedin the same simple example. The last section is devotedto the state space
aggregationmethod.
InChapter5the randomprocessesdeterminedbythe characteristics ofthe semi-
Markovprocessare considered.First is a renewal processgeneratedby returntimes
ofagivenstate.Thesystemsofequationsforthedistributionandexpectationofthem
havebeenderived.Thelimittheoremfortheprocessisformulatedbytheadoptionof
atheoremoftherenewaltheory.Thelimitingpropertiesofthealternatingprocessand
integralfunctionalsofthesemi-Markovprocessarealsopresentedinthischapter.The
chaptercontainsillustrativeexamples.
The semi-Markov reliability model of two different units of a renewable cold
standbysystemandtheSMmodelofahospitalelectricalpowersystemarediscussed
inChapter6.
InChapter7,themodelofmultistageoperationwithoutrepairandthemodelwith
repairareconstructed.Applicationofresultsofsemi-Markovprocesstheoryallowed
tocalculatethereliabilityparametersandcharacteristicsofthemultistageoperation.
Themodelsareappliedformodelingthemultistagetransportoperationprocesses.
In Chapter 8, the semi-Markov model of the load rate process is discussed. The
speedofacarandtheloadrateofashipengineareexamplesoftherandomloadrate
process.Theconstructionofdiscretestatemodeloftherandomloadrateprocesswith
continuoustrajectoriesleadstothesemi-Markovrandomwalk.Estimatingthemodel
parameters and calculating the semi-Markov process characteristics and parameters
giveusthepossibilitytoanalyzethesemi-Markovloadrate.
Chapter9containsthesemi-Markovmodelofthemultitaskoperationprocess.
Chapter 10 is devoted to the semi-Markov failure rate process. In this chapter,
the failure rate is assumed to be a stochastic process with nonnegative and right-
continuous trajectories. The reliability function is defined as an expectation of a
function of that random process. Particularly, the failure rate can be defined by
the discrete state space semi-Markov process. The theorem concerning the renewal
equations for the conditional reliability function with a semi-Markov process as a
failurerateispresented.Thereliabilityfunctionwitharandomwalkasafailurerate
isinvestigated.ForPoissonfailurerateprocessandFurry-Yulefailurerateprocessthe
reliabilityfunctionsarepresented.
InChapter11,timetoapreventiveserviceoptimizationproblemisformulated.The
semi-Markovmodeloftheoperationprocessallowedustoformulatetheoptimization
problem. A theorem containing the sufficient conditions of the existing solution is
formulatedandproved.Anexampleexplainsandillustratesthepresentedproblem.
In Chapter 12, a semi-Markov modelof system componentdamageis discussed.
Themodelspresentedheredealwithunrepairablesystems. Themultistate reliability
functionsandcorrespondingexpectations,secondmoments,andstandarddeviations
Preface xiii
areevaluatedforthepresentedcasesofthecomponentdamage.Aspecialcaseofthe
model is a multistate model with two kinds of failures. A theorem dealing with the
inverseproblemforasimpledamageexponentialmodelisformulatedandproved.
In Chapter 13, some results of investigation of the multistate monotone system
withcomponentsmodeledbytheindependentsemi-Markovprocessesarepresented.
We assume that the states of system components are modeled by the independent
semi-Markov processes. Some characteristics of a semi-Markov process are used as
reliability characteristicsofthesystem components.Inthechapter,thebinaryrepre-
sentationofthemultistatemonotonesystemsisdiscussed.Thepresentedconceptsand
modelsareillustratedbysomenumericalexamples.
The semi-Markov models of functioning maintenance systems, which are called
maintenance nets, are presented in Chapter 14. Elementary maintenance operations
form the states of a SM model. Someconcepts and results of Semi-Markovprocess
theoryprovidethepossibilityofcomputingimportantcharacteristicsandparameters
of the maintenance process. Two semi-Markov models of maintenance nets are
discussedinthechapter.
In Chapter 15, basic concepts and results of the theory of semi-Markov decision
processes are presented. The algorithm of optimizing a SM decision process with a
finitenumberofstatechangesisdiscussedhere.Thealgorithmisbasedonadynamic
programming method. To clarify it, the SM decision model for the maintenance
operation is shown. The optimization problem for the infinite duration SM process
and the Howard algorithm, which enables us to find the optimal stationary strategy
arealsodiscussedhere.Toexplainthisalgorithm,adecisionproblemforarenewable
seriessystemispresented.
Thebookisprimarilyintendedforresearchersandscientists dealingwith mathe-
maticalreliabilitytheory(mathematicians)andpractitioners(engineers)dealingwith
reliabilityanalysis.Thebookisaveryhelpfultoolforscientists, Ph.D.students,and
M.Sc.studentsintechnicaluniversitiesandresearchcenters.
Acknowledgments
I would like to thank Professor Krzysztof Kołowrocki for mobilization to write this
book,thecouncilforits effortsto release itandvaluablecommentson itscontent. I
am gratefulto DrAgata-Załe¸ska Fornalforherfriendlyfavorininsightfullinguistic
revisionofmybookandvaluablecomments.IgratefullyacknowledgeDrErinHill-
Parks,Associate AcquisitionsEditor,forkindcooperationandassistance inmeeting
therequirementsofElseviertosignapublishingcontract.IamgratefultoCariOwen,
EditorialProjectManager,forhercareandkindassistanceindealingwiththeeditors.
Finally,IthankmywifeMarcelinaforherhelpincomputerproblemsintheprocess
ofwritingabookaswellasherpatienceandgreatsupport.
FranciszekGrabski
Discrete state space Markov 1
processes
Abstract
The Markov processes are an important class of the stochastic processes. The Markov
property means that evolution of the Markov process in the future depends only on the
presentstateanddoesnotdependonpasthistory.TheMarkovprocessdoesnotremember
thepastifthepresent stateisgiven.Hence,theMarkovprocessiscalledtheprocesswith
memorylessproperty.Thischaptercoverssomebasicconcepts,properties,andtheoremson
homogeneous Markovchainsandcontinuous-timehomogeneous Markovprocesseswitha
discretesetofstates.Thetheoryofthosekindsofprocessesallowsustocreatemodelsof
realrandomprocesses,particularlyinissuesofreliabilityandmaintenance.
Keywords: Markov process, Homogeneous Markov chain, Poisson process, Furry-Yule
process,Birthanddeathprocess
1.1 Basic definitions and properties
Definition1.1.Astochasticprocess{X(t):t∈T}withadiscrete(finiteorcountable)
state space S is said to be a Markov process, if for all i,j,i0,i1,...,in−1 ∈S and
t0,t1,...,tn,tn+1 ∈R+ suchthat0(cid:2)t0 <t1 <···tn <tn+1,
P(X(tn+1)=j|X(tn)=i,X(tn−1)=in−1,...,X(t0)=i0)
=P(X(tn+1)=j|X(tn)=i). (1.1)
If t0,t1,...,tn−1 are interpreted as the moments from the past, tn as the present
instant,andtn+1asthemomentinthefuture,thentheabove-mentionedequationsays
that the probability of the future state is independent of the past states, if a present
state is given. So, we can say that evolution of the Markov process in the future
dependsonlyonthepresentstate.TheMarkovprocessdoesnotrememberthepastif
thepresentstateisgiven.Hence,theMarkovprocessiscalledthestochasticprocess
withmemorylessproperty.
From the definition of the Markov process it follows that any process with
independentincrementsistheMarkovprocess.
IfT=N ={0,1,2,...},theMarkovprocessissaidtobeaMarkovchain,ifT=
0
R+ =[0,∞),itiscalledthecontinuous-timeMarkovprocess.Lettn =u, tn+1 =τ.
Theconditionalprobabilities
p (u,s)=P(X(τ)=j|X(u)=i), i,j∈S (1.2)
ij
aresaidtobethetransitionprobabilitiesfromthestateiatthemomentu,tothestate
jatthemoments.
Semi-MarkovProcesses:ApplicationsinSystemReliabilityandMaintenance.http://dx.doi.org/10.1016/B978-0-12-800518-7.00001-6
Copyright(cid:4)c2015ElsevierInc.Allrightsreserved.
2 Semi-MarkovProcesses:ApplicationsinSystemReliabilityandMaintenance
Definition1.2.TheMarkovprocess{X(t):t∈T}iscalledhomogeneous,ifforall
i,j∈Sandu, s∈T,suchthat0(cid:2)u<s,
p (u,s)=p (s−u). (1.3)
ij ij
It means that the transition probabilities are the functions of a difference of the
momentssandu.Substitutingt=s−uweget
p (t)=P(X(s−u)=j|X(u−u)=i)=P(X(t)=j|X(0)=i), i,j∈S, t(cid:3)0.
ij
The number p (t) is called a transition probability from the state i to the state j
ij
duringthetimet.
If {X(t):t∈R+} is a process with the stationary independentincrements, taking
valuesonadiscretestatespaceS,then
P(X(t+h)=j, X(h)=i)
p (t)=P(X(t+h)=j|X(h)=i)=
ij P(X(h)=i)
P(X(t+h)−X(h)=j−i, X(h)=i)
=
P(X(h)=i)
P(X(t+h)−X(h)=j−i)P(X(h)=i)
=
P(X(h)=i)
=P(X(t+h)−X(h)=j−i).
Therefore, any process with the stationary independent increments is a homoge-
neousMarkovprocesswithtransitionprobabilities
p (t)=P(X(t+h)−X(h)=j−i). (1.4)
ij
For the homogeneous Markov process with a discrete state space, the transition
probabilitiessatisfythefollowingconditions:
(a) p(cid:2)ij(t)(cid:3)0, t∈T,
(b) p (t)=1,
ij
i∈S (cid:2) (1.5)
(c) p (t+s)= p (t)p (s), t∈T, s(cid:3)0.
ij ik kj
k∈S
ThelastformulaisknownastheChapman-Kolmogorovequation.
1.2 Homogeneous Markov chains
As we have mentioned, a Markov chain is a special case of a Markov process. We
willintroducethebasicpropertiesoftheMarkovchainswiththediscretestatespace.
ProofsofpresentedtheoremsomittedheremaybefoundinRefs.[3,9,22,47,88,90].
1.2.1 Basic definitions and properties
Now let us consider a discrete time homogeneous Markov process {X :n∈N },
n 0
havingafiniteorcountablestatespaceSthatiscalledahomogeneousMarkovchain
(HMC).Recallthatforeachmomentn∈Nandallstatesi,j,i0,...,in−1 ∈Sthereis
DiscretestatespaceMarkovprocesses 3
P(Xn+1 =j|Xn =i,Xn−1 =in−1,...,X1 =i1,X0 =i0)
=P(Xn+1 =j|Xn =i) (1.6)
whenever
P(Xn =i,Xn−1 =in−1,..., X1 =i1,X0 =i0)>0.
TransitionprobabilitiesoftheHMC
p (n,n+1)=P(X(n+1)=j|X(n)=i), i,j∈S, n∈N (1.7)
ij 0
areindependentofn∈N :
0
p (n,n+1)=p , i,j∈S, n∈N . (1.8)
ij ij 0
Thesquarenumbermatrix
(cid:3) (cid:4)
P= p : i,j∈S (1.9)
ij
is said to be a matrix of transition probabilities or transition matrix of the HMC
{X(n):n∈N }.Itiseasytonoticethat
0
(cid:5)
∀i,j∈S pij (cid:3)0 and∀ j∈S pij =1. (1.10)
j∈S
(cid:3) (cid:4)
The matrix P= p :i,j∈S having the above-mentioned properties is called a
ij
stochastic matrix. There exists the natural question: Do the stochastic matrix P and
discrete probability distribution p(0)=[p =i:i∈S] define completely the HMC?
i
Thefollowingtheorema(cid:3)nswersthisq(cid:4)uestion.
Theorem 1.1. Let P= pij :i,j∈S be a stochastic matri(cid:2)x and p=[pi :i∈S] be
the one-row matrix with nonnegative elements, such that p =1. There exists a
i
i∈S
probabilityspace((cid:3),F,P)anddefinedonthisspaceHMC{X(n(cid:3)):n∈N0}(cid:4)withthe
initialdistributionp=[p :i∈S]andthetransitionmatrixP= p :i,j∈S .
i ij
Proof:[9,47].
From(1.6)weobtain
P(X =i ,X =i ,...,X =i )=p p p ...p . (1.11)
0 0 1 1 n n i0 i0i1 i1i2 in−1in
Anumber
p (n)=P(X(n)=j|X(0)=i)=P(X(n+w)=j|X(w)=i) (1.12)
ij
denotesatransitionprobabilityfromstateitostatejthroughouttheperiod[0, n],(in
nsteps).Now,theChapman-Kolmogorovequationisgivenbyaformula
(cid:5)
p (m+r)= p (m)p (r), i,j∈S (1.13)
ij ik kj
k∈S
orinmatrixform
P(m+r)=P(m)P(r),
(cid:3) (cid:4)
whereP(n)= p (n):i,j∈S .
ij