Relative Frequencies of Generalized Simulated Annealing JanHannig DepartmentofStatistics,ColoradoStateUniversity,FortCollins,CO80523-1877. email:Jan.Hannig@ColoState.edu EdwinK.P.Chong DepartmentofElectricalandComputerEngineering,ColoradoStateUniversity,FortCollins,CO80523-1373. email:EChong@Engr.ColoState.edu SanjeevR.Kulkarni DepartmentofElectricalEngineering,PrincetonUniversity,Princeton,NJ08544. email:Kulkarni@EE.Princeton.edu. Weconsideraclassofnon-homogeneousMarkovchainsarisinginsimulatedannealingandrelatedstochasticsearchalgorithms. Usingonlyelementaryfirstprinciples,weanalyzetheconvergenceandrateofconvergenceoftherelativefrequenciesofvisits tostatesintheMarkovchain. Wedescribeindetailthreeexamples,includingthestandardsimulatedannealingalgorithm,to showhowourframeworkappliestospecificstochasticsearchalgorithms—theseexampleshavenotpreviouslybeenrecognized tobesufficientlysimilartosharecommonanalyticalgrounds.Ouranalysis,thoughelementary,providesthestrongestsample- pathconvergenceresultstodateforsimulatedannealingtypeMarkovchains. Ourresultsservetoillustratethatbytakinga purelysample-pathview,surprisinglystrongstatementscanbemadeusingonlyrelativelyelementarytools. Keywords:Simulatedannealing;randomizedoptimization;coupling;relativefrequency MSC2000SubjectClassification:Primary:65C05,Secondary:60J10 OR/MSsubjectclassification:Primary:Stochasticmethodsofoptimization,Secondary:Simulation 1. Introduction For at least the last 20 years, there has been an interest in stochastic search algorithms for globaloptimizationbasedonnon-homogeneousMarkovchains. Theprimeexampleissimulatedannealing,first suggested for optimization by Kirkpatrick et al. [12] based on techniques of Metropolis et al. [14]. An early application to image processing was described by Geman and Geman [9]. The basic procedure in simulated annealing is to explore the search space by setting up a graph over the space and jumping from point (vertex) to point in this graph according to a non-homogeneous Markov chain. The non-homogeneity arises from the graduallydecreasingprobabilityofjumpingfromonepointtoa“worse”pointinthecourseofthesearch(butsuch ajumpalsocannotbeprecluded,becauseoftheneedto“climbout”of“cups”aroundlocalminimizers).Thespeed atwhichthisdecreaseinthetransitionprobabilitiesoccursdependsonasequencecalledthe“coolingschedule” (describedinmoredetailinSection3). Inaseminalpaper,Hajek[11]providesadetailedtreatmentofthebehavioroftheMarkovchainassociatedwith the simulated annealing algorithm. Specifically, he provides a necessary and sufficient condition on the cooling schedule for convergence in probability of the algorithm to the set of global minimizers. Tsitsiklis [17] proves essentially the same result, but using different techniques. Around the same time, Connors and Kumar [3] also studysimulatedannealingtypeMarkovchains,providingyetadifferentviewofsuchprocesses. Inthelast15years,theliteratureontheanalysisofsimulatedannealinghasgrownsignificantly. Inparticular, therehavebeenseveralgeneralizationsofsimulatedannealing. Forexample, GelfandandMitter[8]andTsallis andStariolo[16]consideracontinuous-spaceversionofsimulatedannealing,andMoraiandMiclo[5],Cotand Catoni[4], andTrouve[15]consideranevenfurthergeneralizationoftheMarkovprocessinstandardsimulated annealing. Theanalysisofthesegeneralizationsofsimulatedannealinginvolverelativelysophisticatedtools. Inthispaper,westudyanon-homogeneousMarkovchainthatisalsoageneralizationofsimulatedannealing. Our generalization is different from those of the above papers—ours is much closer to the original simulating annealing framework of Hajek [11]. For convenience, in this paper we refer to our generalization simply as generalized simulated annealing (even though this same term is used also for other generalizations). The main reasonforintroducingourgeneralizationistofacilitatetheanalysisofrelativefrequenciesinnon-homogeneous Markovchainsarisinginsimulatedannealingandotherstochasticsearchalgorithms. Ourfocusonrelativefrequenciesinournon-homogeneousMarkovchainsharplydifferentiatesourstudyfrom previous studies in the literature. At the same time, our approach offers several advantages. First, we use only elementaryfirstprinciples—ourtoolsconsistessentiallyofapplicationsofKolmogorov’sthree-seriestheoremand 1 2 Hannig,Chong,Kulkarni:GeneralizedSimulatedAnnealing MathematicsofOperationsResearchxx(x),pp.xxx–xxx,(cid:13)c200xINFORMS coupling. Incontrast,thepaperofHajek[11],whichwasalsobasedonfirstprinciples,requiresrelativelycomplex arguments. Second,ourgeneralization,whilesimple,allowsstudyingratherdisparatesearchalgorithmswithina single unified framework. We illustrate this claim by considering two other search algorithms (besides standard simulated annealing)—these two other algorithms have not previously been recognized to be sufficiently akin to simulatedannealingtohaveacommonanalytical“ancestry.” Third,ourapproachprovideswhatwebelievetobe thestrongestsample-pathcharacterizationsofsimulatedannealingtypeMarkovchainstodate. Weestablishnot only the convergence to zero of the relative frequencies of all non-global-minimizers, but also the rate at which theserelativefrequenciesvanish. There is significant appeal in characterizing convergence and rates in purely sample-path terms. Our com- mitment to this program of study is evident in our previous work on sample-path analyses of various stochastic algorithms;seeKulkarniandHorn[13],Wangetal.[18],Wangetal.[19],WangandChong[20],andChonget al.[2]. Thetypicalconclusionwefindisthatalthoughthesepurelysample-pathanalysesinvolveonlyelementary tools,theresultsaresurprisinglystrong—theresultsinthispapercorroboratethisconclusion.Wecontrastthiswith theprobabilisticanalysisofHajek[11]: althoughhisanalysisprovidesthestrongestpossibleconditionforconver- gencebasedonfirstprinciples,ratesofconvergencedonotfallouteasily. Inouranalysisofrelativefrequencies, ontheotherhand,rateestimatesfollowrelativelyeasilyandnaturally. Fromfirstprinciplesitisextremelydifficult to obtain the kind of “sharp” estimates needed in Hajek’s probabilistic analysis to characterize rates in addition to convergence. Since Hajek’s paper, there have certainly been results on convergence rates of probabilities in simulatedannealinganditsgeneralizations; however, moresophisticatedmachinerythanHajek’sfirst-principles approach has to be brought to bear (e.g., see Catoni [1], who uses results from Freidlin and Wentzell [7]). This paper and our previous work along similar lines suggest that the same is not the case in a purely sample-path setting. The rest of this paper is organized as follows. We begin below with some notation and terminology we will need, and a brief discussion of relative frequencies. In Section 2 we define our generalized simulated annealing processandstateourmainresults. InSection3wediscussthreeexamplestoshowhowourgeneralizedsimulated annealing framework applies to specific stochastic search algorithms. For convenience and ease of presentation, theproofsofourresultsarerelegatedtoSection4. WeendwithsomefinalremarksinSection5 Somenotationandterminology Wefirstintroducesomenotationusedthroughoutthispaper. Fortwoposi- tivesequences{a }and{b },wewrite: n n • a ∼b ifa /b →1; n n n n • a =O b iflimsupa /b <∞,andlimsupb /a <∞;and n n n n n n • a ≈O b if(loga −logb )/logn→0. n n n n The differencebetween a =O b and a ≈O b is thatwhile “=O” impliesthat the twosequences are ofthe same n n n n “order,”theweaker“≈O”allowstheirordertodifferbyaslowlyvaryingfunction,e.g.,apoweroflogn. Givenasequencex={x }={x ,x ,...}andasetA,wedefinethenotation n 1 2 (cid:26) 1 ifx ∈A I(x ∈A)= i i 0 otherwise. The notation I(x ∈ A) represents an “indicator” of the condition x ∈ A. We define the relative frequency of i i visitstoAuptotimenas n 1 X F (x∈A)= I(x ∈A). n n i i=1 IfAisthesingleton{v},wewriteF (x=v). Similarly,weusethenotationF (x6=v)=F (x6∈{v}). n n n Ifx ∈ Aforaninfinitenumberofi,thenwesaythatAisvisitedinfinitelyoften. Otherwise,wesaythatAis i visitedfinitelyoften. Whenconsideringrandomsequences,weusecapitalletters: X ={X ,X ,...},F (X =x∗),etc. 1 2 n Relativefrequenciesofrandomsequences Ourresultsarestatedintermsofconvergence(a.s.)ofrelative frequencies.Ingeneral,a(discretestate-space)randomsequenceX ={X ,X ,...}thatconvergesinprobability 1 2 to x∗ may or may not also have convergent relative frequencies of the form F (X = x∗). If the sequence is n Hannig,Chong,Kulkarni:GeneralizedSimulatedAnnealing 3 MathematicsofOperationsResearchxx(x),pp.xxx–xxx,(cid:13)c200xINFORMS independent,thenconvergenceinprobabilityisstrongerthanitsrelativefrequencycounterpart,asillustratedinthe simplelemmabelow. Theproofofthislemmaisofinterestbecauseitemploysatechniquewewilluserepeatedly intheproofofourmainresults(weelaborateonthisbelow). LEMMA1.1 LetX ={X1,X2,...}beanindependent,discretestate-space,randomsequencethatconvergesto x∗inprobability. Then,F (X =x∗)→1a.s. n PROOF. SupposethatX convergestox∗ inprobability;i.e.,P(Xn 6= x∗) → 0. Fixε > 0. Then,P(Xn 6= x∗) ≤ ε eventually (for sufficiently large n). Let {U } be an i.i.d. sequence with uniform distribution on [0,1], n independentofX. DefinetheBernoullisequenceB ={B }by n (cid:18) ε−P(X 6=x∗)(cid:19) B =I(X 6=x∗)+I(X =x∗)I U ≤ n n n n n P(X =x∗) n (ifP(X = x∗) = 0, thentakethesecondtermtobe0; thisissuedisappearswhennissufficientlylarge). Itis n clearthat{B }isanindependentsequence,andP(B = 1) = εforsufficientlylargen. Moreover,ifX 6= x∗ n n n then B = 1 (a.s.), which implies that F (X 6= x∗) ≤ F (B = 1). By the strong law of large numbers, n n n F (B = 1) → εa.s. Therefore,limsup F (X 6= x∗) ≤ εa.s. Becausethisargumentholdsforallε > 0, n n→∞ n weconcludethatF (X 6=x∗)→0a.s.,asrequired. (cid:3) n Weprovidetheproofabovenotbecausewebelievetheresulttobeoriginal,buttoillustrateamethodofproof thatwe willuse repeatedly inprovingour mainresults: coupling. This methodinvolvesconsidering asequence that is related to another sequence such that some property involving sample paths of both sequences holds a.s. In the above proof, we explicitly constructed the sequence B from X—we say that the sequence B is coupled withX. ThepropertiesofB ofinteresttousherearethatP(B = 1) = εandifX 6= x∗ thenB = 1(a.s.). n n n ThiscouplingpropertyofB allowsustoboundtherelativefrequencyF (X 6= x∗)byF (B = 1), aquantity n n thatismucheasiertocharacterize. Weusesuchcouplingargumentsrepeatedlyinprovingourmainresults(see Section4). Weshouldpointoutthatintheindependentcase,convergenceinprobabilityisstrictlystrongerthanitsrelative frequency counterpart, because there are instances where F (X = x∗) → 1 a.s. but the sequence does not n converge to x∗ in probability. To see this, consider the sequence X = {X ,X ,...} on the state-space {0,1}, 1 2 whereX = 1a.s.forallnexceptforthosenoftheformn = 2k,k = 1,2,...,inwhichcaseX = 0a.s. Itis n n clearthatP(X =1)doesnotconvergeto1,butF (X =1)→1a.s. n n In our generalized simulated annealing framework, the sequences are non-homogeneous Markov chains. In these cases, it is not clear a priori whether convergence in probability is weaker or stronger than its relative frequencycounterpart. Itwillturnoutthatinfacttheyareequivalent. 2. GeneralizedSimulatedAnnealing Inthissectionwepresentourgeneralizedsimulatedannealingframe- workandourmainresults. TheproofsoftheseresultswillbeprovidedinSection4,afterwegivethreeexample applicationsofourframeworkinSection3. Consider a finite, directed, connected graph G = (V,E), where V is a set of vertices and E a set of directed edges. Assumethateachvertexv ∈ V isassignedavaluef(v). Ourgoalistofindtheminimumofthefunction f;i.e.,wewishtofindv ∈V suchthatf(v )≤f(v)forallv ∈V. min min Weassumethatallvaluesoff(v),v ∈V,aredistinct.Wemakethisassumptiontosimplifythepresentation.In particular,underthisassumption,v = argmin f(v)isunique. However,allourresultsremainvalidwith min v∈V appropriateadjustmentsifweremovethisassumption. Weelaborateonthisinourfinalremarks(Section5). Nowdefineanon-homogeneousMarkovprocess{X }onthegraphG,asfollows. Associatewitheachedge n uv ∈E,u6=v,twovaluesg (u,v)≥0andg (u,v)>0. Thetransitionprobabilitiesoftheprocess{X }satisfy, r c n foru6=v, ( P(X =v|X =u) ∼gc(u,v)n−gr(u,v) ifuv ∈E n n−1 =0 otherwise, and,asusual, X P(X =u|X =u)=1− P(X =v|X =u). n n−1 n n−1 v6=u 4 Hannig,Chong,Kulkarni:GeneralizedSimulatedAnnealing MathematicsofOperationsResearchxx(x),pp.xxx–xxx,(cid:13)c200xINFORMS Thus,theasymptoticbehaviorofthetransitionprobabilitiesisdeterminedbythevaluesofg (u,v)andg (u,v). r c Wewillcall{X }ageneralizedsimulatedannealingprocess. AswewillseeinSection3,generalizedsimulated n annealing reduces not only to the familiar simulated annealing process, but also processes associated with other stochasticsearchalgorithms. For convenience, define for each vertex u ∈ V two neighborhoods: N (u) = {v 6= u : uv ∈ E} and out N (u) = {v 6= u : vu ∈ E}. Withthisnotation,weseethatbecauseprobabilitiesmustbeboundedaboveby1, in forallu, X g (u,v)≤1. c v∈Nout(u):gr(u,v)=0 Wenowdescribeanassumptionthatlinksthefunctionf(v)withthetransitionprobabilitiesof{X }.Asusual, n we say that p = {u ,u ,u ,...,u ,u } is a path from u to v if u = u, u = v, and u ∈ N (u ), 1 2 3 k−1 k 1 k i+1 out i i=1,...,k−1. Forapathp={u,u ,u ,...,u ,v}wedefineitsheightby 2 3 k−1 h(p)=max{f(u)+g (u,u ),f(u )+g (u ,u ),...,f(u )+g (u ,v)}. r 2 2 r 2 3 k−1 r k−1 (Thisdefinitionismotivatedbythenotionof“height”inHajek[11]forsimulatedannealing.) Foranytwovertices uandv,wethendefine h(u,v)=min{h(p): pisapathfromutov}. (1) Next,weintroducetwodefinitionsinvolvingthenotionofheights:weakreversibilityandheightnormalization. These are needed in the statements of our main results. Our notion of weak reversibility reduces to that of Ha- jek’s[11]similarlynamedpropertyforthespecialcaseofsimulatedannealing. Heightnormalizationplaysakey roleinconvergence. DEFINITION2.1 We say that the generalized simulated annealing process is weakly reversible if, for any two verticesuandv,h(u,v)=h(v,u). DEFINITION2.2 Wesaythatthegeneralizedsimulatedannealingprocessisheight-normalizedif,foranyvertex v 6=v ,h(v,v )−f(v)≤1. min min Notethatheightnormalizationimpliesthatthegraphisconnected. Wearereadytogiveourmainconvergence result, which essentially states that height normalization is necessary and sufficient for global convergence of the process (i.e., a.s. convergence to a global minimizer regardless of initial condition) in the sense of relative frequencies. THEOREM2.1 ConsideraweaklyreversiblegeneralizedsimulatedannealingprocessX ={X1,X2,...}. Ifthe processisheight-normalized,thenF (X =v )→1a.s.regardlessofthestartingpoint. n min On the other hand, suppose that the process is not height-normalized. Then, there is a vertex v 6= v min such that h(v,v ) − f(v) > 1, and if X = v, then P (F (X =v)→1) > 0 (which implies that min 1 n P (F (X =v )→1)<1). n min Ournextresultcharacterizestherateofconvergenceintermsofrelativefrequencies. THEOREM2.2 Consider a weakly reversible, height-normalized generalized simulated annealing process X = {X ,X ,...}. Supposevisavertexsuchthat 1 2 h(v ,v)−f(v )<1. (2) min min Then,Fn(X =v)=O n−(f(v)−f(vmin))a.s.regardlessofthestartingpoint. Otherwise,if(2)isnotsatisfiedbut h(v ,v)−f(v )=1, (3) min min thenFn(X =v)≈O n−(f(v)−f(vmin))a.s.regardlessofthestartingpoint. Finally,ifforsomevneither(2)nor(3)issatisfied,thenvisvisitedfinitelyoftena.s.regardlessofthestarting point,inwhichcaseeitherF (X =v)=0orF (X =v)=O n−1a.s. n n Hannig,Chong,Kulkarni:GeneralizedSimulatedAnnealing 5 MathematicsofOperationsResearchxx(x),pp.xxx–xxx,(cid:13)c200xINFORMS REMARK2.1 Suppose in the definition of the transition probabilities we replace gc(u,v)n−gr(u,v) by g (u,v)a−gr(u,v). If a /n → 0, then convergence to the global minimizer holds, but at a slower rate. On the c n n otherhand,ifa /n → ∞,thenthesameconclusionastheconversetoTheorem2.1holds—theprocessmaynot n convergetotheglobalminimizer. Recall that “=O” is stronger than “≈O.” Thus, for simplicity, we can summarize the essence of Theorem 2.2 as follows: If v is visited infinitely often a.s., then Fn(X = v) ≈O n−(f(v)−f(vmin)) a.s. regardless of the starting point. In Section 3, we will use this simplified version of Theorem 2.2 in applying our framework to specific examples. Theonlythingmissingfromthissimplifiedstatementofthetheoremisthatincertaincaseswehavethe strongerresultwith“=O”insteadof“≈O.” TheproofsofTheorems2.1and2.2arerelegatedtoSection4.Inthenextsection,wedescribethreeexamplesof stochasticsearchprocessesforwhichwecanapplyourresultstocharacterizeconvergenceandratesofconvergence ofrelativefrequencies. 3. Applications Inthissectionweshowthatgeneralizedsimulatedannealingprovidesaunifyingframework tostudyvariousstochasticoptimizationalgorithms. Inparticular,weshowthattheclassicalsimulatedannealing algorithm,the“stochasticruler”algorithmofYanandMukai[21],andthe“stochasticcomparison”algorithmof Gongetal.[10]areallspecialcasesofgeneralizedsimulatedannealing. Indoingso,ourconvergenceresultscan be brought to bear in the analysis of these algorithms. We show that our analysis in fact yields stronger results thanareavailableforthesealgorithms. Forthecaseofsimulatedannealing, anecessaryandsufficientcondition for convergence in probability is already available in Hajek [11], though as far as we know, rates on the relative frequencieshavenotbeenpreviouslyobtained. Astochasticoptimizationalgorithmaimstominimizeafunctionl(v)definedonadiscretesetV viaastochastic search process. The search process gives rise to a non-homogeneous Markov chain of the kind that we will show fits within our framework. When showing that a particular stochastic optimization algorithm is a special case of generalized simulated annealing, we first relate the functions g (u,v) and g (u,v) with the transition r c probabilities of the stochastic algorithm. We then show how to define a function f(v) that makes the process weaklyreversible. Itistemptingatfirstglancetotreattheobjectivefunctionl(v)asthefunctionf(v). However, ingeneral,wecannotusel(v)directlyinplaceoff(v)becausethefunctionf(v)containsinformationaboutthe rateofconvergence,whilel(v)mightnot. Somemodificationtol(v)isneededtolinkittotheunderlyinggraph. Theneededmodificationtol(v)toobtainf(v)shouldbecomeapparentinourdiscussionofthethreeexamplesin thissection. 3.1 SimulatedAnnealing Considertheproblemofminimizingl(v)withv ∈V. Insimulatedannealing,we beginwithagraphG =(V,E)anddefineanon-homogeneousMarkovchain{X }withtransitionprobability n P(X =v|X =u)=R(u,v)exp(−[l(v)−l(u)] /T ), n n−1 + n where[x] =max{x,0},andR(u,v)isatransitionprobabilitysuchthat + ( >0 ifv ∈N (u), R(u,v) out =0 otherwise, P and R(u,v) = 1. The sequence {T } is a positive sequence called the cooling schedule. We focus v∈Nout(u) n our attention on cooling schedules of the form T = d/logn, popularized by Geman and Geman [9]. In [11], n Hajek shows that {X } converges in probability to the global minimizer if and only if d ≥ d∗, where d∗ is a n quantityHajekcallsthe“depthoftheseconddeepestcup,”aparameterwedefinepreciselybelow. Ourgoalhere is to show that, based on our main results, the same condition as Hajek’s (involving d∗ above) is also necessary andsufficientforconvergenceintherelative-frequencysense. Moreover,weprovideacharacterizationoftherate ofconvergenceoftherelativefrequencies. Tobegin,consideracoolingschedulesatisfyingT ∼d/logn,withd>0fixed.Then,thesimulatedannealing n 6 Hannig,Chong,Kulkarni:GeneralizedSimulatedAnnealing MathematicsofOperationsResearchxx(x),pp.xxx–xxx,(cid:13)c200xINFORMS algorithmaboveisreadilyseentobeaninstanceofgeneralizedsimulatedannealingwith l(v) f(v)= , d [l(v)−l(u)] g (u,v)= +, r d g (u,v)=R(u,v). c Itremainstoseewhentheheight-normalizationconditionholds. Tothisend,denotethesetofallpathsfromu tovbyP(u,v). Thendefine d∗ = max min max{l(u)−l(v)}. v6=vminp∈P(v,vmin) u∈p Here,u∈pmeansthatthevertexuispartofthepathp. To understand the connection between d∗ and height normalization, first observe that by the definitions of f, g ,andg givenabove,thequantityh(v,v )(definedby(1)inSection2)simplifiesto r c min 1 h(v,v )= min maxl(u). min dp∈P(v,vmin) u∈p Fromthis,itiseasytoseethatd∗canberewrittenas (cid:18) (cid:19) d∗ =d max {h(v,v )−f(v)} . min v6=vmin Weconcludethattheprocessisheight-normalizedifandonlyifd≥d∗. Combining the above with Theorems 2.1 and 2.2 gives the following convergence theorem for simulated annealing. Note that weak reversibility in simulated annealing (in the Hajek sense) implies that the corresponding generalizedsimulatedannealingprocessisweaklyreversiblealso. THEOREM3.1 For a weakly reversible simulated annealing process with cooling schedule Tn ∼ d/logn, F (X = v ) → 1 a.s. regardless of the starting point if and only if d ≥ d∗. Moreover, assuming d ≥ d∗, n min ifv 6=vminisvisitedinfinitelyoften,thenFn(X =v)≈O n−(l(v)−l(vmin))/da.s.regardlessofthestartingpoint. ThefirstpartofTheorem3.1exactlyparallelsthatofHajek’s(thenecessaryandsufficientconditionforconver- genceisidenticaltothatofHajek[11]).Thisshowsthatconvergenceinprobability(Hajek’sresult)isequivalentto a.s.convergenceoftherelativefrequency. ThesecondpartofTheorem3.1characterizestherateofconvergencein termsofrelativefrequencies. Asnotedbefore,wecansharpentherateresulttoFn(X =v)=O n−(l(v)−l(vmin))/d for those v such that h(v ,v) − f(v ) < 1. As far as we know, these results on relative frequencies for min min simulatedannealinghavenotpreviouslybeenavailable. TheconvergenceresultinHajek[11]goesbeyondthecasewhereT ∼ d/logn. Inparticular,healsoshows n that if T logn → 0 then simulated annealing might converge to a “local” rather than global minimizer, and n if T logn → ∞ then the algorithm converges to the global minimizer. Our framework addresses the case of n T logn→0asonecanshowthatthealgorithmmightconvergetoalocalminimizer,usingacouplingargument n involvingageneralizedsimulatedannealingprocessthatdoesnotsatisfytheconditionsofTheorem2.1. Onthe otherhand,wedonotdirectlyrecoverthecaseofT logn→∞. However,inthiscaseacouplingargumentwith n ageneralizedsimulatedannealingprocessshowsthattherateofconvergenceisslowerthenanypower;i.e.,forall v,wehaveF (X =v)≈O 1,suggestingthatacoolingscheduleforwhichT logn→∞shouldnotbeused. n n 3.2 StochasticRulerAlgorithm YanandMukai[21]considertheproblemofminimizinganobjectivefunc- tionl(v),v ∈V,thatisassumedtobeoftheforml(v)=EH(v),whereH(v)israndomwithfinitevariance.They assumewedonotactuallyhaveaccesstol(v);instead,wecanonlyobserveindependentsamples(realizations)of H(v). Theyconverttheproblemtooneofmaximizing p(v,a,b)=P(H(v)≤Θ(a,b)), whereΘ(a,b)isarandomvariableuniformlydistributedon(a,b)(andindependentofH(v)). Theyprovethatfor asmallenoughandblargeenough,anyuthatmaximizesp(u,a,b)alsominimizesl(v). (Weassumehenceforth thataandbarechosensuchthatthisconclusionholds.) Hannig,Chong,Kulkarni:GeneralizedSimulatedAnnealing 7 MathematicsofOperationsResearchxx(x),pp.xxx–xxx,(cid:13)c200xINFORMS Tofindthemaximizerofp(u,a,b)theysetupanon-homogeneousMarkovchainX satisfying n P(X =v|X =u)=R(u,v)(p(v,a,b))Mn, n n−1 where M → ∞ is called the “testing sequence.” (It is useful to think of the testing sequence as the reciprocal n of a cooling schedule.) As in simulated annealing, the probabilities R(u,v) satisfy R(u,v) > 0 if and only if v ∈N (u).YanandMukai[21]imposetheadditionalrestrictionthatthegraphhas“symmetricneighborhoods,” out i.e.,v ∈N (u)ifandonlyifu∈N (v). Weadoptthisassumptionintheremainderofthissection. out out YanandMukai[21]considerthespecifictestingsequence M =blog(n+n +1)/dc, n 0 where bxc is the integer part of x, and n and d > 0 are fixed constants. They show how to implement the 0 searchalgorithmusingonlysamplesofH: supposethatatthenthiterationtheprocessisinstateu,andarandom candidatenext-statevisgeneratedaccordingtoR(u,v).Then,generateblog(n+n +1)/dcindependentsamples 0 (realizations) of H(v) and Θ(a,b), and transition to v if and only if H(v) ≤ Θ(a,b) for all the samples. It is convenienttocalltheabovealgorithmthestochasticruleralgorithm,becausethesamplesofH(v)arecompared toa“stochasticruler”Θ(a,b). The main convergence result in Yan and Mukai [21] is that with the above testing sequence, provided some technical assumptions hold (which we elaborate below), {X } converges in probability to the global minimizer. n Below,weshowthatthestochasticruleralgorithmfallswithintheframeworkofgeneralizedsimulatedannealing, andhenceourrelative-frequencyconvergenceresultsapply,includingacharacterizationoftheconvergencerates oftherelativefrequencies. Moreover,aswewillseebelow,thetechnicalassumptionsinYanandMukai[21]can beweakened—weprovideanecessaryandsufficientconditionforconvergence. Inouranalysis,weconsidertheslightlymoregeneralcasewherethetestingsequence{M }satisfies n logn M ∼ . n d Inthiscase,weseethatforv 6=u, P(X =v|X =u)=R(u,v)(p(v,a,b))Mn ∼R(u,v)n(logp(v,a,b))/d. n n−1 Thetransitionprobabilitiesofthisnon-homogeneousMarkovchainsuggestthefollowingspecializationofgener- alizedsimulatedannealing: −logp(v,a,b) f(v)= , d −logp(v,a,b) g (u,v)= , r d g (u,v)=R(u,v). c Notice that f(v) ≥ 0, and maximizing p(v,a,b) is equivalent to minimizing f(v). Moreover, the symmetric- neighborhood assumption is sufficient for the above particular choice of f(v) to guarantee weak reversibility of theresultingprocess. Asbefore,denotethesetofallpathsfromutovbyP(u,v). Thendefine d∗ = max min max{logp(v,a,b)−logp(u,a,b)−logp(u0,a,b)}. (4) v6=vminp∈P(v,vmin)uu0∈p Here,uu0 ∈ pmeansthatthelinkuu0 ispartofthepathp. Thisvalueofd∗ isanalogoustoHajek’snotionofthe “depthoftheseconddeepestcup”forsimulatedannealingHajek[11]. Itwillturnoutthatd∗ characterizesanec- essaryandsufficientconditionforconvergenceofthestochasticruleralgorithm(seebelow). Therefore,although YanandMukai[21]arecarefultopointoutthattheirapproachis“differentfromthetechniqueofsimulatedanneal- ing,” theanalysisofsimulatedannealingactuallybearsontheanalysisofthestochasticruleralgorithm, through ourgeneralizedsimulatedannealingframework. Toseewhyd∗ playsthesamerolehereasinsimulatedannealing, notethatbytheabovedefinitionsoff and g ,wecanonceagainwrited∗intheform r (cid:18) (cid:19) d∗ =d max {h(v,v )−f(v)} . min v6=vmin andhenceconcludethattheprocessisheight-normalizedifandonlyifd≥d∗. ApplyingTheorems2.1and2.2tothestochasticruleralgorithm,weobtainthefollowingconvergenceresult. 8 Hannig,Chong,Kulkarni:GeneralizedSimulatedAnnealing MathematicsofOperationsResearchxx(x),pp.xxx–xxx,(cid:13)c200xINFORMS THEOREM3.2 ForthestochasticruleralgorithmwithtestingsequenceMn ∼(logn)/dappliedtoasymmetric- neighborhoodgraph, F (X = v ) → 1a.s.regardlessofthestartingpointifandonlyifd ≥ d∗. Moreover, n min assumingd ≥ d∗, ifv 6= vmin isvisitedinfinitelyoften, thenFn(X = v) ≈O n−(logp(v,a,b)−logp(vmin,a,b))/d a.s. regardlessofthestartingpoint. WeendwithabriefdiscussionofsometechnicalassumptionsusedintheanalysisofYanandMukai[21].First, somenotation. Letµ(a,b)=min p(s,a,b),andletrbethe“radius”ofthegraph,i.e.,r =min max d(s,s0), s s s0 whered(s,s0)isthenumberofedgesintheshortestpathfromstos0. YanandMukaiuseaparticularchoiceof dintheirconvergenceanalysis: d = (logσ)/c,wherec ≤ 1/r andσ ≥ 1/µ(a,b). Itisstraightforwardtoshow that this choice of d in fact satisfies the condition d ≥ d∗ of Theorem 3.2. Hence, our analysis shows that the assumptionsofYanandMukaicanbeweakened. We note one more side benefit of our approach, besides improving the conditions under which the algorithm provablyconvergesandgivingustherateofconvergenceoftherelativefrequencies:ourapproachgivesusforfree adirectgeneralizationofthisalgorithmtonon-symmetricneighborhoodgraphsthatstillimplyaweaklyreversible generalizedsimulatedannealingprocess. 3.3 StochasticComparisonAlgorithm Gongetal.[10]considerasetupthatissimilartothatofYanand Mukai[21],exceptthattheirMarkovchain{X }satisfies,forv 6=u, n P(X =v|X =u)=R(u,v)(P(H(v)−logµ≥−logF(l(v )−l(v ))=d∗, c (2) (1) whichshowsthatthechoiceofdinGongetal.[10]satisfiestheconditiond≥d∗ofTheorem3.3. It is interesting to note that even though the original graph of Gong et al. is complete, the resulting general- izedsimulatedannealingprocessisonlyweaklyreversible. Moreover,theminimal-heightpathbetweenanytwo verticesalwaysgoesthroughtheglobalminimizer. Thus,wecangeneralizeourresulttographsthatarenotcom- pletebutwhereeverynodeisaneighboroftheglobalminimizer. Inthiscase,asinthecomplete-graphcase,the globalminimizerofl isstilltheglobalminimizeroff. However,thefunctionf mightnolongerbeamonotone transformationofl,incontrasttothecaseofacompletegraph. Thishasimplicationsonrates. Specifically,ifthe transformationfromltof ismonotone,thena(non-globalminimizing)pointvthathassmallerl(v)valuealsohas aslowerrateofdecayofitsrelativefrequency. However, ifthetransformationfroml tof isnotmonotone, this natural monotone relationship between l(v) and therate ofdecay of therelative frequencyof v no longerholds. Thisshowsthatgeneralizationofthisalgorithmtographsthatarenotcompletemightbenon-trivial. We also remark that we did not need any assumptions on the distribution function F, rendering the assump- tion3.1ofGongetal.[10]unnecessary. 10 Hannig,Chong,Kulkarni:GeneralizedSimulatedAnnealing MathematicsofOperationsResearchxx(x),pp.xxx–xxx,(cid:13)c200xINFORMS 4. Proofs Thissectionisdevotedentirelytotheproofsofourmainresults,Theorems2.1and2.2.Tofacilitate ourpresentation,wefirststateandproveseveraltechnicallemmas. LEMMA4.1 Let0 < ∆ ≤ 1andletA1,A2,... beindependenteventssuchthatP(An) = pn ∼ dn−∆. Then, a.s., 1 Xn ( d n−∆ if0<∆<1; I ∼ 1−∆ (5) n Ai d(logn)n−1 if∆=1. i=1 PROOF. Set n n c = 1 Xp , Y =I −p , and S =X Yi. n n i i Ai i n ic i i=1 i=1 Wefirstconsiderthecasewhere0<∆<1. Definep0 =dn−∆. Bytheintegralapproximation, n n 1 X d c0 = p0 ∼ n−∆. n n i 1−∆ i=1 Ingeneral,ifp ∼dn−∆thenforall0<(cid:15)<1smallenough, n (1−ε)p0

Relative Frequencies of Generalized Simulated Annealing

Similar Relative Frequencies of Generalized Simulated Annealing