DTIC AD1006926: Convergence Results for Ant Routing Algorithms via Stochastic Approximation and Optimization

0.77 MB·English

by  Defense Technical Information Center

The InsTITuTe for sysTems research
Isr TechnIcal rePorT 2007-23
convergence results for ant routing
algorithms via stochastic
approximation and optimization
Punyaslok Purkayastha, John s. Baras
ISR develops, applies and teaches advanced methodologies of design and
analysis to solve complex, hierarchical, heterogeneous and dynamic prob-
lems of engineering technology and systems for industry and government.
ISR is a permanent institute of the University of Maryland, within the
A. James Clark School of Engineering. It is a graduated National Science
Foundation Engineering Research Center.
www.isr.umd.edu
Convergence Results for Ant Routing Algorithms
via Stochastic Approximation and Optimization
Punyaslok Purkayastha and John S. Baras
Abstract—“Ant algorithms” have been proposed to solve a nication networks. It has been observed in an experiment
variety of problems arising in optimization and distributed conductedby biologists called the double bridgeexperiment
control. They form a subset of the larger class of “Swarm
[7], that a colony of ants, when presented with two paths
Intelligence” algorithms. The central idea is that a ”swarm”
to a source of food, is able to collectively converge to the
of relatively simple agents can interact through simple mecha-
nisms and collectively solve complex problems. Instances that shorter path. Every ant lays a trail of a chemical substance
exemplifytheaboveideaaboundinnature.Theabilitiesofant called pheromoneas it walks along a path; subsequent ants
colonies to collectively accomplish complex tasks have served follow and reinforce this trail. This leads progressively to a
as sources of inspiration for the design of “Ant algorithms”.
largeaccumulationofpheromoneon the shorterpath,which
Examplesof “Antalgorithms”aretheset of“AntRouting”al-
is how ants discover the shorter path. Most of the Ant-
gorithmsthathavebeenproposedforcommunicationnetworks.
We analyze in this paper Ant Routing Algorithms for packet- Based Routing Algorithms (called Ant Routing Algorithms,
switched wireline networks. The algorithm retains most of the for short) proposed in the literature are inspired by this
salient and attractive features of Ant RoutingAlgorithms. The basic idea. These algorithms employ probe packets called
scheme is a multiplepath probabilisticrouting scheme, that is
ant packets (analogues of ants) to explore the network
fully adaptive and distributed. Using methods from adaptive
and measure various quantities related to network routing
algorithms and stochastic approximation, we show that the
evolution of the link delay estimates can be closely tracked by performance like link and path delays. These measurements
adeterministicODE system. A studyof theequilibriumpoints are used to construct and update the routing tables at the
of the ODE gives us the equilibrium behavior of the routing networknodes.Theupdatealgorithmstendtoreinforcethose
algorithm, in particular, the equilibrium routing probabilities,
outgoing links which lead to paths with lower delays.
and mean delaysin thelinksunderequilibrium.We also show
Schoonderwoerd et. al. [11], [5] tested an Ant Routing
that the fixed-point equations that the equilibrium routing
probabilities satisfy are actually the necessary and sufficient Algorithm on the British Telecom telephone network, and
conditionsofanappropriateoptimizationproblem.Simulations reportedsuperiorperformancecomparedto otheralgorithms
supporting the analytical results are provided. including shortest paths based schemes. This generated in-
terest in the study of Ant Routing Algorithms for both
I. INTRODUCTION
connection-oriented networks and for packet-switched net-
“Antalgorithms”constituteaclassofalgorithmsthathave
works (both wired and wireless). Ant Routing Algorithms
been proposed to solve a variety of problems arising in
for wireless networks have been proposed and analyzed in
optimization and distributed control. They form a subset
papers by Baras and Mehta [1] and Gunes et. al. [9].
of the larger class of what are referred to as “Swarm
Weconsiderinthispaperpacket-switchedwirednetworks.
Intelligence”algorithms,atopicextensivelydiscussedin the
Ant Routing Algorithms for such networks have been pro-
book by Bonabeau, Theraulez, and Dorigo [5]. The central
posedin theworksofDiCaro andDorigo[8] andBean and
idea here is that a “swarm” of relatively simple agents can
Costa [2]. Though a large number of ant routing algorithms
interact through simple mechanisms and collectively solve
havebeenproposed,veryfewanalyticalstudiesareavailable
complex problems. Instances that exemplify the above idea
in the literature. Examples of such analytical studies can be
abound in nature. Bonabeau, Theraulez, and Dorigo [5]
foundin[12],[10],and[2].Yoo,La,andMakowski[12]and
give examples of insect societies like those of ants, honey
Gutjahr [10] consider very simple network cases where the
bees, and wasps, which accomplish fairly complex tasks of
delaysinthelinksofthenetworkaredeterministicquantities,
building intricate nests, finding food, responding to external
independent of the offered traffic loads. They show that
threats etc., even though the individual insects themselves
ant algorithms converge to the shortest path solutions in
have limited capabilities. The abilities of ant colonies to
the network. Bean and Costa [2] propose a multiple-path
collectivelyaccomplishcomplextaskshaveservedassources
routing scheme which tries to form estimates of the delays
of inspiration for the design of “Ant algorithms”.
and use them to update routing tables. The link delays
Examples of “Ant algorithms” are the set of “Ant-Based
can be stochastically varying. The authors employ a time-
Routing” algorithms that have been proposed for commu-
scale separation approximation whereby the delay estimates
are computed before the routing probabilities are updated.
Research supported by the U.S. Army Research Laboratory under the
CTA C&N Consortium Coop. Agreement DAAD19-01-2-0011 and by the They find that results from a numericalstudy based on their
U.S.ArmyResearchOfficeunderMURI01GrantNo.DAAD19-01-1-0465 model and those from simulations agree well. However, the
P.PurkayasthaandJ.S.BarasarewiththeInstituteforSystemsResearch
exact nature of the time-scale separation is not clear nor
andtheDept.ofElectr.&Comp.Engin.,University ofMaryland, College
Park,MD20742,USA.((punya, baras)@isr.umd.edu) is any convergence result provided. Borkar and Kumar [6]
use stochastic approximation methods to prove convergence of routing an incoming packet at node i and bound for
of their scheme called Wardrop routing. Their framework is destination k via the neighbor j ∈ N(i,k). The matrix
similar to ours - they have a delay estimation scheme and a L(i)hasthesamedimensionsasR(i),andits(i,j)-thentry
probabilityupdateschemewhichutilizesthedelayestimates. containsvariousstatisticspertainingtotheroute(i,j,...,k).
Their probability update scheme moves on a slower time- Examples of such statistics could be mean delay and delay
scale than their delay estimation scheme. By two time-scale variance estimates of the route (i,j,...,k). These statistics
stochasticapproximationmethodstheystudyconvergenceof aremaintainedandupdatedbasedontheinformationtheant
their algorithm. packets collect about the route. The matrix L(i) represents
We consider the algorithm proposed by Bean and Costa the characteristics of the network that are learned by the
[2] inthispaper.Thealgorithmretainsmostoftheattractive nodesthroughtheantpackets,basedonwhichlocaldecision-
features of Ant Routing Algorithms. The scheme is fully making,andtheupdatingoftheroutingtableR(i),aredone.
adaptiveanddistributed.Weconsiderinthispaperthesimple The iterative algorithms that are used to update L(i) and
routing scenario where data traffic entering a single source R(i) will be referred to as the learning algorithms.
nodehasto be routedto a single destinationnode,andthere We now describe the mechanism of operation of ant-
are N available parallel paths between them. We model the based routing algorithms. For ease of exposition, we restrict
arrival processes and packet lengths of both the ant and the attention to a particular fixed destination node, and consider
data streams that arrive at the source node, and argue, using the problem of routing from every other node to this node,
methods from adaptive algorithms and stochastic approxi- which we label as D.
mation, that the evolution of the link delay estimates can Forward ant generation and routing. At certain inter-
be closely tracked by a deterministic ODE system, when vals, forward ant (FA) packets are launched from a node i
the step size of the estimation scheme is small. Then a towardsthedestinationnodeDtodiscoverlowdelaypathsto
study of the equilibrium points of the ODE gives us the it. The FA packets sample walks on the graph representing
equilibrium behavior of the routing algorithm; in particular, the communication network, based on the current routing
theequilibriumroutingprobabilities,andmeandelaysin the probabilitiesatthenodes.FA packetssharethesamequeues
N paths under equilibrium can be obtained. We also show as data packets and so experience similar delay characteris-
that the fixed-point equations that the equilibrium routing tics as data packets. Every FA packet maintains a stack of
probabilities satisfy are actually the necessary and sufficient datastructurescontainingtheIDsofnodesinitspathandthe
conditions of a convex minimization problem. per hop delays (or other relevant information) encountered.
Our paper is organizedas follows. Section II providesthe The per hop delay measurements are obtained through time
generalframeworkofantroutingandtheroutingschemewe stampingofthepacketsastheypassfromthevariousnodes.
consider.SectionIIIprovidesadetaileddescriptionofourN Backward ant generation and routing. Upon arrival of
parallel paths model. Section IV contains an analysis of the anFAatthedestinationnodeD,abackwardant(BA)packet
routing scheme. Section V provides illustrative simulation isgenerated.TheFApackettransfersitsstacktotheBA.The
results, and in Section VI we provide a few conclusive BApacketthenretracesbacktothesourcethepathtraversed
remarks. by the FA packet. BA packets travel back in high priority
queues,soastominimizethepossibilityofoutdatedorstale
II. GENERAL FRAMEWORK OFANT ROUTING
measurements. At each node that the BA packet traverses
ALGORITHMS AND OURSCHEME
through,ittransferstheinformationthatwasgatheredbythe
We provide,in thissection,a briefdescriptionof thegen- correspondingFA packet.Thisinformationisusedtoupdate
eral framework of ant routing for a (wired) communication thematricesLandRattherespectivenodes.Thusthearrival
network.Theframeworkthatwe followisthe onedescribed of the BA packet at the nodes triggers the iterative learning
in Di Caro and Dorigo [7], [8]. Alongside, we describe the algorithms.Ofthevariouslearningalgorithmsthathavebeen
scheme due to Bean and Costa [2], that we analyse in this proposed in the literature, we consider the one proposed by
paper. Bean and Costa [2]. In the following sections of this paper,
Every node i in the network maintains two key data we analyse this scheme for the simple problem involving
structures - a matrix of routing probabilities, the routing routingofincomingtrafficbetweenanorigin-destinationpair
table R(i), anda matrixofvariouskindsofstatistics, called through N parallel paths.
the local network information table, L(i). For a particular Bean and Costa suggest the following scheme for the
node i, let N(i,k) denote the set of neighbors of i through learning algorithms. Suppose that an FA packet measures
which node i routes packets towards destination k. For a the delay ∆iD associated with a walk (i,j,...,D). When
j
communication network consisting of M nodes, the matrix the corresponding BA packet arrives at node i the delay
of routing probabilities, R(i), has M −1 columns, corre- informationisusedto updatetheestimateofthemeandelay
sponding to the M −1 destinations towards which node i XiD using the simple exponential estimator
j
could route packets, and M −1 rows, corresponding to the
XiD :=XiD+η(∆iD −XiD), (1)
maximum number of neighbor nodes through which node i j j j j
could route packets to a particular destination. The entries where η >0 is a small constant. The mean delay estimates
ofR(i)aretheprobabilitiesφik.φik denotestheprobability XiD, correspondingto the other neighborsm of node i, are
j j m
left unchanged. acts as a probing stream in our system collecting samples
Simultaneously, the routing probabilities at the nodes are ofdelayswhile traversingthroughthequeuesalongwith the
updated using the scheme: datapackets.Thus,thepacketsofthisstreamwouldbemuch
smaller in size compared to the data packets.
β
1
φijD = (cid:16)XjiD(cid:17) β, j ∈N(i,D), (2) Ant Stream Capacity C1
1
Pk∈N(i,D)(cid:16)XkiD(cid:17)
.
where β is a constant positive integer. β influences the
.
extenttowhichoutgoinglinkswithlowerdelayestimatesare
favored compared to the ones with higher delay estimates. .
We can interpret the quantity 1 as analogous to the Source S Destination D
XiD
j
pheromonecontenton the outgoing link (i,j). Equation (2)
shows that the outgoing link (i,j) is more desirable when Data Stream Capacity CN
XjiD, the delay in routing through j, is smaller (i.e., when Fig.1. ThenetworkwithN parallel paths
the pheromone content is higher).
III. THEN PARALLEL PATHS MODEL
Themodelthatweconsiderpertainstotheroutingscenario
where arriving traffic at a single source node S has to be
Ant Stream
routedtoa singledestinationnodeD.ThereareN available λ
A
parallel paths between the source and the destination node
φ
1
through which the traffic could be routed. The network and
Q
itsequivalentqueueingtheoreticmodelareshowninFigures 1
.
1and2respectively.Thequeuesrepresenttheoutputbuffers .
(which we assume to be infinite) at the source and are . Destination
Source S D
associated with the N outgoing links. We assume in our
φ
model that the queueing delays dominate the propagation N
and the packet processing delays in the N branches. These
Data Stream QN
additional delay components can be incorporated into our λ
D
model with no additional complexity, but to keep the dis-
Fig.2. N parallel paths :Thequeueing theoretic model
cussion simple, we assume they are negligible. Two traffic
streams,anantandadatastream,arriveatthesourcenodeS.
Let {∆ (m)} denote the sequence of delays experienced
At node S, every packet of the combined stream is routed i
by successive ant packets traversing Q . Here delay refers
with probabilities φ ,...,φ (the current values) towards i
1 N
to the total waiting time in the system Q (waiting time in
the queues Q ,...,Q , respectively. These probabilities i
1 N
the queueplus packetservice time). Also, let {δ(n)} denote
are updated dynamically based on running estimates of
thesequenceofsuccessivearrivaltimesofantpacketsatthe
the means of the delays (waiting times) in the N queues.
destination node D. Then the n-th arrival of an ant packet
Samples of the delays in the N queues are collected by the
atD occursatδ(n). Supposethatthisantpackethasarrived
ant packets (these are forward ant packets) as they traverse
viaQ .WedenotethedecisionvariableforroutingbyR(n);
throughthequeues.Thesesamplesarethenusedtoconstruct i
that is, for i=1,...,N, we say that the event {R(n)=i}
the running estimates of the means of the delays in the N
has occurred if the n-th ant packet that arrives at D has
queues. We now describe our model in detail.
beenroutedviaQ .ψ (n)= n I ,thus,givesthe
We model the arrival processes of ant and data stream i i Pk=1 {R(k)=i}
number of ant packets that have been routed via Q among
packets at the source node S as independent Poisson pro- i
a total of n ant arrivals at destination D (I is the indicator
cesses of rates λ and λ packets/sec, respectively. The A
A D
randomvariableofeventA).Oncetheantpacketarrives,the
lengths of the packets of the combined stream constitute
estimate X (n) of the mean of the delay through queue Q
an i.i.d. sequence, which is also statistically independent of i i
isimmediatelyupdatedusingasimpleexponentialaveraging
the packet arrival processes. The capacity of link i is C
i
estimator
bits/sec (i=1,...,N). We assume that the length of an ant
packet is generally distributed with mean L bits, and that
A X (n)=X (n−1)+ǫ (∆ (ψ (n))−X (n−1)), (3)
i i i i i
thelengthofadatapacketisgenerallydistributedwithmean
LD bits. If we denote the service times of an ant and a data 0<ǫ<1, being a small constant.
packetin queueQ by the generic randomvariablesSA and
i i The delay estimates for the other queues are left un-
SD, then SA andSD are generallydistributed (accordingto
i i i changed, i.e.,
some c.d.f.’s, say GA and GD) with means E[SA] = LA
i i i Ci
and E[SD] = LD, respectively. The ant stream essentially X (n)=X (n−1), j ∈{1,...,N}, j 6=i. (4)
i Ci j j
Ingeneral,thus,theevolutionofthedelayestimatesinthe matters considerably. The key observation is that, when ǫ
N queuescanbedescribedbythefollowingsetofstochastic is small, the delay estimates X evolve much more slowly
i
iterative equations compared to the waiting time (delay) processes ∆ . Also,
i
because the probabilities φ are (memoryless) functions of
i
Xi(n) = Xi(n−1)+ǫ I{R(n)=i} (cid:16)∆i(ψi(n))− the delay estimates Xi, they, too, evolve at the same time-
scale as the delay estimates. Consequently, with the vec-
X (n−1) , i=1,...,N, (5)
i (cid:17) tor (X (n),...,X (n)) fixed at (z ,...,z ) (equivalently,
1 N 1 N
XaloNn(g0)w=ithxNa. set of initial conditions X1(0) = x1,..., φii=(n1),,i.=..1,,N..).,,Nthe, fidxeleadyaptrφocie=ssePsNj∆(=z1ii)((−z.)jβ,)−iβ=, 1, ..., N,
At time δ(n), besides the delay estimates, the routing can be considered as converged to a stationary distribution,
probabilities φi(n),i = 1,...,N, are also updated simul- which depends on (z1,...,zN). Also, when ǫ is small, the
taneously according to the equations evolution of the delay estimates can be tracked by a system
(X (n))−β of ODEs. A heuristic analysis of the algorithm, as provided
φ (n)= i , i=1,...,N, (6) in the Appendix,showsthat the ODE system for our case is
i PNj=1(Xj(n))−β given by
β being a constant positive integer. The initial values of the (z (t))−β D (z (t),...,z (t))−z (t)
probabilities are φ (0) = (xi)−β ,i = 1,...,N. We dz1(t) = 1 (cid:16) 1 1 N 1 (cid:17),
note that φ1(n)+·i··+φN(PnNj)==1(x1j,)−foβr all n. dt N (zk(t))−β
P
In the abovemodelwe consideronly forwardantsand do j=1
. .
notincorporatetheeffectsofbackwardants.Moreprecisely, . .
. .
we assume that the estimates X (.) of the means of the
i (z (t))−β D (z (t),...,z (t))−z (t)
delays and the probabilities φi(.) are updated as soon as dzN(t) = N (cid:16) N 1 N N (cid:17),
the (forward) ant packets arrive at the destination D, and dt N
(z (t))−β
this informationis available instantaneouslythereafterat the k
P
j=1
sourcenodeS. We do notconsiderthusthe additionaldelay
(7)
involvedasthebackwardantpacketstravelbackcarryingthe
delay information to the source. Because backward ants are with the set of initial conditions z1(0) = x1,...,zN(0) =
expectedtotravelbacktothesourcethroughpriorityqueues, xN. Di(z1,...,zN),i = 1,...,N, are the mean waiting
the effects of delayed information might not be very signif- times in the queues under stationarity (as seen by arriving
icant, except for large-sized networks. On the other hand, ant packets) with the delay estimates considered fixed at
incorporating the effect of delays in our model introduces z1,...,zN.
additional asynchrony, making the problem harder. Formally, the ODE approximation result can be stated as
Another important point to note is that our delay esti- follows(see Benveniste,Metivier,and Priouret[3]). For any
mation scheme (5) is a constant step size scheme. As is fixed ǫ > 0 and for i = 1,...,N, consider the piecewise
well knownin the literature on adaptivealgorithms(see, for constant interpolation of Xi(n) given by the equations :
example [3]), this enables the scheme to adapt to (track) ziǫ(t) = Xi(n) for t ∈ [nǫ,(n + 1)ǫ ), n = 0,1,2,...,
longtermchangesinstatisticsofthedelayprocesses.Thisis with the initial value ziǫ(0) = Xi(0). Then the processes
importantforcommunicationnetworks,becausethestatistics {ziǫ(t),t≥0},i=1,...,N, converge to the solution of the
of arrival processes at the nodes as well as the network ODE system (7) in the following sense : as ǫ ↓ 0, for any
characteristics typically change with time. 0≤T <∞,
sup |zǫ(t)−z (t)|−P→0, (8)
IV. ANALYSISOFTHE ALGORITHM i i
0≤t≤T
We view the routing algorithm, consisting of equations
P
where −→ denotes convergence in probability.
(5) and (6), as a set of discrete stochastic iterations of
In order to obtain the evolution of the ODE, we need
the type usually considered in the literature on stochastic
to compute the quantities D (z ,...,z ), for our queueing
approximation methods [3]. We provide below the main i 1 N
system.WerecallthatD (z ,...,z ),i=1,...,N,referto
convergenceresultwhichstatesthat,whenǫissmallenough, i 1 N
themeansofthewaitingtimesasseenbyantpacketarrivals
the discrete iterations are closely tracked by a system of
to the queues when the delay estimates are considered fixed
Ordinary Differential Equations (ODEs). In the Appendix,
atz ,...,z .ThentheroutingprobabilitiestotheN queues
we providea heuristic analysiswhich enablesus to arriveat 1 N
are φ = (zi)−β ,i = 1,...,N. We now discuss
the appropriate ODE. i PNj=1(zj)−β
how to compute the quantities D (z ,...,z ) given our
i 1 N
A. The ODE Approximation
assumptions on the statistics of the arrival processes and on
An analysis of the dynamics of the system, as given the packet lengths of the arrival streams.
by equations (5) and (6), is fairly complicated. However, Undersuchconditions,everyincomingarrivalatsourceS
when ǫ > 0 is small, a time-scale decomposition simplifies fromeitherofthePoissonstreams(theantorthedatastream)
is routed (independent of other arrivals) with probability (zN∗ )−β . D (z∗,...,z∗)−z∗ = 0. (10)
φ towards queue Q . Thus the incoming arrival process in N h N 1 N Ni
i i (z∗)−β
queue Qi (for each i) is a superposition of two independent jP=1 j
Poisson processeswith ratesλ φ and λ φ . Consequently,
every incoming packet into QiAisi, with pDrobiability λAλ+AλD, Ttohtehseteaavdeyrasgteatdeerloauytiensgtimpraotbeas,bzil∗it,i.e.s,.φ,z∗1∗,.,.t.h,rφou∗Ng,hartheereelqautead-
an ant packet, and with probability λD , a data packet. 1 N
Also, under our assumptions on the λstAa+tiλstDics of the packet tions, φ∗i = PN(zi∗()z−∗β)−β, i = 1,...,N. Because we have
j=1 j
lengths of the arrival streams and on the arrival processes, assumedthatourqueuesareinthestableregionofoperation,
all of the queues evolve as independent M/G/1 queues. the steady state estimates of average delays must be finite,
The cumulative incoming stream into Q is Poisson with and so z∗ must be finite for every i = 1,...,N. Then
i i
irsatedi(sλtrAib+uteλdD)aφcic,oarnddinegvetorytihnecocm.din.fgGpaAckweti’tshseprrvoibceabtiilmitye the steady state routing probabilities, φ∗i = PNj(=z1i∗()z−j∗β)−β,
i i=1, ..., N, are all strictly positive. Equations(10) then
λλAAλλ++DAλλDD. aWnde faucrcthoerdrinasgsutmoethtehatc.tdh.ef.qGueDiuewsiathrepwroitbhainbiltihtye nreodtiuccee, ftorom: zei∗qu=atiDoni((z91∗),,..th.a,tzN∗fo)r,iea=ch1i,,.D..i,(Nz1∗.,W..e.,azlN∗so)
stability region of operation given by the inequalities : is a function solely of φ∗, and so, with a slight abuse of
i
(λA + λD)φiE[Si] < 1, i = 1, ..., N, where E[Si], notation, we denote it by Di(φ∗i). Then, utilizing the fact
the mean packet service time in Qi, is given by E[Si] = that φ∗ = (zi∗)−β , we find that the equilibrium routing
iλnAEth[SeλiAA]s++yλλsDDteEm[SiDas]. eWxpeernioetnecetdhatbythesuacvceersasgiveewaanittinagrritvimales probabiilitiePs,Nj=φ1∗1(,z.j∗.)−.,βφ∗N, must satisfy the following fixed-
point system of equations
to queue Q , is the same as the average waiting time in
i
Qi by the PASTA (Poisson Arrivals See Time Averages) φ∗ = (D1(φ∗1))−β ,
property. Thus, using the Pollaczek-Khinchin formula for 1 N
the average waiting time and assuming that the queues are P (Dj(φ∗j))−β
j=1
stable, we finally obtain the expression for D (z ,...,z )
i 1 N . .
(i=1,...,N): .. ..
Di(z1,...,zN)=E[Si]+ (λA+λD)φiE[Si2] , (9) φ∗N = N(DN(φ∗N))−β . (11)
2(1−(λA+λD)φiE[Si]) (D (φ∗))−β
P j j
j=1
where E[S ] and E[S2] are given by E[S ] =
λAE[SiA]+λDEi[SiD] andE[Si2]= λAE[(SiA)2]+λDE[(SiD)i2],and Notice that φ∗1+···+φ∗N =1.
λA+λD i λA+λD An important point to note is that the steady state proba-
φi = PNj(=z1i)(−zjβ)−β. bilities, φ∗1,...,φ∗N, must not only satisfy the above system
OncetheexpressionsforD (z ,...,z )areavailable,we of equations, but must all be strictly positive and satisfy
i 1 N
can numerically solve the ODE system (7), starting with the following inequalities which are the stability conditions
the initial conditions z1(0),...,zN(0). We observe in our for the system: (λA +λD)φ∗iE[Si] < 1,i = 1,...,N. We
simulationsthatifwe startthesystemwith initialconditions now show that the system of equations (11) are actually
such that we are inside the stability region,the system stays the necessary and sufficient optimality conditions for an
within the stability region thereafter. optimizationprobleminvolvingtheminimizationofaconvex
objective function of (φ ,...,φ ) subject to the above
1 N
B. Equilibrium behavior of the routing algorithm mentioned constraints. We show as a consequence that, if
there exists a solution to the set of equations (11) that
We now obtain the equilibriumpointsof the ODE system
also satisfies the above mentioned constraints, then such a
(7)whichwould,inturn,enableustoobtaintheequilibrium
solution is unique.
routing behavior of the system. In particular, we can obtain
Consider the optimization problem
the equilibriumrouting probabilitiesand the mean delays in
thesystemundersteadystateoperationofthenetwork.Forǫ Minimize F(φ ,...,φ )= N φix[D (x)]βdx,
1 N Pi=1R0 i
small, the steady state valuesofthe estimates of the average subject to φ +···+φ =1,
1 N
waiting times (delays) in the N queues are approximately 0<φ <a ,
1 1
given by the components of the equilibrium points, z∗, of ..
.
the ODE system (7). The equilibrium points of the ODE,
0<φ <a ,
z∗, must satisfy the set of equations given by N N
where a = 1 ,i=1,...,N.
(z1∗)−β . D (z∗,...,z∗)−z∗ = 0, Let usi den(λoAte+bλDy)CE[Stih]e set defined by the constraints of
N h 1 1 N 1i the above optimization problem – the feasible set. It is easy
P (zj∗)−β to see that C is a convex subset of RN. It is possible that
j=1
the set C is empty(for a givenset of valuesof λ , λ , and
. . A D
. .
. . E[S ],i=1,...,N),whichmeansthatthereare nofeasible
i
solutions to the above optimization problem in such a case. conditions. Then for every other vector φ ∈ C, we have
We assume, in what follows, that there exists at least one N (φ −φ∗)=0. So, the quantity
Pi=1 i i
feasible solution to the above optimization problem, i.e., C
N N
is non-empty. ∂F (φ∗)(φ −φ∗)= ∂F (φ∗) (φ −φ∗)=0.
Before we attempt to solve the optimization problem, we X∂φi i i ∂φ1 X i i
i=1 i=1
make certain natural assumptions on the delay functions
Then, because F is convex over C, by Proposition 2.1.2
D (x),i = 1,...,N. We assume that the functions D (x)
i i of Bertsekas [4], φ∗ is a local minimum. (cid:3)
are positive, real-valued, differentiable and monotonically
Forourmodel,itiseasytocheckthatthefunctionsD (x),
increasing on their domains of definition. This holds true in i
as given by (9), are positive, real-valued, differentiable and
most cases of interest, because when the routing probability
monotonically increasing on their domains of definition.
for an outgoing link increases, the amount of traffic flow
Thus, there is a unique equilibrium probability vector φ∗
into that link also increases, resulting in an increase of the
which satisfies the fixed-point equations (11).
delay. The following proposition characterizes the optimal
solutions φ∗ of the above optimization problem. V. SIMULATION RESULTS AND DISCUSSION
Proposition 1: Given the aboveassumptionson the delay
We describe in this section an illustrative example. The
functions Di(x),i = 1,...,N, a probability vector φ∗ is a queueing system as described in Section III has been im-
local minimum of F over C if and only if φ∗ satisfies the plemented using a discrete event simulator. We present here
set of fixed-pointequations (11). φ∗ is then also the unique results for the case when the number of parallel paths is
global minimum of F over C. N =3. The step size ǫ and the parameter β were set at the
Proof: The Hessian of F is a diagonal matrix given by
values 0.002 and 1, respectively.
∇2F(φ ,...,φ ) = diag [D (φ )]β−1{D (φ )+ The ant anddata traffic arrivalprocessesare Poisson with
1 N (cid:16) i i i i
rates λ =1 and λ =1, respectively. For the ant packets,
A D
βφ D′(φ )} , (12) the service times in the three queues are exponential with
i i i (cid:17)
meansE[SA]=1/3.0,E[SA]=1/4.0andE[SA]=1/5.0,
1 2 3
whereDi′(.)denotesthederivativeofDi(.).Undertheabove respectively. For the data packets also, the service times in
assumptions on the Di(x)’s, ∇2F(φ1,...,φN) is positive thethreequeuesareexponentialwithmeansE[SD]=1/3.0,
1
definite over C, and so F is a strictly convex function on E[SD]=1/4.0andE[SD]=1/5.0,respectively.Theinitial
2 3
C. Consequently, any local minimum of F is also a global values of the delay estimates in the three queueswere set at
minimumofF overC;furthermore,thereisatmostonesuch X (0) = 0.8, X (0) = 2.8, and X (0) = 5.6. Then, the
1 2 3
global minimum [4]. initial routing probabilities are φ (0) = 0.7, φ (0) = 0.2,
1 2
If φ∗ = (φ∗1,...,φ∗N) is a local minimum of F over C, and φ3(0)=0.1, which ensures that, initially, we are inside
we must have (Proposition 2.1.2 of Bertsekas [4]), the stability region of the queueing sytem. We observed in
N our simulations that as the queueing system evolved over
∂F
(φ∗)(φ −φ∗)≥0,∀φ∈C. (13) time, never did the system become unstable.
X∂φ i i
i=1 i Figures 3, 4 and 5 provide plots of the interpolated
Let us fix a pair of indices i,j,i 6= j. Then choose delay estimates (zǫ(t),i = 1,2,3) in the three queues,
i
φ = φ∗+δ and φ = φ∗ −δ, and let φ = φ∗,∀k 6= i,j. averaged over ten sample paths, versus the ODE approxi-
i i j j k k
Now, choosing δ > 0 small enough that the vector φ = mation, z1(t),z2(t),z3(t), obtained by numerically solving
(φ ,...,φ ) is also in C, the above condition becomes (7). We see that the theoretical ODE tracks the simulated
1 N
delay estimates fairly well. Figures 6, 7, and 8 provide
∂F ∂F
(φ∗)− (φ∗) δ ≥0, plots of routing probabilities φ (n),φ (n), and φ (n). The
(cid:16)∂φ ∂φ (cid:17) 1 2 3
i j routingprobabilitiesconvergetotheequilibriumvaluesφ∗ =
1
or, φ∗[D (φ∗)]β ≥φ∗[D (φ∗)]β. 3/12,φ∗ = 4/12,φ∗ = 5/12, which is actually the unique
i i i j j j 2 3
solution to equations (11). These probabilities are in the
By a similar argument, we can show that φ∗[D (φ∗)]β ≥
j j j reverse order as packet service times in the three queues,
φ∗i[Di(φ∗i)]β. Thus, the necessary conditions for φ∗ to be a with link 3 having the highest equilibrium probability, and
local minimum are link 1 the lowest.
φ∗1[D1(φ∗1)]β =···=φ∗N[DN(φ∗N)]β. VI. CONCLUSIONS
Combiningthis with the normalizationcondition,φ∗+···+ We haveprovidedconvergenceresults foranAntRouting
1
φ∗ =1, gives us the system of equations (11). Algorithm for a simple network consisting of N parallel
N
The necessary conditionsabovecan also be written in the paths between a source-destination pair. We have explicitly
form modeled the link delays using a stochastic queueing model,
∂F ∂F
(φ∗)=···= (φ∗). and we have studied a routing scheme where the routing
∂φ1 ∂φN probabilities are updated based on estimates of path delays.
We check that these conditions are also sufficient for φ∗ to Wehavealsoshownthattheequilibriumroutingprobabilities
be a local minimum. Suppose φ∗ ∈ C satisfies the above are solutions of a fixed-point system of equations, which in
turn, form the necessary and sufficientoptimality conditions following approximate equations
for a convex optimization problem. We aim to extend the
X (n+M) ≈ X (n)+Mǫ P (n,M)−
analysis to the network case, where multiple traffic streams 1 1 (cid:16) 1
with different destinations share a network of links.
Q (n,M)X (n) ,
1 1 (cid:17)
. .
. .
. .
REFERENCES
X (n+M) ≈ X (n)+Mǫ P (n,M)−
N N (cid:16) N
[1] J. S. Baras, and H. Mehta, A Probabilistic Emergent Routing Algo-
Q (n,M)X (n) . (15)
rithm for Mobile AdHoc Networks, Proc. WiOpt03: Modeling and N N (cid:17)
Optimization in Mobile, Ad Hoc and Wireless Networks, Sophia-
Antipolis, France, March3-5,2003. Now we try to find approximations to the quantities
[2] RNo.uBtienagn,AAlg.orCitohsmtas, tAhnatAUnsaely“tAicntM-liokdee”linMgoAbipleprAoagcehntsfo,rCNometpwuoterkr Pi(n,M) = PMk=1I{R(n+k)M=i}∆i(ψi(n+k)) and Qi(n,M) =
Networks, pp.243-268,vol.49,2005. PMk=1I{R(n+k)=i} for i = 1,...,N, when the values of
M
[3] A.Benveniste, M.Metivier, andP.Priouret,AdaptiveAlgorithmsand the mean delay estimates X (.),...,X (.) are considered
1 N
Stochastic Approximation, Appl.ofMathematics, Springer, 1990.
fixed at X (n),...,X (n), and M is large. Then the
[4] D.P.Bertsekas, Nonlinear Programming,AthenaScientific, 1995. 1 N
routing probability vector (φ (.),...,φ (.)), too, can be
[5] E.Bonabeau,M.Dorigo,andG.Theraulaz,SwarmIntelligence:From 1 N
NaturaltoArtificialSystems,SantaFeInstituteStudiesintheSciences regarded as essentially constant in the interval {n,...,n+
ofComplexity, OxfordUniversity Press;1999. M}, because the probabilities are continuous functions of
[6] V.S.Borkar,P.R.Kumar,DynamicCesaro-WardropEquilibrationin
the mean delay estimates. The routing probabilities are
Networks, IEEETr.onAutom.Control,pp.382-396,v.48,3,2003.
[7] CGo.mDmiCunaircoa,tiMon.DNoertwigoor,kAs,nJtNouertn:aDliostfriAbruttiefidciaSltiIgnmteelrliggeetnicceCRonetsreoalrcfohr, then approximately equal to φi(n) = PNj(=X1i((Xnj))(−n)β)−β,i =
1, ..., N. Now, assuming that M is large enough that
pp.317-365, vol.9,1998.
a law of large numbers effect takes over, the average
[8] M.Dorigo,T.Stutzle,AntColonyOptimization.TheMITPress;2004.
[9] M. Gunes, U. Sorges, and I. Bouazizi, ARA-The Ant-colony Based PMk=1I{R(n+k)=i}, which is the fraction of ant packets that
RoutingAlgorithmforMANETs,inS.Olariu(Ed.),Proc.2002ICPP M
have arrived at destination via Q when the routing prob-
WorkshoponAdHocNetworks,pp.79-85,IEEEComp.Soc.Press. i
[10] W. J. Gutjahr, A Generalized Convergence Result for the Graph- abilities are φi(n), can be approximated by φi(n). With
based Ant System Metaheuristic, Probability inthe Engineering and the routing probabilities fixed, the delay processes ∆ (.),
i
Informational Sciences, pp.545-569,vol.17,2003.
can converge to a stationary distribution, the mean un-
[11] R.Schoonderwoerd etal,Ant-BasedLoadBalancing inTelecommu-
der stationarity being denoted by D (X (n),...,X (n)).
nications Networks, AdaptiveBehavior,pp.169-207,vol.5,1996. i 1 N
[12] J.-H.Yoo,R.J.La,andA.M.Makowski,ConvergenceResultsforAnt The quantities PMk=1I{R(n+k)=i}∆i(ψi(n+k)) can then be
Routing, Proc.Conf.onInf.Sc.andSystems,Princeton, NJ,2004. M
approximated by φ (n).D (X (n),...,X (n)). Note that
i i 1 N
D (X (n),...,X (n)) are the mean waiting times as seen
i 1 N
by ant packets. Employing the approximations as described
APPENDIX
above, we notice from (15) that the evolution of the vector
(X (n),...,X (n)) resembles that of a discrete-time ap-
The heuristic analysis provided below is largely inspired 1 N
proximation to the following ODE system when ǫ is small
by Benveniste, Metivier, and Priouret (Section 2.2, Chapter
enough,
2) [3]. Let us consider the mean delay estimation scheme
given by equation (5). We can write, for a positive integer (z (t))−β D (z (t),...,z (t))−z (t)
dz (t) 1 (cid:16) 1 1 N 1 (cid:17)
M, 1 = ,
dt N
(z (t))−β
k
M jP=1
X (n+M) = X (n)+ǫ I . .
1 1 X {R(n+k)=1} .. ..
k=1
(z (t))−β D (z (t),...,z (t))−z (t)
(cid:16)∆1(ψ1(n+k))−X1(n+k−1)(cid:17), dzN(t) = N (cid:16) N 1 N N (cid:17),
dt N
.. .. (z (t))−β
. . k
P
j=1
M
(16)
X (n+M) = X (n)+ǫ I
N N X {R(n+k)=N}
k=1 with the set of initial conditions z (0) = x ,...,z (0) =
1 1 N
(cid:16)∆N(ψN(n+k))−XN(n+k−1)(cid:17). xN.
(14)
If ǫ>0 is small enough, the vector (X (n),...,X (n))
1 N
can be assumed to have not changed much in the discrete
interval {n,n + 1,...,n + M}, and we can write the
1 0.9
zε (t), obtained from simulation
1
0.9 The ODE z (t) 0.8
1
0.8 0.7
0.7
0.6
0.6
0.5
0.5 φ (n)10.4
0.4
0.3
0.3
0.2 0.2
0.1 0.1
0 0
0 5 10 15 20 25 30 35 40 45 0.5 1 1.5 2 2.5
t n x 104
Fig.3. TheODEapproximation forX1(n) Fig.6. Theplotforφ1(n)
3.5 0.9
zε (t), obtained from simulation
2
The ODE z (t) 0.8
3 2
0.7
2.5
0.6
2 0.5
1.5 φ (n)20.4
0.3
1
0.2
0.5
0.1
0 0
0 5 10 15 20 25 30 35 40 45 0.5 1 1.5 2 2.5
t n x 104
Fig.4. TheODEapproximation forX2(n) Fig.7. Theplotforφ2(n)
0.9
zε (t), obtained from simulation
6 3
The ODE z (t) 0.8
3
5 0.7
0.6
4
0.5
3 φ (n)30.4
0.3
2
0.2
1
0.1
0 0
0 5 10 15 20 25 30 35 40 45 0.5 1 1.5 2 2.5
t n x 104
Fig.5. TheODEapproximation forX3(n) Fig.8. Theplotforφ3(n)