Ant Colony Optimization and its Application to Adaptive Routing in Telecommunication Networks

370 Pages·2012·2.59 MB·English

UNIVERSITE· LIBRE DE BRUXELLES
FACULTE· DES SCIENCES APPLIQUE·ES
Ant Colony Optimization and its Application to
Adaptive Routing in Telecommunication Networks
Gianni Di Caro
Dissertationpre·sente·e envuedel’obtentiondugradede
DocteurenSciencesApplique·es
Bruxelles,September2004
PROMOTEUR: PROF. MARCO DORIGO
IRIDIA, InstitutdeRecherchesInterdisciplinaires
etdeDe·veloppementsenIntelligenceArti(cid:2)cielle
ANNE·E ACADE·MIQUE 2003-2004

Abstract
Inantsocieties,and,moreingeneral,ininsectsocieties,theactivitiesoftheindividuals,aswell
asofthesocietyasawhole,arenotregulatedbyanyexplicitformofcentralizedcontrol. Onthe
otherhand,adaptiveandrobustbehaviorstranscendingthebehavioralrepertoireofthesingle
individualcanbeeasilyobservedatsocietylevel. Thesecomplexglobalbehaviorsaretheresult
ofself-organizingdynamicsdrivenbylocalinteractionsandcommunicationsamonganumber
ofrelativelysimpleindividuals. Thesimultaneouspresenceoftheseandotherfascinatingand
unique characteristics have made ant societies an attractive and inspiring model for building
newalgorithmsandnewmulti-agentsystems.Inthelastdecade,antsocietieshavebeentakenasa
referenceforanevergrowingbodyofscienti(cid:2)cwork,mostlyinthe(cid:2)eldsofrobotics,operations
research,andtelecommunications.
Amongthedifferentworksinspiredbyantcolonies,theAntColonyOptimizationmetaheuristic
(ACO)isprobablythemostsuccessfulandpopularone.TheACOmetaheuristicisamulti-agent
frameworkforcombinatorialoptimizationwhosemaincomponentsare: asetofant-likeagents,
the use of memory and of stochastic decisions, and strategies of collective and distributed learning.
It (cid:2)nds its roots in the experimental observation of a speci(cid:2)c foraging behavior of some ant
coloniesthat,underappropriateconditions,areabletoselecttheshortestpathamongfewpossi-
blepathsconnectingtheirnesttoafoodsite. Thepheromone,avolatilechemicalsubstancelaid
onthegroundbytheantswhilewalkingandaffectinginturntheirmovingdecisionsaccording
toitslocalintensity,isthemediatorofthisbehavior. Alltheelementsplayinganessentialrole
in the ant colony foraging behavior were understood, thoroughly reverse-engineered and put
toworktosolve problemsofcombinatorial optimization byMarcoDorigoandhisco-workers
at the beginning of the 1990’s. From that moment on it has been a (cid:3)ourishing of new com-
binatorial optimization algorithms designed after the (cid:2)rst algorithms of Dorigo’s et al., and of
related scienti(cid:2)c events. In 1999 the ACO metaheuristic was de(cid:2)ned by Dorigo, Di Caro and
Gambardellawiththepurposeofprovidingacommonframeworkfordescribingandanalyzing
allthesealgorithmsinspiredbythesameantcolonybehaviorandbythesamecommonprocess
ofreverse-engineeringofthisbehavior. Therefore, theACOmetaheuristicwasde(cid:2)nedaposte-
riori,astheresultofasynthesisefforteffectuatedonthestudyofthecharacteristicsofallthese
ant-inspired algorithms and on the abstraction of their common traits. The ACO’s synthesis
wasalsomotivatedbytheusuallygoodperformanceshownbythealgorithms(e.g.,forseveral
importantcombinatorialproblemslikethequadraticassignment, vehicleroutingandjobshop
scheduling,ACOimplementationshaveoutperformedstate-of-the-artalgorithms).
Thede(cid:2)nitionandstudyoftheACOmetaheuristicisoneofthetwofundamentalgoalsofthe
thesis. Theotherone,strictlyrelatedtothisformerone,consistsinthedesign,implementation,
andtestingofACOinstancesforproblemsofadaptiveroutingintelecommunicationnetworks.
This thesis is an in-depth journey through the ACO metaheuristic, during which we have
(re)de(cid:2)nedACOandtriedtogetaclearunderstandingofitspotentialities,limits,andrelation-
shipswithotherframeworksandwithitsbiologicalbackground. Thethesistakesintoaccount
allthedevelopmentsthathavefollowedtheoriginal1999’sde(cid:2)nition,andprovidesaformaland
comprehensivesystematizationofthesubject,aswellasanup-to-dateandquitecomprehensive
review of current applications. We have also identi(cid:2)ed in dynamic problems in telecommuni-
cation networks the most appropriate domain of application for the ACO ideas. According to
this understanding, in the most applicative part of the thesis we have focused on problems of
adaptiveroutinginnetworksandwehavedevelopedandtestedfournewalgorithms.
Adopting an original point of view with respect to the way ACO was (cid:2)rstly de(cid:2)ned (but
maintaining full conceptual and terminological consistency), ACO is here de(cid:2)ned and mainly
discussedinthetermsofsequentialdecisionprocessesandMonteCarlosamplingandlearning.More
precisely,ACOischaracterizedasapolicysearchstrategyaimedatlearningthedistributedpa-
rameters(calledpheromonevariablesinaccordancewiththebiologicalmetaphor)ofthestochastic
decisionpolicywhichisusedbyso-calledantagentstogeneratesolutions. Eachantrepresents
inpracticeanindependentsequentialdecisionprocessaimedatconstructingapossiblyfeasibleso-
lutionfortheoptimizationproblemathandbyusingonlyinformationlocaltothedecisionstep.
Antsarerepeatedlyandconcurrentlygeneratedinordertosamplethesolutionsetaccordingtothe
currentpolicy. Theoutcomesofthegeneratedsolutionsareusedtopartiallyevaluatethecurrent
policy,spotthemostpromisingsearchareas,andupdatethepolicyparametersinordertopossibly
focusthesearchinthosepromisingareaswhilekeepingasatisfactorylevelofoverallexploration.
ThiswayoflookingatACOhasfacilitatedtodisclosethestrictrelationshipsbetweenACO
and other well-known frameworks, like dynamic programming, Markov and non-Markov decision
processes,andreinforcementlearning. Inturn,thishasfavoredreasoningonthegeneralproperties
ofACOintermsofamountofcompletestateinformationwhichisusedbytheACO’santstotake
optimized decisions and to encode in pheromone variables memory of both the decisions that
belongedtothesampledsolutionsandtheirquality.
The ACO’s biological context of inspiration is fully acknowledged in the thesis. We report
with extensive discussions on the shortest path behaviors of ant colonies and on the identi(cid:2)-
cation and analysis of the few nonlinear dynamics that are at the very core of self-organized
behaviors in both the ants and other societal organizations. We discuss these dynamics in the
generalframeworkofstigmergicmodeling,basedonasynchronousenvironment-mediatedcom-
munication protocols, and (pheromone) variables priming coordinated responses of a number
of(cid:147)cheap(cid:148)andconcurrentagents.
ThesecondhalfofthethesisisdevotedtothestudyoftheapplicationofACOtoproblems
ofonlineroutingintelecommunicationnetworks. Thisclassofproblemshasbeenidenti(cid:2)edinthe
thesisasthemostappropriatefortheapplicationofthemulti-agent, distributed, andadaptive
natureoftheACOarchitecture. FournovelACOalgorithmsforproblemsofadaptiveroutingin
telecommunicationnetworksarethroughlydescribed. Thefouralgorithmscoverawidespec-
trumofpossibletypesofnetwork: twoofthemdeliverbest-efforttraf(cid:2)cinwiredIPnetworks,one
isintendedforquality-of-service(QoS)traf(cid:2)cinATMnetworks,andthefourthisforbest-efforttraf-
(cid:2)c in mobile ad hoc networks. The two algorithms for wired IP networks have been extensively
testedbysimulationstudiesandcomparedtostate-of-the-artalgorithmsforawidesetofrefer-
encescenarios. Thealgorithmformobileadhocnetworksisstillunderdevelopment,butquite
extensive results and comparisons with a popular state-of-the-art algorithm are reported. No
resultsarereportedforthealgorithmforQoS,whichhasnotbeenfullytested. Theobservedex-
perimentalperformanceisexcellent,especiallyforthecaseofwiredIPnetworks:ouralgorithms
always perform comparably or much better than the state-of-the-art competitors. In the thesis
wetrytounderstandtherationalebehindthebrilliantperformanceobtainedandthegoodlevel
ofpopularityreachedbyouralgorithms. Moreingeneral,wediscussthereasonsofthegeneral
ef(cid:2)cacyoftheACOapproachfornetworkroutingproblemscomparedtothecharacteristicsof
moreclassicalapproaches. Movingfurther,wealsoinformallyde(cid:2)neAntColonyRouting(ACR),
a multi-agent framework explicitly integrating learning components into the ACO’s design in
ordertode(cid:2)neageneralandinasensefuturisticarchitectureforautonomicnetworkcontrol.
Most of the material of the thesis comes from a re-elaboration of material co-authored and
publishedinanumberofbooks,journalpapers,conferenceproceedings,andtechnicalreports.
ThedetailedlistofreferencesisprovidedintheIntroduction.
Acknowledgments
First,Ishallthanktheants! Withoutthoselittlematesbuggingaroundthisthesissimplywould
have not happened. When I was a little kid, I was kind of concerned about the little ant-hills
and their tiny inhabitants sprouting all over my garden. So, one day, I started thinking that
duringthewintertheycouldfeelcold,andIunilaterallydecidedtoequiptheirprimitivenests
withatechnologicallyadvancedheatingsystem. Therefore,Istarteddrillingandputtingpipes
underground,connectingthepipestofaucetsandsinks,andlettinghotwaterpassingthrough
in order to warm up my little mates during the cold winter days. How disappointed I was
whenIrealizedthattheydidn’tappreciatemyheatingtechnologyandquiteinashorttimethey
just vanished, likely moving their nests somewhere else! Well, actually, I then tried to let my
adorablecattoenjoymyconcernsaboutcoldandIbroughtmynowwellconsolidatedheating
technologyintothecartonboxwherehewasusedtospend,sleeping,mostofhisprecioustime.
Unfortunately,hetoodidn’tseemreadytoenjoythepleasuresoftechnology, soIgaveupand
focusedonotheramenities...So,Idoreallyfeelnowthatinasensemylittlemateshadactually
appreciatedthosemaybeabitawkwardeffortsandrewardedmeback,layingsomewhereinthe
universethepheromonetrailthatledmetoMarcoDorigoandhisant-inspiredalgorithms. He’s
thepersonIshallthankthemoreforwhatisinthisthesis,andforreallybuggingmetowriteit!
Thecontent ofthe thesis covers part oftheresearch work thatI’ve done during thelast six
years.Theseyearshavebeenalongjourneyintoscienceandintolife.I’vemoved(cid:2)rsttoBrussels
(IRIDIA), then to Japan (ATR), then back to Belgium (IRIDIA again), and (cid:2)nally to Lugano, in
Switzerland (IDSIA). All these places have been great places to work and enjoy life. They are
fullyinternationalenvironmentswhereIhadthechancetomakereallygoodfriendsandtomeet
so many great scientists from whom I could really learn so much! I want to thank for this the
directors of these laboratories where I have been (Philippe Smets, Hugues Bersini and Marco
Dorigo,KatsunoriShimohara,KenjiDoya,andLucaGambardella)whohavedoneandarestill
doingagreatjob!
Thisthesisisreallytheoutgrowthofallthediscussions,brainstorming,collaborations,that
havehappenedduringtheseyears. It’slikeapuzzle,whosetileshavebeenaddeddaybyday:
after a discussion in front of several beers, or after attending a seminar about how Japanese
can learn to distinguish between the sounds (cid:147)lock(cid:148) and (cid:147)rock(cid:148), or after a long telephone call
withsomefriendthousandsofkilometersfarawaydiscussingaboutmarkovdecisionprocesses
or antibodies at 3am...Thinking back, I realize that I should thank so many people across the
world, but they are really too many. I want to mention here just few of them, those who con-
sciously or unconsciously (!), in different ways, have had a major impact on my growth as a
scientist. Thelistisin(stochastic)orderofappearance: AlessandroSaf(cid:2)otti,BrunoMarchal,Vit-
torio Gorrini, Marco Dorigo, Hugues Bersini, Gianluca Bontempi, Katsunori Shimohara, Kenji
Doya,ThomasHalvaLabella,LucaMariaGambardella,andFrederickDucatelle.
TherearealsofewveryspecialpeoplethatIreallywanttomention(instrictalphabeticorder,
this time). They have been the closest ones, the most important ones, either from a scienti(cid:2)c
pointofviewand/orfromapersonalpointview. Tothemareallyspecial(cid:147)Grazie!(cid:148) goesdeep
down from my heart: Carlotta Piscopo, Luca Di Mauro, Masako (Maco-chan) Hosomi, Mauro
Birattari, Silvana Valensin. I hope there’ll always be a trail of pheromone somewhere in the
universeto(cid:2)ndeachother.
TothislistofveryspecialpeopleImustaddmymother,alwaystheretolistentomymisfor-
tunes,togivemeawarmword,toencourageme,andwhogavemethe(cid:2)rstrealimpetustosit
downandwritethe(cid:2)rstwordsofthisthesis: (cid:147)GrazieMamma!(cid:148).
AspecialthankgoesalsotoLucaGambardella, whoverykindlyletmealsotoworkatthe
thesis while I was/am in his group at IDSIA, and who has always been a source of valuables
suggestionsandpointsofview.
Part of the work in this thesis was supported by two Marie Curie fellowships (contract n.
CEC-TMRERBFMBICT961153andHPMF-CT-2000-00987)awardedtotheauthorbythescien-
ti(cid:2)cinstitutionsoftheEuropeanCommunity.Theinformationprovidedisthesoleresponsibility
oftheauthoranddoesnotre(cid:3)ecttheCommunity’sopinion. TheCommunityisnotresponsible
foranyusethatmightbemadeofdataappearinginthisthesis.
Contents
ListofFigures xii
ListofTables xiii
ListofAlgorithms xv
ListofExamples xvii
ListofAbbreviations xix
1 Introduction 1
1.1 Goalsandscienti(cid:2)ccontributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.1 Generalgoalsofthethesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.1.2 Theoreticalandpracticalrelevanceofthethesis’goals. . . . . . . . . . . . . 6
1.1.3 Generalscienti(cid:2)ccontributions . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.2 Organizationandpublishedsourcesofthethesis . . . . . . . . . . . . . . . . . . . . 10
1.3 ACOinanutshell: apreliminaryandinformalde(cid:2)nition . . . . . . . . . . . . . . . 17
I FromrealantcoloniestotheAntColonyOptimizationmetaheuristic 21
2 Antcolonies,thebiologicalrootsofAntColonyOptimization 23
2.1 Antscoloniescan(cid:2)ndshortestpaths . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.2 Shortestpathsastheresultofasynergyofingredients . . . . . . . . . . . . . . . . . 27
2.3 Robustness,adaptivity,andself-organizationproperties. . . . . . . . . . . . . . . . 30
2.4 Modelingantcolonybehaviorsusingstigmergy: theantway. . . . . . . . . . . . . 31
2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3 Combinatorialoptimization,constructionmethods,anddecisionprocesses 37
3.1 Combinatorialoptimizationproblems . . . . . . . . . . . . . . . . . . . . . . . . . . 39
3.1.1 Solutioncomponents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
3.1.2 Characteristicsofproblemrepresentations . . . . . . . . . . . . . . . . . . . 44
3.2 Constructionmethodsforcombinatorialoptimization . . . . . . . . . . . . . . . . . 46
3.2.1 Strategiesforcomponentinclusionandfeasibilityissues . . . . . . . . . . . 49
3.2.2 Appropriatedomainsofapplicationforconstructionmethods . . . . . . . . 51
3.3 Constructionprocessesassequentialdecisionprocesses . . . . . . . . . . . . . . . . 52
3.3.1 Optimalcontrolandthestateofaconstruction/decisionprocess . . . . . . 54
3.3.2 Stategraph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.3.3 Constructiongraph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
3.3.4 ThegeneralframeworkofMarkovdecisionprocesses . . . . . . . . . . . . . 67
3.3.5 Genericnon-Markovprocessesandthenotionofphantasma . . . . . . . . . 72
viii CONTENTS
3.4 Strategiesforsolvingoptimizationproblems . . . . . . . . . . . . . . . . . . . . . . 75
3.4.1 Generalcharacteristicsofoptimizationstrategies . . . . . . . . . . . . . . . 76
3.4.2 Dynamicprogrammingandtheuseofstate-valuefunctions . . . . . . . . . 80
3.4.3 Approximatevaluefunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.4.4 Thepolicysearchapproach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4 TheAntColonyOptimizationMetaheuristic(ACO) 95
4.1 De(cid:2)nitionoftheACOmetaheuristic . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
4.1.1 Problemrepresentationandpheromonemodelexploitedbyants . . . . . . 98
4.1.1.1 Stategraphandsolutionfeasibility . . . . . . . . . . . . . . . . . . 98
4.1.1.2 Pheromonegraphandsolutionquality . . . . . . . . . . . . . . . . 99
4.1.2 Behavioroftheant-likeagents . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.1.3 Behaviorofthemetaheuristicatthelevelofthecolony . . . . . . . . . . . . 107
4.1.3.1 Schedulingoftheactions . . . . . . . . . . . . . . . . . . . . . . . . 108
4.1.3.2 Pheromonemanagement . . . . . . . . . . . . . . . . . . . . . . . . 110
4.1.3.3 Daemonactions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.2 AntSystem: the(cid:2)rstACOalgorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
4.3 DiscussionongeneralACO’scharacteristics . . . . . . . . . . . . . . . . . . . . . . 115
4.3.1 Optimizationbyusingmemoryandlearning . . . . . . . . . . . . . . . . . . 115
4.3.2 Strategiesforpheromoneupdating. . . . . . . . . . . . . . . . . . . . . . . . 122
4.3.3 Shortestpathsandimplicit/explicitsolutionevaluation . . . . . . . . . . . 126
4.4 Revisedde(cid:2)nitionsforthepheromonemodelandtheant-routingtable . . . . . . . 128
4.4.1 Limitsoftheoriginalde(cid:2)nition . . . . . . . . . . . . . . . . . . . . . . . . . . 128
4.4.2 Newde(cid:2)nitionstousemoreandbetterpheromoneinformation . . . . . . . 129
4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
5 ApplicationofACOtocombinatorialoptimizationproblems 137
5.1 ACOalgorithmsforproblemsthatcanbesolvedincentralizedway . . . . . . . . . 140
5.1.1 Travelingsalesmanproblems . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
5.1.2 Quadraticassignmentproblems . . . . . . . . . . . . . . . . . . . . . . . . . 147
5.1.3 Schedulingproblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
5.1.4 Vehicleroutingproblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151
5.1.5 Sequentialorderingproblems . . . . . . . . . . . . . . . . . . . . . . . . . . . 153
5.1.6 Shortestcommonsupersequenceproblems . . . . . . . . . . . . . . . . . . . 154
5.1.7 Graphcoloringandfrequencyassignmentproblems . . . . . . . . . . . . . 154
5.1.8 Binpackingandmulti-knapsackproblems . . . . . . . . . . . . . . . . . . . 156
5.1.9 Constraintsatisfactionproblems . . . . . . . . . . . . . . . . . . . . . . . . . 157
5.2 Parallelmodelsandimplementations . . . . . . . . . . . . . . . . . . . . . . . . . . 158
5.3 Relatedapproaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 166
II ApplicationofACOtoproblemsofadaptiveroutingin
telecommunicationnetworks 169
6 Routingintelecommunicationnetworks 171
6.1 Routing: De(cid:2)nitionandcharacteristics . . . . . . . . . . . . . . . . . . . . . . . . . . 172
6.2 Classi(cid:2)cationofroutingalgorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
6.2.1 Controlarchitecture: centralizedvs. distributed . . . . . . . . . . . . . . . . 175
6.2.2 Routingtables: staticvs. dynamic . . . . . . . . . . . . . . . . . . . . . . . . 175
CONTENTS ix
6.2.3 Optimizationcriteria: optimalvs. shortestpaths . . . . . . . . . . . . . . . . 177
6.2.4 Loaddistribution: singlevs. multiplepaths . . . . . . . . . . . . . . . . . . . 177
6.3 Metricsforperformanceevaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
6.4 Mainroutingparadigms: Optimalandshortestpathrouting . . . . . . . . . . . . . 182
6.4.1 Optimalrouting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182
6.4.2 Shortestpathrouting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.4.2.1 Distance-vectoralgorithms . . . . . . . . . . . . . . . . . . . . . . . 184
6.4.2.2 Link-statealgorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 187
6.4.3 Collectiveandindividualrationalityinoptimalandshortestpathrouting . 190
6.5 AnhistoricalglanceattheroutingontheInternet . . . . . . . . . . . . . . . . . . . 192
6.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 194
7 ACOalgorithmsforadaptiverouting 197
7.1 AntNet: traf(cid:2)c-adaptivemultipathroutingforbest-effortIPnetworks . . . . . . . 200
7.1.1 Thecommunicationnetworkmodel . . . . . . . . . . . . . . . . . . . . . . . 202
7.1.2 Datastructuresmaintainedatthenodes . . . . . . . . . . . . . . . . . . . . . 205
7.1.3 Descriptionofthealgorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.1.3.1 Proactiveantgeneration . . . . . . . . . . . . . . . . . . . . . . . . 208
7.1.3.2 Storinginformationduringtheforwardphase. . . . . . . . . . . . 210
7.1.3.3 Routingdecisionpolicyadoptedbyforwardants . . . . . . . . . . 210
7.1.3.4 Avoidingloops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212
7.1.3.5 Forwardantschangeintobackwardantsandretracethepath . . . 213
7.1.3.6 Updatingofroutingtablesandstatisticaltraf(cid:2)cmodels . . . . . . 214
7.1.3.7 Updatesofallthesub-pathscomposingtheforwardpath . . . . . 219
7.1.3.8 Acompleteexampleandpseudo-codedescription . . . . . . . . . 221
7.1.4 Acriticalissue: howtomeasuretherelativegoodnessofapath? . . . . . . 221
7.1.4.1 Constantreinforcements . . . . . . . . . . . . . . . . . . . . . . . . 223
7.1.4.2 Adaptivereinforcements . . . . . . . . . . . . . . . . . . . . . . . . 224
7.2 AntNet-FA:improvingAntNetusingfasterants . . . . . . . . . . . . . . . . . . . . 226
7.3 AntColonyRouting(ACR):aframeworkforautonomicnetworkrouting . . . . . 228
7.3.1 Thearchitecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 231
7.3.1.1 Nodemanagers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232
7.3.1.2 Activeperceptionsandeffectors . . . . . . . . . . . . . . . . . . . . 237
7.3.2 TwoadditionalexamplesofAntColonyRoutingalgorithms . . . . . . . . . 238
7.3.2.1 AntNet+SELA:QoSroutinginATMnetworks. . . . . . . . . . . . 239
7.3.2.2 AntHocNet: routinginmobileadhocnetworks . . . . . . . . . . . 242
7.4 Relatedworkonant-inspiredalgorithmsforrouting . . . . . . . . . . . . . . . . . . 245
7.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 254
8 ExperimentalresultsforACOroutingalgorithms 257
8.1 Experimentalsettings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
8.1.1 Topologyandphysicalpropertiesofthenetworks . . . . . . . . . . . . . . . 259
8.1.2 Traf(cid:2)cpatterns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
8.1.3 Performancemetrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262
8.2 Routingalgorithmsusedforcomparison . . . . . . . . . . . . . . . . . . . . . . . . 263
8.2.1 Parametervalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
8.3 ResultsforAntNetandAntNet-FA . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266
8.3.1 SimpleNet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
8.3.2 NSFNET . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
8.3.3 NTTnet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
x CONTENTS
8.3.4 6x6Net . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
8.3.5 Largerrandomlygeneratednetworks . . . . . . . . . . . . . . . . . . . . . . 274
8.3.6 Routingoverhead. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 276
8.3.7 SensitivityofAntNettotheantlaunchingrate . . . . . . . . . . . . . . . . . 277
8.3.8 Ef(cid:2)cacyofadaptivepathevaluationinAntNet . . . . . . . . . . . . . . . . . 278
8.4 ExperimentalsettingsandresultsforAntHocNet . . . . . . . . . . . . . . . . . . . . 278
8.5 Discussionoftheresults . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
8.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
III Conclusions 289
9 Conclusionsandfuturework 291
9.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
9.2 Generalconclusions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
9.3 Summaryofcontributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
9.4 Ideasforfuturework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 301
Appendices
A De(cid:2)nitionofmentionedcombinatorialproblems 305
B Modi(cid:2)cationmethodsandtheirrelationshipswithconstructionmethods 309
C ObservableandpartiallyobservableMarkovdecisionprocesses 313
C.1 Markovdecisionprocesses(MDP) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
C.2 PartiallyobservableMarkovdecisionprocesses(POMDP) . . . . . . . . . . . . . . 314
D MonteCarlostatisticalmethods 317
E Reinforcementlearning 319
F Classi(cid:2)cationoftelecommunicationnetworks 321
F.1 Transmissiontechnology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321
F.2 Switchingtechniques . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322
F.3 Layeredarchitectures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
F.4 Forwardingmechanisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 325
F.5 Deliveredservices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 327
Bibliography 328