Simulated Annealing Evolution

18 Pages·2012·0.54 MB·English

Chapter 10
Chapter0
Simulated Annealing Evolution
SergioLedesma,JoseRuizandGuadalupeGarcia
Additionalinformationisavailableattheendofthechapter
http://dx.doi.org/10.5772/50176
1.Introduction
Artificial intelligence (AI) is a branch of computer science that seeks to create intelligence.
Whilehumanshavebeenusingcomputerstosimplifyseveraltasks,AIprovidesnewoptions
tousecomputers.Forinstance,voicerecognitionsoftwareusesAItotransformthesoundsto
theequivalenttextwords. ThereareseveraltechniquesthatAIincludes. Anartificialneural
network(ANN)isoneofthesetechniques.
Humans usetheirintelligencetosolvecomplexproblemsandperformdailytasks. Human
intelligence is provided by the brain. Small processing units called neurons are the main
components of the human brain. ANNs try to imitate partially some of the human brain
behavior.Thus,artificialneuronsaredesignedtomimictheactivitiesofbiologicalneurons.
Humanslearnbyexperience:theyareexposedtoeventsthatencouragetheirbrainstoacquire
knowledge.Similarly,ANNsextractinformationfromadataset;thissetistypicallycalledthe
trainingsetandisorganizedinthesamewaythatschoolsdesigntheircourses’content.ANNs
provideanexcellentwaytounderstandbetterbiologicalneurons.Inpractice,someproblems
maybedescribedbyadataset.Forinstance,anANNistypicallytrainedusingadataset.For
someproblems,buildingadatasetmaybeverydifficultorsometimesimpossibleasthedata
sethastocaptureallpossiblecasesoftheexperiment.
Simulatedannealing(SA)isamethodthatcanbeusedtosolveanamplesetofoptimization
problems.SAisaveryrobusttechniqueasitisnotdeceivedwithlocalminima.Additionally,
amathematicalmodelisnotrequiredtoapplySAtosolvemostoptimizationproblems.
Thischapter exploresthe useofSAto trainanANNwithoutthe requirementofadataset.
ThechapterendswithacomputersimulationwhereanANNisusedtodriveacar. Figure1
showsthesystemarchitecture. SAisusedtoprovideanewsetofweightstotheANN.The
ANN controlsthe accelerationand rotationspeedofthe car. The car providesfeedbackby
sendingvisioninformationtotheANN.ThedistancetraveledalongtheroadfromtheStartis
usedbythemethodofSA.AtthebeginningofthesimulationtheANNdoesnotknowhow
to drive the car. As the experimentcontinues, SA is used to train the ANN. Each time the
©2012Ledesmaetal.,licenseeInTech.Thisisanopenaccesschapterdistributedunderthetermsofthe
CreativeCommonsAttributionLicense(http://creativecommons.org/licenses/by/3.0),whichpermits
unrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalworkisproperly
cited.
202 2Simulated Annealing – Advances, Applications and Hybridizations Will-be-set-by-IN-TECH
Newsetof
weights
Artificial
Neural Simulated
Network Annealing
Accelerationand
RotationSpeed
Vision Distance
Information fromtheStart
Car
Figure1.Systemarchitecture.
temperaturedecreases,the ANNimprovesits drivingskills. Bythe end ofthe experiment,
whenthetemperaturehasreacheditsfinalvalue,theANNandthecarhaveevolvedtothe
pointthattheycaneasilynavigateamaze.
2.Artificialneuralnetworks
AnANNisacomputational methodinspiredinbiologicalprocessestosolveproblemsthat
areverydifficultforcomputersorhumans. OneofthekeyfeaturesofANNsisthattheycan
adapttoabroadrangeofsituations. Theyaretypicallyusedwhereamathematicalequation
ormodelismissing,see[4]. ThepurposeofanANNistoextract, map, classifyoridentify
somesortofinformationthatisallegedlyhiddenintheinput,[13].
2.1.Neuron
Thehumanbrainiscomposedofprocessingunitscalledneurons. Eachneuronisconnected
to otherneuronstoformaneuralnetwork. Similarly,the basiccomponentsofanANNare
theneurons. NeuronsarearrangedinlayersinsidetheANN.Eachlayerhasafixednumber
ofneurons,see[5]. Forinstance,theANNshownintheFigure2hasthreelayers: theinput
layer,thehiddenlayer,andtheoutputlayer.
AnANNacceptsanykindofinputthatcanbeexpressedasasetofnumericvalues;typical
inputs may include: an image, a sound, a temperature value, etc. The output of an ANN
isalwaysdependentuponitsinput. Thatis,aspecificinputwillproduceaspecificoutput.
WhenasetofnumericvaluesisappliedtotheinputofanANN,theinformationflowsfrom
oneneurontoanotheruntiltheoutputlayergeneratesasetofvalues.
2.2.Activationfunction
The internal structure of anartificial neuronisshown inFigure3(a). The output value z is
givenby:
SimulatedAnnealingEvolution Simulated Annealing Evolution3 203
InputLayer HiddenLayer OutputLayer
1 1
Input
Neuron Neuron Output
Input
Neuron Neuron Output
Input
1 1
Figure2.Anartificialneuralnetwork.
z= f(y)= f(w +∑xw), (1)
b i i
i
where each w is called weight. A fixed input, known as Bias, is applied to the neuron, its
i
valueisalways1andw istherespectiveweightforthisinput. Theneuronincludesalsoan
b
activationfunctiondenotedby f(y)inFigure3(a).WithouttheBias,theoutputofthenetwork
wouldbe f(0)whenallinputsarezero. Onecommonactivationfunctionusedinmultilayer
ANNsis:
1
f(y)=logsig(y)= 1+e−y. (2)
x
Neuron
1 z=f(y)
w
1
1
w y
x2 . 2 (cid:2) f(y) z
.
.
w w
n b
x y
n Bias=1
(a) Neuronstructure (b) Activationfunction
Figure3.Artificialneuron.
TheactivationfunctionofEquation2isplottedinFigure3(b).Theactivationfunction(inthis
figure)isreal,continuous,limitedandhasapositivederivative.
Aneuroncanbeactiveorinactive,whentheneuronisactiveitsoutputis1,whenitisinactive
its output value is 0. Some input values activate some neurons, while other values may
activate other neurons. For instance, in Figure 4, the sound of the word Yes will activate
thefirstoutputneuron,whilethesoundofthewordNowillactivatethesecondneuronatthe
204 4Simulated Annealing – Advances, Applications and Hybridizations Will-be-set-by-IN-TECH
outputofthenetwork.ThestructureoftheANNshowninthisfigureisverysimplewithonly
twoneurons.
Input Input
Neuron 1 Neuron 0
Yes No
Input Input
Neuron 0 Neuron 1
Input Input
(a) ActivationwithYes (b) ActivationwithNo
Figure4.ANNactivation.
3.Learning
Beforean ANN can be used forany practical purpose, it mustbe trained. An ANN learns
duringitstraining. ForthedurationofthelearningprocesstheANNweightsarerecurrently
adjusted.
3.1.Structuredlearning
Insomeinstances,ANNsmaylearnfromadataset.Thissetistypicallyknownasthetraining
set,anditisusedonanewANN(asitsnameindicates)fortraining.Thetrainingsethastwo
parts: theinputandthetarget. Theinputcontainsthesetofinputsthatmustbeappliedto
thenetwork. ThetargetincludesthesetofdesiredvaluesattheoutputoftheANN.Inother
words, each sample (case) in the training set completely specifies all inputs, as well as the
outputsthataredesiredwhenthoseinputsarepresented,see[7]. Duringthetraining,each
caseinthetrainingsetispresentedtothenetwork,andtheoutputofthenetworkiscompared
withthedesiredoutput. Afterallcasesinthetrainingsethavebeenprocessed,anepochor
iterationhascompletedbyupdatingtheweightsofthenetwork. Thereareseveralmethods
forupdatingtheweightsateachepochoriteration. Allthesemethodsupdatetheweightsin
suchawaythattheerror(measuredbetweentheactualoutputofthenetworkandthedesired
output)isreducedateachepoch,see[7].
Sometrainingmethodsarebasedonthegradientoftheerror(theyarecalledgradientbased
methods). These methods quickly converge to the closest local minima. The most typical
gradient based methods to train an ANN are: the variable metric method, the conjugate
gradient method, and method of Levenberg-Marquardt. To make the training of an ANN
robust,itisalwaysrecommendedtocombinegradientbasedmethodswithotheroptimization
methodssuchasSA.
SimulatedAnnealingEvolution Simulated Annealing Evolution5 205
When an ANN is trained using a data set, typically the set includes many training cases,
and the trainingisdoneatonce. Thiskindoftrainingiscalledstructuredlearningbecause
the knowledge is organized in the data set for the network to learn. One of the main
disadvantages of structured learning is that the training set has to be prepared to describe
the problem at hand. Another disadvantage of structure learning is that if more cases are
added to the training set, the ANNhas again to be trained starting fromscratch. As ANN
trainingistimeconsuming,structuredlearningmaybeinadequateforproblemsthatrequire
continuousadaptation.
3.2.Continuouslearning
Incontinuouslearning,anANNdoesnotrequireadatasetfortraining,theANNlearnsby
experienceandisabletoadaptprogressivelybyincorporatingknowledgegradually.Because
someproblemscannotbeappropriatelydescribedbyadataset,andbecausetrainingusing
adatasetcanbetimeconsuming,continuouslearningisimportantforreal-timecomputing
(RTC)wherethereisa"real-timeconstraint".
3.3.Validation
After the ANNtraining has been completed, the network performancehas to be validated.
When using ANNs, the validation process is extremely important, as a matter of fact, the
validationisasimportantasthetraining,see[7]. Thepurposeofthevalidationistopredict
howwelltheANNwillbehaveinotherconditionsinthefuture. Thevalidationprocessmay
be performedusing a data set called the validation set. The validation set is similar to the
trainingsetbutnotequal.Undernormalcircumstances(whentheANNisproperlyused),the
errorobtainedduringtrainingandduringvalidationshouldbesimilar.
4.Simulatedannealing
SA is an optimization method that can be used to solve a broad range of problems, [11].
SAisrecommendedforcomplexoptimizationproblems. Thealgorithmbeginsataspecific
temperature;astimepassesthetemperaturegraduallydecreasesfollowingacoolingschedule
as shown in Figure 5. The solution is typically described by a set of variables, but it can
be described by other means. Once the algorithm has started, the solution approaches
progressivelytheglobalminimumthatpresumablyexistsinacomplexerrorsurface,see[16]
and [15]. Because of its great robustness, SA has been used in many fields including the
trainingofANNswithstructuredlearning,[9].
OneofthekeyfeaturesofSAisthatitalwaysprovidesasolution,eventhoughthesolution
maynotbeoptimal.Forsomeoptimizationproblemsthatcannotbeeasilymodeled,SAmay
provideapracticaloptiontosolvethem.
5.Simulatedannealingevolution
Simulated annealing evolution includes the use of: ANNs, continuous learning and SA. In
simulated annealing evolution, an ANN does not require a training set; instead the ANN
206 6Simulated Annealing – Advances, Applications and Hybridizations Will-be-set-by-IN-TECH
Temperature
Time
Figure5.Cooling:threeANNslearning.
graduallylearnsnewskillsorimprovesexistingonesbyexperience. Figure5showshowSA
evolutionworks. Inthisfigure,atypicalcoolingscheduleusedinSAisdisplayed. Suppose
thatthereisa2Dlandscapewithvalleysandhillsasshowninthisfigure. Supposealsothat
it is desired to find the deepest valley on this landscape. Each of the balls, in this figure,
representsanANN.Atthebeginningofthesimulation,thehighinitialtemperatureproduces
astateofhighenergy;theballsshakepowerfullyandareabletotraverseeasilythroughthe
high hills ofthe 2D terrain. In other words, each ANN is exploring,that is, the ANNis in
the initial stepoflearning. Asthe temperaturedecreases,the energyofthe balls decreases,
and the movement of the balls is more restricted than at high temperatures, see [1]. Thus,
asthetemperaturediminishes,theANNhaslessfreedomtoexplorenewinformationasthe
network has to integrate new knowledgewith the previousone. By the end of the cooling
schedule, it is desired that one of the balls reached the deepest valley in the landscape, in
otherwords,thatoneANNlearnedasetofskills.
Ateachtemperature,anANN(intheset)hasthechance touseitsknowledgetoperforma
specifictask. Asthe temperaturedecreases,eachANNhasthe chance toimproveitsskills.
IftheANNsarerequiredtoincorporatenewskills,temperaturecyclingcanbeused,see[6].
Specifically,anANNmaylearnbyacombinationofSAandsomesortofhands-onexperience.
Thus,simulatedannealingevolutionisthetrainingofanANNusingSAwithoutatraining
set.
EachANNmayberepresentedbyasetofcoefficientsorweights. Forillustrativepurposes
only,supposethatanANNmaybedescribedbyasetoftwoweightsw andw .InFigure6,
11 12
there are three individuals, each is represented by a small circle with its two weights: w
11
and w . At every temperature, each ANN is able to exploreand use its abilities. At high
12
temperatures, each ANN has limited skills and most of the ANNs will perform poorly at
the required tasks. The gray big shadow circle in the figure indicates how the ANNs are
consideringdifferentsetofvaluesw andw fortestingandforlearning. Astemperature
11 12
SimulatedAnnealingEvolution Simulated Annealing Evolution7 207
decreases, the ANNs get closer to each other, illustrating the fact that most of them have
learnedasimilarsetofskills.Bythetimethetemperaturehasreacheditsfinalvalue,theskills
ofeachANNwillbehelpfultothedegreethatthenetworkisabletoperformthetaskathand.
At this moment, the simulationmayend orthe temperaturemayincrease, ifthere arenew
skillsthatneedtobeincorporated.
Temperature
w
11
w
12
w
11 w
11
w
12 w
12
Time
Figure6.EachANNtriestolearnthesamesetofskills.
6.Problemdescription
Toillustratehowtousesimulatedannealingevolution,thissectionpresentsasimplelearning
problem. TheproblemconsistsofusinganANNtodriveacarinasimpleroad. Clearly,the
problemhas twoobjects: the roadand thecar. TheroadincludesaStartpointand aFinish
pointasshowninFigure7. Theroadincludesseveralstraightlinesandturningpoints. The
carisinitiallyplacedattheStart. ThedrivingisperformedbyanANNthatwasintegrated
withthecar. Specifically,theANNmanipulatesindirectlyseveralparametersinthecarsuch
asposition,speedanddirection. ThepurposeofthesimulationistotraintheANNwithout
atrainingset. Atthebeginningofthesimulation,theANNdoesnotknowhowtodrive;as
theevolutioncontinuestheANNlearnsandimprovesitsskillsbeingitsgoaltodrivethecar
quicklyfromtheStarttotheFinishwithouthittingtheboundsoftheroad(thatis,thecarmust
alwaysstayinsidetheroad). Tosolvethisproblemsimulatedannealingevolutionwasused
totraintheANN.
The simulation was performed using object oriented programming (OOP); the respective
UML diagrams for the simulation are shown in Figures8 and 9. The two basis classes are
showninthediagramofFigure8.Thisdiagramincludestwoclasses:PointandBounds.The
Pointclassinthediagramrepresentsapointina2-Dimensionalspace,thediagramindicates
thatthisstructureincludesonlytwofloatingpointvariables:xandy.TheBoundsclass,inthe
sameUMLdiagram,isusedtodescribetheboundsofanobjectwhentheobjectisatdifferent
positionsandrotationsangles. ThemainpurposeoftheBoundsclassistodetectcollisions
whenoneobjectmovesaroundotherobjects.
208 8Simulated Annealing – Advances, Applications and Hybridizations Will-be-set-by-IN-TECH
Finish
B
A
s
Start
Figure7.Thecarandtheroad.
Point Bounds
+x:double +points:Point[]
+y:double
+Intersect(bounds:Bounds):bool
Figure8.UMLdiagramshowingthebasicclasses.
6.1.Theobjectclass
Figure9showstherespectiveUMLdiagramfortheObjectclass.Thisclassrepresentsastatic
objectina2-Dimensionalspace.Theclassnameisdisplayedinitalicsindicatingthatthisclass
isabstract.AsitcanbeseentheRender()methodisdisplayedinitalics,andhence,itisavirtual
method and must be implemented by the non-abstract derived classes. If the experiment
includes some sort of visualization, the Render() method may be used to perform custom
drawing. Therearemanycomputertechnologiesthatcanbeusedtoperformdrawing,some
of them are: Microsoft DirectX, Microsoft GDI, Microsoft GDI+, Java Swing and OpenGL.
FromtheUMLdiagramofFigure9,itcanbeseenthateachobjecthasaposition,arotation
angle(theta)andasetofbounds.ThemethodIsCollision()takesanotherobjectoftypeObject
andreturnstruewhentheboundsofoneobjectcollidewiththeboundsofanotherobject.This
methodwasimplementedusingasimpleversionofthealgorithmpresentedby[14].
6.2.Themobileclass
This class represents a moving object in a 2-Dimensional space. Each Mobile object has a
speed,anaccelerationandarotationspeedasshownintheUMLdiagramofFigure9. This
class is derived directlyfrom the Object class, and therefore, is also abstract as the Render()
methodisnotimplemented. TheUpdateBounds()methodforthisclasstakesthenumberof
secondsatwhichtheobjectboundswillbecomputed.Thismethodisextremelyusefulwhen
movinganobjectaroundotherobjects,forinstance,iftheboundsofoneobjectintersectwith
SimulatedAnnealingEvolution Simulated Annealing Evolution9 209
Object
+position:Point
+theta:double
#bounds:Bounds
+Render()
+IsCollision(object:Object):bool
Mobile Road
+speed:double
+Render()
+acceleration:double
+rotationSpeed:double
+UpdateBounds(sec:double)
Car
+neuralNetwork:ANN
+Render()
+RunANN(road:Road)
+GetDistance(road:Road):double
Figure9.UMLdiagramshowingtheCarandtheRoadclasses.
the bounds of another object, the object cannot move, this is implemented internally in the
methodIsCollision(). ToupdatetheboundsofaMobileobject, thespeedoftheobjectmay
becomputedusingEquation3,
v =v +at, (3)
2 1
where v isthe initialvelocityofthe object, v isitsfinal velocity, a isthe acceleration, and
1 2
t stands for the time for which the object moved. Similarly, the position of the object can
be updated using another Kinematic equation. To compute the new position of the object
Equation4canbeused,
1
d=v t+ at2, (4)
1 2
wherethesymboldstandsforthedisplacementoftheobject. Inmostcases,theobjectmoves
along a line described by its rotation angle and its position. Thus d has to be accordingly
projectedinthecoordinatesystemasshownbyEquations5and6,
1
x =x +v tcosθ+ at2cosθ, (5)
2 1 1
2
1
y =y +v tsinθ+ at2sinθ. (6)
2 1 1 2
Insomecases, the objectmayberotatingataconstant speedand, hence, therotationangle
hasalsotobeupdatedateachperiodoftime.
210 1S0imulated Annealing – Advances, Applications and Hybridizations Will-be-set-by-IN-TECH
Duringthe experiments,themethodsUpdateBounds()andIsCollision()oftheMobileclass
are used together to prevent object collision. First, the simulator calls the UpdateBounds()
method to compute the bounds of the object at some specific time, and then will call the
IsCollision()methodtocheckforpotentialcollisionswithotherobjects.
6.3.Theroad
Thesimulationexperimentswereperformedusingonlytwoclasses: theRoadclassandthe
Carclass.TheUMLdiagramoftheRoadclassisshowninFigure9.TheRoadclassisderived
directlyfromtheObjectclass;themethodRender()isimplementedtodrawtheroaddisplayed
inFigure7usingMicrosoftGDIandScalableVectorGraphics(SVG).Whenthecarleavesthe
◦
Start,theANNhastomakeitsfirstrightturnat90 ,asthecarisjustacceleratingthisturnis
◦
easy. Thefollowingturnisalsototherightat90 andtheANNshouldnothaveanytrouble
makingthisturn.Then,iftheANNwantstodrivethecartopointA,ithastomaketwoturns
totheleft.
The straight segment from A to B should be easy to drive; unfortunately, the ANN may
continually accelerate, and reach point B at a very high speed. The turn at point B is the
◦
mostdifficultofalltheturns,becausethecarhastomakearightturnat90 athighspeed.
Becausethesimulationisoverwhenthecarhitstheboundsoftheroad,assoonastheANN
canseetheturnofpointB,ithastostartreducingthespeedofthecar. OncetheANNhas
managedtodrivetopointB,reachingtheFinishshouldnotbedifficult.
6.4.Thecar
ThecarusedforthesimulationisshowninFigure10. Thecarhasapositionrepresentedby
xandyinFigure10;thecarrotationisrepresentedbyθ. Thespeedandaccelerationvectors
are representedby v and a respectively. The arrow next to the rotation speed in Figure 10
indicates that the car is capable of turning. The car has several variables to store its state
(position,theta,speed,acceleration,rotationSpeedandneuralNetwork)asshownintheUML
diagramofFigure9. TheCarclassderivesdirectlyfromtheMobileclassandimplementsthe
methodRender()todrawthecarofFigure10usingMicrosoftGDIandSVG.Whenv=0and
a = 0the valuesof x and y donot change. When v (cid:3)= 0and a = 0, the values of x and y
willchangewhilevremainsconstant. Whena (cid:3)= 0,thevaluesofx,yandvwillchange. The
method GetDistance() computes the distance that the car has traveledalong the roadfrom
theStart,thisdistanceisrepresentedbysinFigure7.
Figure11 illustrateshow the car is able to receiveinformationabout its surroundings. The
carhadsevenvisionpointsillustratedbythearrowsinthefigure. TopreventtheANNfrom
drivingbackwards, novisionlineswereplacedinthe back ofthe car. Eachvalueof d , d ,
1 2
... d , representsthedistance fromthe centerofthecar tothe closestobjectinthedirection
7
of the vision line. These values were computed using the bounds of the road. To create a
moreinterestingenvironmentfortheANN,thevaluesofd ,d ,... d werecomputedatlow
1 2 7
resolutionandthecarcouldnotseeobjectslocatedawayfromit.
In reallife, acardriverisnotable tomodifydirectlythe positionorvelocityof the car, the
driveronlycontrolstheaccelerationandtheturningspeed.Asmentionbefore,eachcarinthe