applied sciences Article Modified Adversarial Hierarchical Task Network Planning in Real-Time Strategy Games LinSun,PengJiao,KaiXu,QuanjunYin*andYabingZha CollegeofInformationSystemandManagement,NationalUniversityofDefenseTechnology,Changsha410073, China;[email protected](L.S.);[email protected](P.J.);[email protected](K.X.); [email protected](Y.Z.) * Correspondence:[email protected];Tel.:+86-0731-8457-4590 Received:31July2017;Accepted:20August2017;Published:25August2017 Abstract: Theapplicationofartificialintelligence(AI)toreal-timestrategy(RTS)gamesincludes considerablechallengesduetotheverylargestatespacesandbranchingfactors,limiteddecision times,anddynamicadversarialenvironmentsinvolved. Toaddressthesechallenges,hierarchical tasknetwork(HTN)planninghasbeenextendedtodevelopamethoddenotedasadversarialHTN (AHTN), and this method has achieved favorable performance. However, the HTN description employed cannot express complex relationships among tasks and accommodate the impacts of the environment on tasks. Moreover, AHTN cannot address task failures during plan execution. Therefore, this paper proposes a modified AHTN planning algorithm with failed task repair functionalitydenotedasAHTN-R.ThealgorithmextendstheHTNdescriptionbyintroducingthree additionalelements: essentialtask,phase,andexitcondition. Ifanytaskfailsduringplanexecution, theAHTN-RalgorithmidentifiesandterminatesallaffectedtasksaccordingtotheextendedHTN description, and applies a novel task repair strategy based on a prioritized listing of alternative plans to maintain the validity of the previous plan. In the planning process, AHTN-R generates the priorities of alternative plans by sorting all nodes of the game search tree according to their primaryfeatures. Finally,empiricalresultsarepresentedbasedonaµRTSgame,andtheperformance ofAHTN-RiscomparedtothatofAHTNandtotheperformancesofotherstate-of-the-artsearch algorithmsdevelopedforRTSgames. Keywords: HTNplanning;real-timestrategygame;taskrepair 1. Introduction The application of artificial intelligence (AI) to real-time strategy (RTS) games includes considerable challenges due to the very large state spaces and branching factors, limited decision times,anddynamicadversarialenvironmentsinvolved[1]. RTSgamesareregardedasasimplification ofreal-lifeenvironments,andcouldthereforeserveasatestbedforinvestigatingactivitiessuchas real-timeadversarialplanninganddecisionmakingunderuncertainty[2]. Moreover,taskplanning techniquesthathavebeendemonstratedtobeeffectiveinRTSgamescouldalsobeappliedinreal-world domains[2]. Comparedwithconventionalboardgames,RTSgameshavethefollowingprimarydifferences[2]: 1. Playerscanpursueactionssimultaneouslywiththeactionsofotherplayers,andneednottake turns, asingamessuchaschess. Moreover, playeractionscanbeconductedoververyshort decisiontimes,allowingforrapidsequencesofactions. 2. Playerscanpursueconcurrentactionsemployingmultiplecontrollableunits. Thisismuchmore complexthanconventionalboardgames,whereonlyasingleactionisperformedwitheachturn. Appl.Sci.2017,7,872;doi:10.3390/app7090872 www.mdpi.com/journal/applsci Appl.Sci.2017,7,872 2of24 3. Playeractionsaredurative,inthatanactionrequiresnumeroustimestepstobeexecuted. This isalsomuchmorecomplexthanconventionalboardgames,whereactionsmustbecompleted withinasingleturn. 4. Thestatespaceandbranchfactorsaretypicallyverylarge.Forexample,atypical128×128mapin “StarCraft”generallyincludesabout400playercontrollableunits. Consideringonlythelocation ofeachunit, thenumberofpossiblestatesisabout101685, whereasthestatespaceofchessis typicallyestimatedtobearound1050. Ofcourse,evenlargervaluesareobtainedwhenincluding theotherfactorsintheRTSgame. 5. TheenvironmentofanRTSgameisdynamic. Unlikeinconventionalboardgamessuchaschess, wherethefundamentalnaturesoftheboardandgamepiecesneverchange,theenvironment ofanRTSgamemaychangedrasticallyinresponsetoplayerdecisions,whichcaninvalidatea generatedplan. Becauseofthedifferencesdiscussedabove,standardgametreesearchmethods,whichperform wellforboardgameslikealpha–betasearch[3],cannotbedirectlyappliedinRTSgames[4]. However, research has beenconducted to modify game tree searchmethods to addressthese differences [4]. Chungetal.[5]investigatedtheapplicabilityofgametreeMonteCarlosimulationsinRTSgames. BallaandFern[6]appliedtheupperconfidenceboundfortrees(UCT)algorithminanRTSgame to address the complications associated with durative actions. The UCT algorithm is a variant of MonteCarlotreesearch(MCTS)thatmodifiesthestrategyofexploringchildnodes. Themethodis believedtoadaptautomaticallytotheeffectivesmoothnessofthetree. Churchilletal.[7]addressed thecomplicationsassociatedwithsimultaneousanddurativeactionsbyextendingthealpha–beta search process. However, the large branching factors caused by independently acting objects still remainedasubstantialchallenge. Methodssuchascombinatorialmulti-armedbanditshaveattempted toaddressthechallengeoflargebranchingfactors[8,9]. OntañónandBuro[10]addressedverylarge statespaceandbranchfactorsbycombiningthehierarchicaltasknetwork(HTN)planningapproach withgametreesearchtodevelopwhatwasdenotedasadversarialHTN(AHTN).Here,ratherthan exploringtheentirecombinationofpossibleactions,HTNplanningcouldguidethesearchdirection basedondomainknowledge. AHTNalsoextendedtheHTNplanningapproachtoaccommodate simultaneousanddurativeactions,andappliedHTNplanningdirectlyintothegamesearchtreeto takeadvantageofstandardoptimizationssuchasalpha–betapruning. AlthoughAHTNhasachievedgoodperformancecomparedtoothersearchalgorithms[10],itstill suffersfromthefollowingweaknesses. 1. TheHTNdescriptionusedbyAHTNcannotexpresscomplexrelationshipsamongtasksand accommodatetheimpactoftheenvironmentontasks. Forexample,considerataskinvolvingthe forcedoccupationofafortifiedenemyemplacement,denotedasthecapture-blockhousetask,which consistsoftwosubtasks. Thefirstsubtaskinvolvesluringtheenemyawayfromtheemplacement (denotedastheluring-enemysubtask),andthesecondsubtaskinvolvesattackingtheemplacement (denotedastheattackingsubtask). Here,thecapture-blockhousetaskwillfailiftheattackingsubtask fails. However,failureoftheluring-enemysubtaskwouldnotnecessarilyleadtoanoverallfailure ofthecapture-blockhousetask,butwouldrathertendtoincreasethecostofcompletingtheparent taskbecausetheattackingsubtaskcanstillbeexecutedeventhoughtheluring-enemysubtaskfails. ThistypeofrelationcannotbeexpressedbytheHTNdescriptionemployedinAHTN.Relations relatedtoconditionswheresubtasksaretriggeredbytheenvironmentorwheretheexecution resultsofaparenttaskdependonboththeenvironmentandthesuccessfulcompletionofits subtasksalsocannotbeexpressedbytheHTNdescriptionemployedinAHTN. 2. AHTN planning cannot effectively address task failures that occur during plan execution. The planning process in AHTN generates a new plan at each frame of the game whenever idle units which are not assigned or execute actions exist. The new plan processing cannot cancelassignedactionsofthepreviousplanning. Ifunitshavebeenassignedactionsinprevious Appl.Sci.2017,7,872 3of24 planning, they will keep executing those actions until those actions are failed or completed. InAHTN,eachfailedtaskofthepreviousplanissimplyremovedatthetimeoffailurewithout consideringitsimpactsonremainingexecutingtasks.Theremainingtaskswillcontinueexecuting thoughtheseexecutionsmaybemeaningless. Toillustratethispoint,wenotethat,ifthefailure ofataskleadstothefailureofitsplandirectly,theremainingexecutingtasksoftheplanshould notbecontinuedexecutionbecausetheycannotaffectthefailureoftheplan. Theremaining executingtasksshouldthereforebeterminatedsothatthereleasedresourcescanbeemployedto repairfailedtasksorformulateanewplan. Thus,whenonetaskfailsandcannotberepaired, allrelatedtasksintheplanshouldbeterminated,andtheAIplayershouldattempttorepairthe tasktomaintainthevalidityoftheoriginalplan. In this paper, we propose a modified AHTN algorithm with failed task repair functionality, denotedasAHTN-R,toaddressthetwoproblemsdiscussedabove. First,AHTN-RemploysanHTN description extended by adding the elements essential task, phase, and exit condition to enhance its capability for expressing complex relationships among tasks and for accommodating the impacts oftheenvironment. Second,AHTN-RemploysamonitoringstrategybasedontheextendedHTN description to identify all tasks affected by a failed task. Finally, we develop a novel strategy for repairingfailedtasksbasedonaprioritizedlistofalternativeplans. Theprioritiesofalternativeplans aregeneratedbysortingallnodesofthegamesearchtreeaccordingtotheirprimaryfeatures,andwe employ the valid alternative plan with the highest priority to repair the failed task. This method canreducetimeconsumptionbytakingadvantageofhistoricalinformation. Ineachdecisioncycle, themethodattemptstorepairfailedtasks,followedbyanewplanningprocess. The remainder of this paper is organized as follows. Related work is presented in Section 2. TheextendedHTNdescriptionispresentedinSection3.TheAHTN-Rplanningalgorithmispresented inSection4. ExperimentalresultsindicativeoftheperformanceoftheproposedAHTN-Ralgorithm arecomparedwiththeperformancesofotheralgorithmsinSection5. Finally,weconcludethepaper andpresentanindicationoffutureworkinSection6. 2. RelatedWork SincethefoundationofHTNplanningwasfirstproposedbySacerdoti[11]in1975,manyHTN plannershavebeenproposed,suchasthesimplehierarchicalorderedplanner(SHOP)[12]andits successorSHOP2[13]. CombiningHTNplanningwithsearchmethodsguidedbyhumanknowledge has been demonstrated to speed up planning dramatically, and this has led some researchers to attempt to apply this approach in typical video games [14]. Menif et al. [15] applied the Simple Hierarchical Planning Engine (SHPE) to Steam Box, which is a type of first person shooter game, usinganalternativeencodingofplanningdata. SoemersandWinands[16]investigatedthereuse ofHTNplansinvideogames. NumerousexamplesexistofthesuccessfuluseofHTNplannersin commercialvideogames[17]. However,tothebestofourknowledge,onlyafewattemptshavebeen madetoapplyHTNplannerstoRTSgames. Muñoz-AvilaandAha[18]employedanHTNplannerto providehumanplayerswithexplanationsforthereasonsleadingtocurrentstatesoreventsaccording touserqueries,andforinterrogatingthebehaviorofautonomousplayersundercomputercontrol. Laagland[19]presentedthedesign,implementation,andevaluationofanHTNplannerinanopen sourceRTSgamedenotedasSpring. TheplanningoftheRTSgamewasdividedintothreelevelswith theHTNplannerbeingemployedinthehigheststrategiclevel. Laagland[20]furthersummarizedthe advantagesanddisadvantagesofHTNplanninginRTSgames. Naveedetal.[21]employedHTN plannerstoreducethesizeofthepathfindingsearchspaceinRTSgames,andtestedtheiralgorithmin gamesdevelopedusingORTSin2010. Mostrecentlyin2015,OntañónandBuro[10]developedthe AHTNalgorithmandappliedittoRTSgames. Of the above studies, we note that no research other than that of Ontañón and Buro [10] has consideredtheadversarialfeatureofRTSgames. Moreover,noresearchhasadequatelyaddressedtask failuresthatoccurduringplanexecutioninRTSgames. However,studiesfocusedinotherfieldshave Appl.Sci.2017,7,872 4of24 addressedtaskrepairbasedonHTN.Garzónetal.[22]proposedataskrepairstrategybasedonHTN planningtoadapttofailuresduringtheexecutionoftreatmentplansgeneratedbytherapyplanning systems. Thestrategyemployedthreemethodsforeffectingtaskrepair: applicationoftheknowledge base,searchofalternativedecompositions,andoperatornotification. Gateauetal.[23]employedHTN torepairfailedtaskstoaslocalizedanextentaspossibleinahigh-leveldistributedarchitecturefor multi-robotcooperation. Thecoreconceptofthestrategywastore-planonlyasub-treeoftheplan basedonahierarchicalstructure. Ayanetal.[24]designedanextensionofSHOPtorepairfailedtasks byintroducingatask-dependencygraph. Here,onlythatportionoftheoriginalplanrelatedtothe failedtaskwassubjectedtore-planning. Thetask-dependencygraphwasemployedtorecordcausal linksinthetasknetwork,andwasthenappliedtoeliminatetheeffectsofre-planningontheother portionsoftheoriginalplan,andtoreducethecostofre-planning. Theapproachappliedre-planning onlytotheaffectedpartsofthetasknetworkusingHTN.However,manyoftheseapproachesfailed totakeadvantageoftheinformationgeneratedinthepreviousplanningprocesstoreducethecostof taskrepair. 3. ExtendedHTNDescription ThissectionpresentsanextendedHTNdescriptiontoexpresscomplexrelationshipsamongtasks andaccommodatetheimpactsoftheenvironmentontasks. Wefirstanalyzetherequirementsfor extendingtheHTNdescription,andthenpresentaformaldefinitionoftheextendedHTNdescription. 3.1. RequirementsAnalysis The HTN planning approach generates plans by decomposing tasks recursively into smaller subtasks [25]. HTN planning includes two types of tasks: primitive tasks and compound tasks. Primitivetaskscorrespondtoactionsthatcanbeexecutedbyanagentdirectlyinthegameenvironment, suchasaspecificmovement. Compoundtasksrepresentgoalsthatmustbeachievedbyahigh-order plananddecomposedintoatasknetwork. 3.1.1. EssentialTaskAttribute InAHTNplanning,theresultofacompoundtaskdependsontheresultsandlogicalrelationships ofitssubtasks. Itstaskdecompositiontreeisanand-or-treewithlogicalrelationshipsbetweensubtasks thatareeitherANDorOR.TheANDrelationshipindicatesthatthefailureofanysubtaskwillleadtoa failureofitsparenttask. TheORrelationshipindicatesthatthesuccessofanysubtaskwillleadtothe successofitsparenttask.However,somedomainknowledgeregardingthecompletionofataskcannot berepresentedbyanand-or-tree. Returningtothecapture-blockhousetaskexamplediscussedinthe introduction,wenotethatthesuccessoftheluring-enemysubtaskmaydecreasethecostofcompleting theparenttask,butitsfailurewouldnotleadtothefailureofitsparenttask. However,thesuccess andfailureofthecapture-blockhousetaskiscompletelydependentonthesuccessandfailureofthe attackingsubtask. Thesemorecomplexlogicalrelationshipscannotberepresentedbytheformationof anand-or-tree. Torepresenttaskdecompositionreflectingmorecomplexlogicalrelationships,weintroducean attributedenotedasessentialtask,whosevalueiseithertrueorfalse,andthisattributeisappliedto alltasksoftheHTN.Here,thefailureofanyessentialtaskwillautomaticallyleadtothefailureofits parenttaskwithoutconsiderationfortheresultsofanyothersubtasks. Fortwosubtaskst andt , 1 2 theirlogicalrelationshipsarerepresentedasfollows: 1. iftheessentialtaskattributesofbotharetrue,therelationshipbetweent andt isAND; 1 2 2. iftheessentialtaskattributesofbotharefalse,therelationshipbetweent andt isOR; 1 2 3. iftheessentialtaskattributeofonlyoneistrue,therelationshipisneitherANDnorOR. Appl.Sci.2017,7,872 5of24 3.1.2. PhaseandExitConditionAttributes Theresultofacompoundtasknotonlydependsontherelationshipsandresultsofitssubtasks, butsometimesalsodependsontheenvironment,whichcanchangedynamically. Theseimpactsfrom theenvironmentcanbeofthefollowingtwotypes. 1. The environment can control the start of tasks once previous tasks have been completed. For example, the task associated with the sudden concealed attack on an enemy, denoted as the ambushing-enemy task, consists of two subtasks: the first is associated with movement to thestagingareaoftheambuscade,denotedasmove-to-ambushing-area,andtheactualactivity of attacking, denoted as attack-enemy. It is obvious that the attack-enemy subtask could not be executed immediately upon completing the move-to-ambushing-area subtask because the attack-enemytaskcannotbeexecuteduntiltheenemyresidesintheambuscadearea. 2. Theenvironmentcouldnecessitatethefailureofatask. Fortheambushing-enemytaskexample, the move-to-ambushing-area subtask must be completed before the enemy has passed the ambuscadestagingarea;otherwise,theambushing-enemytaskwouldfail. WhiletheAHTNalgorithmcanaccommodatethefirsttypeofimpactusingthepreconditionsof tasks,thesecondtypeofenvironmentalimpactcannotbeaddressed. Inthispaper,weproposetheexit conditionattributetoaddressthesetwotypesofenvironmentalimpacts. Theexitconditionattributecan bedividedintosufficientexitconditionandnecessaryexitconditionattributes. Ifataskhassomenecessary exitconditions,itssubsequenttaskcannotbeexecuteduntilallnecessaryexitconditionsaresatisfied. Ifataskhassomesufficientexitconditions,itsparenttaskwillfailwhenanysufficientexitconditionis satisfiedbutthetaskisnotcompleted. TheexitconditionattributeiswrittenusingaLisp-likesyntax likethepreconditiondefinedin[13]. Thephaseattributeisproposedtodescribetherelationshipsbetweensubtasksthatreflectthefact thattheexecutionofacompoundtaskcanbedividedintoseveralsequentialsteps,andeachstepmay includeparallelsubtasks. Becausetasksinthesamephaseareoftenaffectedbythesameexitconditions, weaddthesetofexitconditionstothephaseratherthanaddingthemtoeachtask. For an example, we return to the ambushing-enemy task, where we have modified the task to consistofthreesubtasks: move-to-ambushing-area,surveil-enemy,andattack-enemy. Themodification includesthesubtasksurveil-enemyreflectingtherequirementtofirstdeterminethattheenemyresides in the ambuscade area prior to attacking. The first two subtasks are executed in parallel using differentunitsandtheattack-enemycanbeexecutedonlywhenthefirsttwosubtasksarecompleted. Wedividetheexecutionofambushing-enemyintotwophases. Thesubtasksmove-to-ambushing-areaand surveil-enemyareincludedinthefirstphase,whichisdenotedasthepreparatory-phase. Theattack-enemy subtaskisincludedinthesecondphase,denotedastheattack-phase. Thepreparatory-phasehasboth sufficient and necessary exit conditions, where the ambushing-enemy task will fail if the subtasks of the preparatory-phase have not been completed by the point at which the enemy has passed the ambushing-area (i.e., the sufficient exit condition is satisfied), and the attack-enemy task will not be initiatedifthesubtasksofthepreparatory-phasehavebeencompletedwhiletheenemyhasnotpassed theambushing-area(i.e.,thenecessaryexitconditionisnotsatisfied). Thedatastructuresemployedin theexampleareshowninFigure1,wheretheunitsdenotedas?unitid1and?unitid2willambushthe enemydenotedas?enemyattheambushing-areadenotedas?place. Inthepreparatory-phase,?unitid1 is allocated to move to ?place and ?unitid2 surveils ?enemy. If the preparatory-phase is completed, ?unitid1willattack?enemy. Additionaldefinitionsareprovidedinthefollowingsubsections. Appl.Sci.2017,7,872 6of24 Appl. Sci. 2017, 7, 872 6 of 25 FFiigguurree 11.. AAnn eexxaammppllee ooff tthhee aammbbuusshhiinngg--eenneemmyy ttaasskk.. Including the phase attribute during the decomposition of a compound task is the main Includingthephaseattributeduringthedecompositionofacompoundtaskisthemaindifference difference between the AHTN used in this paper and that in [10]. The definition of the phase attribute betweentheAHTNusedinthispaperandthatin[10]. Thedefinitionofthephaseattributeisasfollows. is as follows. DDeeffiinniittiioonn 1 (1p.h ase()p.haTseh)e: phaTsehea ttribpuhtaeseis defiantterdibuaste a tuipsl e: d(cid:104)neafimnee,d methaosd , suaf exitctounpsleet:, 〈n(cid:1866)e(cid:1853)ce(cid:1865)x(cid:1857)it,c(cid:1865)on(cid:1857)(cid:1872)sℎet(cid:1867),(cid:1856)t,a(cid:1871)s(cid:1873)k(cid:1858)se(cid:1857)t(cid:1876)(cid:105)(cid:1861),(cid:1872)w(cid:1855)(cid:1867)h(cid:1866)er(cid:1871)e(cid:1857)t(cid:1872)h,(cid:1866)e(cid:1857)fo(cid:1855)l(cid:1857)lo(cid:1876)w(cid:1861)(cid:1872)i(cid:1855)n(cid:1867)g(cid:1866)d(cid:1871)e(cid:1857)fi(cid:1872)n,(cid:1872)it(cid:1853)i(cid:1871)on(cid:1863)s(cid:1871)(cid:1857)a(cid:1872)p〉p, lwy.here the following definitions apply. •• (cid:1866)n(cid:1853)am(cid:1865)e(cid:1857) isist htheen anmameoef othfe thphea pshe.ase. •• (cid:1865)m(cid:1857)et(cid:1872)hℎo(cid:1867)d(cid:1856)c ocnosnisstisstosf othfe thnaem neaomfeth oefm tehteh omdetothwohdi ctho twhehpichhas tehbee lpohnagssea bnedloanligsst oafnpda raa mlisett eorsf fpoarrtahmepehteasres. • fsourf tehxeit pcohnassee.t isthesetofsufficientexitconditions. Eachelementisalogicexpressionofliteralswhich • (cid:1871)co(cid:1873)n(cid:1858)s(cid:1857)i(cid:1876)st(cid:1861)(cid:1872)o(cid:1855)f(cid:1867)t(cid:1866)he(cid:1871)(cid:1857)n(cid:1872)a mise tahned saetli sotf osfupffaircaiemnett eexrsit. conditions. Each element is a logic expression of literals • wnehciecxhit ccoonnssiestti osft hthees entaomfnee acensdsa ar yliesxt iotfc ponadraitmioentse.rEs.a chelementisalogicexpressionofliteralswhich • (cid:1866)co(cid:1857)n(cid:1855)s(cid:1857)i(cid:1876)st(cid:1861)(cid:1872)o(cid:1855)f(cid:1867)(cid:1866)th(cid:1871)e(cid:1857)(cid:1872)n aims ethaen sdeta olifs nteocfespsaarraym eextietr cso.nHdietrioe,nss.u Ebsaecqhu eenletmpheanste iss caa nlobgeice exxecpurteesdsioonnl yofw lhiteenraallsl wnehceicshsa rcyonexsiitstc oonfd tihtieo nnsaamrees aatnisdfi ead .list of parameters. Here, subsequent phases can be executed • otanslkys ewthisetnh ealsle nteocfessusbatrays ekxsitth caotnsdhiotuiolndsb aerceo msaptliestfeidedto. accomplishacompoundtask. Thesubtaskscanbe • (cid:1872)(cid:1853)(cid:1871)(cid:1863)(cid:1871)(cid:1857)(cid:1872) is the set of subtasks that should be completed to accomplish a compound task. The eithercompoundtasksorprimitivetasks. Eachelementintasksetconsistsofthesubtask’snameandalist subtasks can be either compound tasks or primitive tasks. Each element in (cid:1872)(cid:1853)(cid:1871)(cid:1863)(cid:1871)(cid:1857)(cid:1872) consists of ofparameters. the subtask’s name and a list of parameters. Because a phase is executed sequentially, any failure of a phase will lead to the failure of the Because a phase is executed sequentially, any failure of a phase will lead to the failure of the methodassociatedwiththatphase. Theresultofaphasedependsontwofactors: tasksandsufficientexit method associated with that phase. The result of a phase depends on two factors: tasks and sufficient conditions. Accordingly,thefollowingfourconditionsleadtothefailureofaphase: exit conditions. Accordingly, the following four conditions lead to the failure of a phase: 1. Anyessentialtaskofthephasehasfailed. 1. Any essential task of the phase has failed. 2. Anysufficientexitconditionissatisfied,withoneormoreessentialtasksstillexecuting. 2. Any sufficient exit condition is satisfied, with one or more essential tasks still executing. 3. Anysufficientexitconditionissatisfied,andnosubtaskshavebeencompleted. 3. Any sufficient exit condition is satisfied, and no subtasks have been completed. 44.. AAllll ssuubbttaasskkss hhaavvee ffaaiilleedd.. FFoorr 11,, tthhee ffaaiilluurree ooff aannyy eesssseennttiiaall ttaasskk mmeeaannss tthhaatt tthhee ppaarreenntt ttaasskk hhaass ffaaiilleedd,, aanndd iittss pphhaassee iiss aallssoo ccoonnssiiddeerreedd aass ffaaiilleedd.. FFoorr2 2,,u unncocommpplelteetdede sessesnetniatliatla staksskcsa cnannontocto ncotinntuineueex eecxuetcinugtinwgh wenhaensu affi scuiefnfitcieexnitt ecxointd ciotinodnitiisosna itsis sfiaetdis,faiendd, athnedp tahree nptatraesnkt atalssok faalislos. fIanilasd. Idni taiodnd,iaticocno,r dacincogrtdoin3g,t thoe 3p,h tahsee wphiallsefa wiliwll hfaeinl wnohesunb ntaos ksusbatraesckosm arpel ectoedmapnledteads uafnfidci ean stuefxfiitciceonnt deixtiiotn coisndsaittiiosnfi eisd .saFtiinsafilelyd,.a Ftiansaklloyb, vai otausskly ofbavilisoiufsalyll fiatsilssu ibf taalsl kitss hsauvbetafsakilse dha.ve failed. 3.2. Definition of the Extended HTN Description Appl.Sci.2017,7,872 7of24 3.2. DefinitionoftheExtendedHTNDescription AnHTNproblemcanbedefinedasatuple:(cid:104)S, M, O, TN, γ(cid:105),wherethefollowingdefinitionsapply: • S is the set of current world states, and consists of all information that is relevant to the planningprocess. • Misthesetoftaskdecompositionmethods. Eachmethodm ∈ Mcanbeappliedtodecomposea taskintoasetofsubtasks. • Oisthesetofoperators. Eachoperatoro ∈Oisanexecutionofaprimitivetask. • TNisthecurrenttasknetwork. Itisatreewhosenodesaretasks,methods,orphases. • γisthestatetransformfunction. Givens ∈ S,γ(s,o)definesthetransitionofthestatewhenan primitivetaskisexecutedbyanagent. Ifγ(s,o),theoperatoroisnotapplicableins. The definition of a method in the proposed extended description is similar to the definition employedinpreviousHTNdescriptions. However,ourmethodusesthelistofphasesratherthansets ofsubtasks,andaddstheattributedenotedasessential. Definition2(method). Amethodisdefinedasatuple: (cid:104)name, task, precon, phases, essential(cid:105),wherethe followingdefinitionsapply. • nameisthenameofthemethod. • taskisthenameofthetasktowhichthemethodisapplied. • preconisthelogicalpreconditionsforthemethod,whichshouldbesatisfiedwhenthemethodisapplied. • phasesisthelistofphases. Thesequenceoftheelementsin phasesistheexecutionsequence. • essentialisaBooleanvalue,andistruetoindicatethatatasktowhichthemethodisappliedisanessentialtask. ThedefinitionofanoperatorisalsosimilarwiththatinpreviousHTNdescriptionsexceptfor addingtheessentialattribute. Definition3(operator). Theoperatorisdefinedasatuple: (cid:104)head, cons, dels, adds, essential(cid:105),wherethe followingdefinitionsapply. • headistheprimitivetaskthatcanbeappliedbythisoperator. • consisthepreconditionthatmustbesatisfiedbeforetaskexecution. • delsarethedeleteeffects. • addsaretheaddeffects. • essentialisaBooleanvalue,andistruetoindicatethatthetasktowhichtheoperatorisappliedisanessentialtask. 4. AHTN-RPlanningAlgorithm TheoverallframeworkoftheAHTN-Ralgorithmisprovidedfirstly,andthefunctionsofitsdifferent componentsareexplained.Then,themonitoringstrategyemployingtheextendedHTNdescriptionis proposedtoaddresstheissueoffailedtasks.Finally,wediscusstheproposedfailedtaskrepairstrategy. 4.1. ATHNRFramework The overall framework of the AHTN-R algorithm is illustrated in Figure 2. The process flow consists of three main components: plan generation, plan execution, and task repair. The plan generation component provides the best plan to the plan execution component, and provides a prioritized list of alternative plans to the task repair component. The plan execution component processesthefailedtasksresponsibleforplanfailureandsendsthetaskneedingrepairtothetask repaircomponent. Thetaskrepaircomponentattemptstorepairfailedtasksatthebeginningofeach decisioncycle. Appl. Sci. 2017, 7, 872 8 of 25 4.1. ATHNR Framework The overall framework of the AHTN-R algorithm is illustrated in Figure 2. The process flow consists of three main components: plan generation, plan execution, and task repair. The plan generation component provides the best plan to the plan execution component, and provides a prioritized list of alternative plans to the task repair component. The plan execution component processes the failed tasks responsible for plan failure and sends the task needing repair to the task repair component. The task repair component attempts to repair failed tasks at the beginning of each Appl.Sci.2017,7,872 8of24 decision cycle. Figure 2. Overall framework of the adversarial hierarchical task network (AHTN)-R algorithm. Figure2.Overallframeworkoftheadversarialhierarchicaltasknetwork(AHTN)-Ralgorithm. At each decision cycle, when there are idle units, the plan generation component will search the best plan using the AHTN algorithm with the extended HTN description. This paper modified the Ateachdecisioncycle,whenthereareidleunits,theplangenerationcomponentwillsearchthe part of the AHTN algorithm obtaining the next action. In AHTN, the subtasks can be obtained if a bestplanusimnegthtohde isA aHppTlieNd. aHlogwoervitehr, mwitwh tihthe itnhtreodeuxctteionnd oef dphaHseT inN AdHeTsNc-rRi,p thtieo pnh.asTehs wisilpl baep oebrtaminoedd ifiedthepart oftheAHTNratahlegr othrainth thme suobbttaaskins iifn ag mtehtheodn ies xatppalcietdio. Tnh.e IAnHATNH-RT aNlg,otrhithems uacbcetassseks sthce agnenebreatoedb ptahainsees difamethod to obtain the next primitive task. Algorithm 1 shows the method employed to obtain the next isapplied.However,withtheintroductionofphaseinAHTN-R,thephaseswillbeobtainedratherthan executable action based on the HTN plan N and execution pointer t, which keeps track of which parts thesubtasksofi fNa hmavee tahlroedadyis baepenp elxieecdu.teTdh. TehAe rHesTulNt o-fR Aalglogroitrhimth 1m is aac pcreimssiteivset htaeskg oern aenr eamtepdty pseht,a sestoobtain thenextpriminditiicvateintga asnk .abAselngcoe roift hanm av1aislahbolew prsimthiteivem taestkh. oLdineesm 1–p4 lsohyowed thtaot, oifb ntoa tiansktsh heavnee yxett ebxeeenc utableaction executed, the algorithm returns the first primitive task in N. Lines 7–19 show that, if all tasks in the basedontheHTNplanNandexecutionpointert,whichkeepstrackofwhichpartsofNhavealready same phase with t have been executed, the algorithm searches the first primitive task from the next beenexecutpehda.seT. Ihf enor epsriumlittivoef tAasklgs oarrei tfohumnd 1ini sthae nperxit mtasikti, vane etmaspkty osert aisn reetumrnpedty. Lsineets, 2in0–d31ic sahtoiwn ganabsence ofanavailabthlaet pthrei amlgiotriivthemt saesakrc.hLesi nfoer sa p1r–im4istihveo twaskt hfroamt, tihfen reomtaainsiknsg thaaskvse iny theet sbaemeen pheaxsee cthuatte hdav,et healgorithm not been executed. returnsthefirstprimitivetaskinN.Lines7–19showthat,ifalltasksinthesamephasewiththave beenexecute d,thealgorithmsearches thefirstprimitivetaskfromthenextphase. Ifnoprimitivetasks arefoundinthenexttask,anemptysetisreturned. Lines20–31showthatthealgorithmsearchesfora primitivetaskfromtheremainingtasksinthesamephasethathavenotbeenexecuted. Algorithm1ReturnsnextAction(N,t) 1. Ift=⊥then 2. GettherootofN,denotedastherootTask 3. ReturnnextAction(N,rootTask) 4. EndIf 5. Acquirethephaseptowhichtbelongs 6. Getalltasksdenotedassubtasksofp 7. Ifalltasksinsubtaskshavebeenexecutedthen 8. Getthenextphasep(cid:48)ofp 9. Ifp(cid:48)=⊥then 10. Return(cid:11) 11. EndIf 12. Getalltasksdenotedassubtasks(cid:48)ofp(cid:48) 13. Forallt(cid:48)∈subtasks(cid:48)do 14. IfnextAction(N,t(cid:48))(cid:54)=(cid:11)then 15. ReturnnextAction(N,t(cid:48)) 16. EndIf 17. EndFor 18. Return(cid:11) 19. EndIf 20. Forallt(cid:48)(cid:48) ∈subtasksthathavenotbeenexecuteddo 21. Ift(cid:48)isaprimitivetaskthen 22. Returnt(cid:48)(cid:48) 23. EndIf 24. Getthefirstphasep(cid:48)(cid:48) ofthemethodappliedtot(cid:48)(cid:48) 25. Getalltasksdenotedassubtasks(cid:48)(cid:48) ofp(cid:48)(cid:48) 26. Forallt(cid:48)(cid:48)(cid:48) ∈subtasks(cid:48)(cid:48) do 27. IfnextAction(N,t(cid:48)(cid:48)(cid:48))(cid:54)=(cid:11)then 28. ReturnnextAction(N,t(cid:48)(cid:48)(cid:48)) 29. EndIf 30. EndFor 31. EndFor 32. Return(cid:11) Appl.Sci.2017,7,872 9of24 Once the best plan is generated, the AHTN-R will obtain the prioritized listing of alternative plansgeneratedbyAHTNthroughsortingthenodesofthegamesearchtree. Themethodofsorting plansisdiscussedinSection4.3. Duringplanexecution,thetaskmayfailbecauseofthedynamics oftheRTSgameenvironment. Onceataskfails,AHTN-Rwillprocessthefailedtaskaccordingto themonitoringstrategydiscussedinSection4.2,whichseekstoidentifyandterminatethesmallest possiblesetoftasksaffectedbythefailedtask. Inthisway,theeffectsofafailedtaskcanbelimitedto thesmallestscopepossible. At the beginning of each decision cycle, AHTN-R first attempts to repair the failed task by selectingasuitableplanfromthelistofalternativeplansaccordingtotheirpriorities. AHTN-Rwill thenexecutethetasksofthealternativeplancorrespondingtothetasksaffectedbythefailedtask. IfAHTN-Risunabletofindasuitablealternativeplan,itwillgenerateanaltogethernewplanforthe failedtaskbyAHTN.ThisprocessisdiscussedattheendofSection4.3. 4.2. FailedTaskMonitoringStrategyUsingtheExtendedHTNDescription Onceataskfails,anAIplayermustidentifyandterminatealltasksaffectedbythefailedtask. Toaccomplishthis,thesourcesoftheplanfailureareanalyzedfirst. Planfailurecanbeattributed totwosources: thefailureofactionsandthestateofthesufficientexitconditionsattribute. Anaction willfailwhenitsunitisdestroyedorwhenavailableresourcescannotbeobtainedtofulfilltheaction. Aplanmayalsofailifthesufficientexitconditionissatisfied,asdiscussedinSection3.1.2. Taskfailure triggersthemonitoringstrategy. Algorithm2showsthemainprocessofthestrategyemployedto monitorfailedtasks. TheinputofAlgorithm2isthefailedtaskt,andtheresultofAlgorithm2isalistdenotedas RepairTaskList,whichcontainsthetasksrequiringrepair. Lines1–4showthedifferentstrategiesfor addressingessentialandunessentialtasks. Lines5–15showhowfailedcompoundtasksareaddressed. Lines7–13showthat,ifafailedcompoundtaskhasavailablemethods,itwillbesavedandrepairedin thenextdecisioncycle. Onlywhenataskhasnoavailablemethodswillitbeconsideredasthefailed task. Inthisway,thenumberofaffectedtaskswillbeassmallaspossible,whichwillreducetheextent ofrepairrequired. Line15showsthestrategyforaddressingprimitiveandcompoundtaskswithout availablemethods. Algorithm2ReturnTaskFailed(t) 1. IfEssential(t)= falsethen 2. SubTaskFailed(t) 3. Return 4. EndIf 5. Iftisacompoundtaskthen 6. Acquireallmethodsmoft 7. Foreachmethodm∈ M∧status(m)!=failed 8. GetthepreconditionsetCofm 9. Ifallc∈Cissatisfiedthen 10. AddtintoRepairTaskList 11. Return 12. Endif 13. EndFor 14. Endif 15. SubTaskFailed(t) According to the extended HTN description, each task, except the root node task, belongs to a single phase, and whether the phase corresponding to a failed task has also failed must be determined. Algorithm3showsthestrategyforaddressingthecorrespondingphaseaccordingto theHTNdescriptionpresentedinSection3.2. TheinputofAlgorithm3isataskt. Line2showsthat, Appl.Sci.2017,7,872 10of24 ifthefailedtaskistherootnodetask,itneednotberepaired. Ifthephaseisfailed,Algorithm4and Algorithm5arecalledtoterminatetheaffectedtasksandprocesscompoundtasks. Thefailedphase servesastheinputofAlgorithms4and5. Algorithm4showsthestrategyforprocessingthefailed phase. Ifthetaskbelongingtothefailedphaseisacompoundtask,itssubtaskswillbeprocessed,and, ifitisaprimitivetask,itwillbeterminateddirectly. Algorithm5showsthestrategyforprocessingthe methodtowhichthefailedphasebelongs. Lines2–6showthatallunfinishedphasesofthemethod arefailedbecausetheyaresequential. Algorithm3SubTaskFailed(t) 1. Acquirethephasepcorrespondingtot 2. Ifp=⊥thenreturn 3. EndIf 4. Ifoneofthesufficientconditionsofpismetthen 5. Ifanyessentialtasksofparefailedthen 6. PhaseFailed(p); 7. MethodFailed(p); 8. ElseIfphasnoessentialtask 9. ∧(alltasksofpwereexecutedordidnotstart) 10. PhaseFailed(p); 11. MethodFailed(p); 12. EndIf 13. EndIf 14. ElseIfanyessentialtasksofpfail∧ 15. (phasnoessentialtask∧alltasksofpfail) 16. PhaseFailed(p); 17. MethodFailed(p); 18. EndIf 19. EndIf Ifanysufficientexitconditionissatisfied,theAIplayerwilldeterminewhetherthephasetowhich the sufficient exit condition belongs has failed. The processing involved is similar to that given by Algorithm3exceptthatitsinputparameterisasufficientexitconditionratherthanafailedtask. Algorithm4PhaseFailed(p) 1. Getalltasksdenotedassubtasksofp 2. Foralltaskst∈subtasks 3. Ifstatus(t)= finished∨status(t)= failedthen 4. continue; 5. EndIf 6. Iftisacompoundtaskthen 7. Getthemethodmappliedtot 8. GetallphaseswithinPhasesofm 9. Forallphasesq∈Phases 10. PhaseFailed(q) 11. EndFor 12. Else 13. Cancel(t) 14. EndIf 15. EndFor
Description: