Table Of Content

ABSTRACT Titleofdissertation: PLANNING UNDER UNCERTAINTY: MOVING FORWARD UgurKuter,DoctorofPhilosophy,2006 Dissertationdirectedby: ProfessorDanaNau DepartmentofComputerScience Reasoning about uncertainty is an essential component of many real-world planning problems, such as robotic and space applications, military operations planning, air and ground traffic control, and manufacturing systems. Planning under uncertainty focuses on how to generate plans that will be executed in environments where actions have nondeterministic effects (i.e., actions may have more than one possible outcome) and the states of the world are not always fully observable. The two predominant approaches for planning under uncertainty are based on Markov Decision Processes (MDPs) and Sym- bolic Model Checking. Despite the recent advances in these approaches, the problem of how to plan under uncertainty is still very hard: the planning algorithms must reason about more than one possible execution path in the world, and the sizes of the solution plans may grow exponentially. In planning environments that do not admit full observability,thecomplexityofplanningincreasesbyanadditionalexponentialfactorsincethe plannerdoesnotknowtheexactstatesoftheworld,andtherefore,itmustreasonoverthe setofallstatesthatitbelievestobein. Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number. 1. REPORT DATE 3. DATES COVERED 2006 2. REPORT TYPE 00-00-2006 to 00-00-2006 4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER Planning Under Uncertainty: Moving Forward 5b. GRANT NUMBER 5c. PROGRAM ELEMENT NUMBER 6. AUTHOR(S) 5d. PROJECT NUMBER 5e. TASK NUMBER 5f. WORK UNIT NUMBER 7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION University of Maryland,Department of Computer Science ,College REPORT NUMBER Park,MD,20742 9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S) 11. SPONSOR/MONITOR’S REPORT NUMBER(S) 12. DISTRIBUTION/AVAILABILITY STATEMENT Approved for public release; distribution unlimited 13. SUPPLEMENTARY NOTES 14. ABSTRACT Reasoning about uncertainty is an essential component of many real-world planning problems, such as robotic and space applications, military operations planning, air and ground traffic control, and manufacturing systems. Planning under uncertainty focuses on how to generate plans that will be executed in environments where actions have nondeterministic effects (i.e., actions may have more than one possible outcome) and the states of the world are not always fully observable. The two predominant approaches for planning under uncertainty are based on Markov Decision Processes (MDPs) and Symbolic Model Checking. Despite the recent advances in these approaches, the problem of how to plan under uncertainty is still very hard: the planning algorithms must reason about more than one possible execution path in the world, and the sizes of the solution plans may grow exponentially. In planning environments that do not admit full observability the complexity of planning increases by an additional exponential factor since the planner does not know the exact states of the world, and therefore, it must reason over the set of all states that it believes to be in. This dissertation describes a suite of new planning algorithms for planning under uncertainty with the assumption of full observability. The new algorithms are much more efficient than the previous techniques; in some cases, they find solutions exponentially faster than the previous ones. In particular, our contributions are as follows ? A method to take any forward-chaining classical planning algorithm, and systematically generalize it to work for planning in nondeterministic planning domains, where the likelihood of the possible outcomes of the actions are not known. In our experiments ND-SHOP2, a generalization of the Hierarchical Task Network (HTN) planner SHOP2 [NAI+03], could find solutions in nondeterministic planning domains about two to three orders of magnitude faster than MBP [BCP+01], which uses symbolic model-checking techniques based on Binary Decision Diagrams (BDDs) [Bry92], and which was one of the best previous planners for such domains. ? A way, called ?Forward State-Space Splitting (FS3),? to take the search control (i.e. pruning) technique of any forward-chaining classical planner, such as TLPlan [BK00] TALplanner [KD01], and SHOP2 [NAI+03], and combine it with BDDs. The result of this combination is a suite of new planning algorithms for nondeterministic planning domains. In our experiments, FS3 SHOP2, one of the new algorithms that combines HTNs as in ND-SHOP2 with BDDs as in MBP, was never dominated by either MBP or ND-SHOP2: FS3 SHOP2 could easily deal with problem sizes that neither MBP nor 15. SUBJECT TERMS 16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF ABSTRACT OF PAGES RESPONSIBLE PERSON a. REPORT b. ABSTRACT c. THIS PAGE Same as 198 unclassified unclassified unclassified Report (SAR) Standard Form 298 (Rev. 8-98) Prescribed by ANSI Std Z39-18 This dissertation describes a suite of new planning algorithms for planning under uncertaintywiththeassumptionoffullobservability. Thenewalgorithmsaremuchmore efficient than the previous techniques; in some cases, they find solutions exponentially fasterthanthepreviousones. Inparticular,ourcontributionsareasfollows: • A method to take any forward-chaining classical planning algorithm, and systematically generalize it to work for planning in nondeterministic planning domains, where the likelihood of the possible outcomes of the actions are not known. In our experiments,ND-SHOP2,ageneralizationoftheHierarchicalTaskNetwork(HTN)planner SHOP2 [NAI+03], could find solutions in nondeterministic planning domains about two to three orders of magnitude faster than MBP [BCP+01], which uses symbolic model-checking techniques based on Binary Decision Diagrams (BDDs) [Bry92], and whichwasoneofthebestpreviousplannersforsuchdomains. • A way, called “Forward State-Space Splitting (FS3),” to take the search control (i.e., pruning)techniqueofanyforward-chainingclassicalplanner,suchasTLPlan[BK00], TALplanner [KD01], and SHOP2 [NAI+03], and combine it with BDDs. The result ofthiscombinationisasuiteofnewplanningalgorithmsfornondeterministicplanning domains. Inourexperiments,FS3 ,oneofthenewalgorithmsthatcombinesHTNs SHOP2 as in ND-SHOP2 with BDDs as in MBP, was never dominated by either MBP or ND-SHOP2: FS3 could easily deal with problem sizes that neither MBP nor SHOP2 ND-SHOP2 could scale up to, and furthermore, it could solve problems about two or threeordersofmagnitudefasterthantheothertwo. • Awaytoincorporatethepruningtechniqueofaforward-chainingclassicalplannerinto the previous algorithms developed for planning with MDPs. The modified algorithms in our experiments were about 10,000 times faster than the original ones on the largest problems the original ones could solve. On another set of problems that were more than 14,000 times larger than the original algorithms could solve, the modified ones tookonlyabout1/3second. The new planning techniques described here have good potential to be applicable to other research areas as well. In particular, this dissertation describes such potentials in Reinforcement Learning, Hybrid Systems Control, and Planning with Temporal Uncer- tainty. Finally,theclosingremarksincludeadiscussiononthechallengesofusingsearch controlinplanningunderuncertaintyandsomepossiblewaystoaddressthosechallenges. PLANNING UNDER UNCERTAINTY: MOVING FORWARD by Ugur Kuter DissertationsubmittedtotheFacultyoftheGraduateSchoolofthe UniversityofMaryland,CollegeParkinpartialfulfillment oftherequirementsforthedegreeof DoctorofPhilosophy 2006 AdvisoryCommmittee: ProfessorDanaNau,Chair/Advisor ProfessorSteveMarcus,Dean’sRepresentative ProfessorJamesHendler ProfessorMichaelFu ProfessorAtifMemon (cid:13)c Copyright by Ugur Kuter 2006 Tomylove,Ferla—forpushingmeforwardinallthoseuncertaintimes ii ACKNOWLEDGMENTS First and foremost, I would like to express my gratitude and uncountable number of thanks to my advisor, Professor Dana Nau, for giving me an invaluable opportunity to work on challenging and extremely interesting projects over the past years. He took a stubborngraniteblockandcarveditouttowhatUgurKuteristoday,withhisnever-lasting patience,understanding,generosity,kindness,andcompassion. Heisatruemasterandit wasaprivilegetobeabletobehisapprenticeduringmydoctoraltraining. Theexperience Ihadunderhissupervisionwillguidemethroughoutmycareerandmylife. IexpressspecialthankstoProfessorsPaoloTraversoandMarcoPistoreforinviting me to Trento, Italy for an internship and for giving me the opportunity to work there on an important part of this dissertation. They have been unbelievably understanding to my mistakes and helpful in developing my ideas, which constitute an important portion of thisdissertation. I also thank to Professors Ju¨rgen Dix, Michael Fu, Steve Marcus, James Hendler, and John Lemmer, with whom I had the most-rewarding opportunity to work and learn fromthemregardingresearchandlife. Finally, I thank to my wife, Ferla, for always being there during all the difficult times of my life in the past 5-6 years. Her strength and her ability to tell me how to pick myself up whenever I fell down always made my life much easier than it would have been, and it will always be so. I also thank to my parents, Gu¨her and Figen Kuter, for iii theirnever-endingsupporttomeinpursuingmygoalsinlife. In the past years, this research has been supported in part by the following grants: NSF grant IIS0412812, Air Force Research Laboratory F30602-00-2-0505, Naval Re- searchLaboratoryN00173021G005,DARPA’sREALinitiative,ArmyResearchLabora- toryDAAL0197K0135,UniversityofMarylandInstituteofSystemsResearchseedfund- ing,andtheFIRB-MIURprojectRBNE0195k5,“KnowledgeLevelAutomatedSoftware Engineering.” The opinions expressed in this paper are those of the author and do not necessarilyreflecttheopinionsofthefunders. iv