Combining Self-Motivation with Planning and Inference in a Self-Motivated Cognitive Agent ...

145 Pages·2012·1.5 MB·English

Combining Self-Motivation with Planning
and Inference in a Self-Motivated
Cognitive Agent Framework
by
Daphne H. Liu
Submitted in Partial Fulfillment
of the
Requirements for the Degree
Doctor of Philosophy
Supervised by
Professor Lenhart Schubert
Department of Computer Science
Arts, Science & Engineering
Edmund A. Hajim School of Engineering & Applied Sciences
University of Rochester
Rochester, New York
2012
ii
To my family
iii
Biographical Sketch
Ms. Liu attended Simon Fraser University from 2000 to 2004, graduating with a
Bachelor of Science degree in Honors Computing Science, with a rank of 2nd of
130 in her class, and a minor in Business Administration. During her undergrad-
uate years, she completed an 8-month full-time internship as a software engineer,
and later worked as a research assistant in artificial intelligence with Prof. James
Delgrande. For her academic and research performance, she was awarded the Un-
dergraduate Research Award by the Natural Sciences and Engineering Research
Council of Canada (NSERC) as well as an Outstanding Undergraduate Award
Honorable Mention by the Computing Research Association (CRA). Also a re-
cipient of the Master’s Level Canada Graduate Scholarship by NSERC, Ms. Liu
went on to complete her M.Sc. thesis under the guidance of Prof. Delgrande,
and earned her first Master of Science degree in Computing Science from Simon
Fraser University in 2006. In the fall of 2006, she commenced her Ph.D. studies
in Computer Science at the University of Rochester, where she was awarded the
Robert L. Sproull Fellowship for the first two years. She pursued her research
in artificial intelligence under the direction of Prof. Lenhart Schubert. In 2008,
she obtained her second Master of Science degree in Computer Science from the
University of Rochester.
iv
List of Publications and Articles Submitted for Publication
• Daphne H. Liu and Lenhart Schubert. An Infrastructure for Self-Motivated,
Continually Planning Agents in Virtual Worlds. Technical Report 2012-985,
Dept. of Computer Science, University of Rochester, December 2012.
• Daphne Liu and Lenhart Schubert. Towards Self-Motivated, Cognitive,
Continually Planning Agents. Manuscript submitted in February 2012 for
publication (under review at Computational Intelligence).
• Daphne Liu and L. K. Schubert. Combining Self-Motivation with Logi-
cal Planning and Inference in a Reward Seeking Agent. In Proceedings of
the International Conference on Agents and Artificial Intelligence (ICAART
2010), vol. 2, Valencia, Spain, January 2010.
• Daphne Liu and Lenhart Schubert. Incorporating Planning and Reasoning
into a Self-Motivated, Communicative Agent. In Proceedings of the Second
Conference on Artificial General Intelligence (AGI 2009), Arlington, VA,
March 2009.
• Daphne Hao Liu. A Survey of Planning in Intelligent Agents: from Exter-
nally Motivated to Internally Motivated Systems. Technical Report 2008-
936, Dept. of Computer Science, University of Rochester, June 2008.
• Daphne Hao Liu. A Consistency-Based System for Knowledge Base Merg-
ing. Master of Science thesis, School of Computing Science, Simon Fraser
University, August 2006.
• J.Delgrande,D.H.Liu,T.SchaubandS.Thiele. COBA2.0: AConsistency-
Based Belief Change System. In Proceedings of the Eleventh International
Workshop on Non-Monotonic Reasoning (NMR 2006), June 2006.
v
• James Delgrande, Daphne H. Liu, Torsten Schaub and Sven Thiele. COBA
2.0: AConsistency-BasedBeliefChangeSystem. Technical Report TR 2005-
17, School of Computing Science, Simon Fraser University, November 2005.
Selected Honors and Awards
• Robert L. Sproull Fellowship, University of Rochester (2006 - 2008)
• Natural Sciences and Engineering Research Council of Canada (NSERC)
Canada Graduate Scholarship - Masters (2005 - 2006)
• ComputingResearchAssociation(CRA)OutstandingUndergraduateAward
Honorable Mention (2005)
• Natural Sciences and Engineering Research Council of Canada
Undergraduate Research Award (2004)
• The Governor General’s Academic Bronze Medal of Canada (2000)
vi
Acknowledgments
My sincere gratitude goes to my advisor, Prof. Lenhart Schubert, for his invalu-
ableinsights, guidanceandfeedbackonmywork. Lenisnotonlyameticulousand
extraordinarily knowledgeable advisor, but also a kind and encouraging person. I
have learned and benefitted considerably from many memorable discussions with
him over the years. Even under the pressure of impending deadlines and course-
teaching obligations, Len is very generous with his time and attention when it
comes to advising students. Len’s passion for research is inspiring, and his im-
peccable attitude towards research is worth emulating for one to work hard and
remain dedicated to one’s craft.
Iamalsoindebtedtomythesiscommitteeasawhole: Prof. LenhartSchubert,
Prof. James Allen, Prof. Greg Carlson, and Prof. Daniel Gildea. Thank you all
for your suggestions, feedback, and guidance throughout my PhD studies.
I appreciate the opportunity to have worked as an intern with Dr. Eric Bier
of the Palo Alto Research Center (PARC). In the summer of 2011, I worked in
the Knowledge, Language and Interaction group at PARC, on simulating human
work practices and identifying tasks that could be automated or streamlined with
tools. Eric’s wonderful mentorship, as well as the privilege to collaborate with a
capable and cordial group of coworkers, made for an rewarding and unforgettable
internship experience.
The prestigious Sproull Fellowship awarded by the University of Rochester,
together with my advisor’s National Science Foundation grants IIS-0535105, IIS-
vii
0916599, and IIS-1016735 and the Department of Defense/ONR grant N00014-
11-10417, have graciously supported me financially during my PhD studies. I also
acknowledge the entire URCS faculty and staff for their teaching and assistance,
respectively, over the years.
On a more personal level, I am thankful to the following people for their
friendship over the years in Rochester: Jerry Charipar, Mary Charipar, Xiao
Zhang, Xiaohua Zhang, Huiling Zhang, Yang Gao, Tongxin Bai, Xiaoming Gu,
Zhuan Chen, and Li Lu.
To my parents, brother, auntie Joyce and grandma, thank you for love and
encouragement. Your contribution to this dissertation cannot be quantified, but
it is forever etched in my heart and my memory.
Lastly, to my husband, Qi Ge, thank you for your patience, understanding
and support. As you and I both know, obtaining one’s PhD degree is an arduous
journey, but having you alongside me through this journey has certainly made
it immeasurably more enjoyable. I am blessed to have you in my life, and I am
tremendously grateful for the loving home we have built together.
viii
Abstract
Mostworkonself-motivatedagentsinArtificialIntelligencehasfocusedonacquir-
ing utility-optimizing mappings from states to actions. However, such mappings
do not allow for explicit, reasoned anticipation and planned achievement of fu-
ture states and pay-offs, based on symbolic knowledge about the environment
and about the consequences of the agent’s own behavior. In essence, such agents
can behave only reflexively, not reflectively. Conversely, planning and reasoning
have been viewed as aimed at fulfillment of explicitly specified user goals, without
regard for the long-range utility of the planner or reasoner’s choices. Moreover,
most work assumes either a completely known environment or a predictable en-
vironment altered only by the agent’s actions.
We strive to integrate self-motivation into logical planning and reasoning, in
a general self-motivated cognitive agent framework. Our self-motivated cognitive
agents are capable of thinking ahead and planning flexibly in an incompletely
known world, question-answering dialogues, mental modeling, introspection and
reasoning over a substantial knowledge base, all while in unceasing pursuit of
optimizing their own cumulative utility. While such agents employ reasoned ex-
ploration of feasible sequences of actions and corresponding states, they also be-
have opportunistically, seizing opportunities and recovering from failures, thanks
to their continual plan updates and quest for rewards. Our framework allows for
bothspecificandgeneral(quantified)knowledge, andforepistemicpredicatessuch
as knowing−that and knowing−whether. Since realistic agents often possess in-
ix
complete knowledge of their world, the reasoning of our agents uses a prudently
restricted version of the closed world assumption; this has consequences for epis-
temic reasoning, in particular positive and negative introspection. The planning
operators allow for quantitative, gradual change and side effects such as the pas-
sage of time, changes in distances and rewards, and language production, using a
uniform procedural attachment technique. Question-answering (involving intro-
spection and reasoning) and experimental runs are shown for our particular agent
ME in a simulated world, demonstrating the benefits of self-awareness as well as
continual, deliberate and reward-driven planning. Our agents can also be easily
configured as single or multiple goal-seeking agents, and as such perform quite
favorably compared with recent experimental agents.
x
Contributors and Funding Sources
The PhD research culminating in this dissertation was supervised by the author’s
dissertation committee: Prof. Lenhart Schubert of the Department of Computer
Science, Prof. James Allen of the Department of Computer Science, Prof. Greg
Carlson of the Department of Linguistics, and Prof. Daniel Gildea of the Depart-
ment of Computer Science, all at the University of Rochester.
Chapter 2 of this dissertation is organized around several systems the author
originally surveyed in (Liu, 2008). Chapters 3 through 5 describe work the author
collaboratively accomplished with her advisor, Prof. Lenhart Schubert. Parts of
Chapter 3 elaborate on some material originally published in (Liu and Schubert,
2010) and (Liu and Schubert, 2009). The contents of Chapter 4 have not yet been
published. Chapters 5 and 6 and Section 7.1, their contents partially done jointly
with Prof. Schubert and also appearing in the Computer Science Department
Technical Report 2012-985, have been submitted in a journal manuscript that is
currently under review.
This material is based upon work supported by the Sproull Fellowship of
the University of Rochester, the National Science Foundation grant IIS-0535105,
National Science Foundation grant IIS-0916599, National Foundation grant IIS-
1016735 and the Department of Defense/ONR grant N00014-11-10417. Any opin-
ions, findings, and conclusions or recommendations expressed in this material are
those of the author and do not necessarily reflect the views of above named orga-
nizations.