Decision Theory Models for Applications in Artificial Intelligence: Concepts and Solutions PDF
Preview Decision Theory Models for Applications in Artificial Intelligence: Concepts and Solutions
Decision Theory Models for Applications in Artificial Intelligence: Concepts and Solutions L. Enrique Sucar National Institute for Astrophysics, Optics and Electronics, Mexico Eduardo F. Morales National Institute for Astrophysics, Optics and Electronics, Mexico Jesse Hoey University of Waterloo, Canada Managing Director: Lindsay Johnston Senior Editorial Director: Heather Probst Book Production Manager: Sean Woznicki Development Manager: Joel Gamon Development Editor: Joel Gamon Acquisitions Editor: Erika Gallagher Typesetters: Mackenzie Snader Print Coordinator: Jamie Snavely Cover Design: Nick Newcomer, Greg Snader Published in the United States of America by Information Science Reference (an imprint of IGI Global) 701 E. Chocolate Avenue Hershey PA 17033 Tel: 717-533-8845 Fax: 717-533-8661 E-mail: [email protected] Web site: http://www.igi-global.com Copyright © 2012 by IGI Global. All rights reserved. No part of this publication may be reproduced, stored or distributed in any form or by any means, electronic or mechanical, including photocopying, without written permission from the publisher. Product or company names used in this set are for identification purposes only. Inclusion of the names of the products or companies does not indicate a claim of ownership by IGI Global of the trademark or registered trademark. Library of Congress Cataloging-in-Publication Data Decision theory models for applications in artificial intelligence: concepts and solutions / L. Enrique Sucar, Eduardo F. Morales and Jesse Hoey, editors. p. cm. Summary: “This book provides an introduction to different types of decision theory techniques, including MDPs, POM- DPs, Influence Diagrams, and Reinforcement Learning, and illustrates their application in artificial intelligence”-- Provided by publisher. Includes bibliographical references and index. ISBN 978-1-60960-165-2 (hardcover) -- ISBN 978-1-60960-167-6 (ebook) 1. Artificial intelligence--Statistical methods. 2. Bayesian statistical decision theory. I. Sucar, L. Enrique, 1957- II. Morales, Eduardo F. III. Hoey, Jesse. Q335.D43 2011 006.3--dc22 2010054421 British Cataloguing in Publication Data A Cataloguing in Publication record for this book is available from the British Library. All work contributed to this book is new, previously-unpublished material. The views expressed in this book are those of the authors, but not necessarily of the publisher. List of Reviewers Concha Bielza, Universidad Politécnica de Madrid, Spain Fabio Cozman, University of Sao Paulo, Brazil Boris Defourny, University of Liege, Belgium Javier Díez, Universidad Nacional de Educación a Distancia, Spain Pantelis Elinas, Australian Centre for Field Robotics, Australia Nando de Freitas, University of British Columbia, Canada Marcel van Gerven, Radboud University Nijmegen, Netherlands Phan H. Giang, George Mason University, USA Robby Goetschalckx, University of Waterloo, Canada Kevin Grant, University of Lethbridge, Canada Jesse Hoey, University of Waterloo, Canada Finn V. Jensen, Aalborg University, Denmark Omar Zia Khan, University of Waterloo, Canada Eduardo F. Morales, INAOE, Mexico Rubén Morales-Menendez, Tecnológico de Monterrey - Campus Monterrey, Mexico Abdel-Illah Mouaddib, University of Caen, France Kasia Muldner, University of British Columbia, Canada Julieta Noguez, Tecnológico de Monterrey - Campus Ciudad de México, Mexico Alberto Reyes, Instituto de Investigaciones Eléctricas, Mexico Claude Sammut, University of New South Wales, Australia Eugene Santos, Dartmouth College, USA Matthijs Spaan, Delft University of Technology, Netherlands L. Enrique Sucar, INAOE, Mexico Jason Williams, AT&T Research, USA Table of Contents Foreword ..............................................................................................................................................xii Preface .................................................................................................................................................xiii Acknowledgment ................................................................................................................................xiv Section 1 Fundamentals Chapter 1 Introduction .............................................................................................................................................1 L. Enrique Sucar, National Institute for Astrophysics, Optics and Electronics, Mexico Eduardo Morales, National Institute for Astrophysics, Optics and Electronics, Mexico Jesse Hoey, University of Waterloo, Canada Chapter 2 Introduction to Bayesian Networks and Influence Diagrams .................................................................9 Luis Enrique Sucar, National Institute for Astrophysics, Optics and Electronics, Mexico Chapter 3 An Introduction to Fully and Partially Observable Markov Decision Processes .................................33 Pascal Poupart, University of Waterloo, Canada Chapter 4 An Introduction to Reinforcement Learning .........................................................................................63 Eduardo F. Morales, National Institute for Astrophysics, Optics and Electronics, México Julio H. Zaragoza, National Institute for Astrophysics, Optics and Electronics, México Section 2 Concepts Chapter 5 Inference Strategies for Solving Semi-Markov Decision Processes .....................................................82 Matthew Hoffman, University of British Columbia, Canada Nando de Freitas, University of British Columbia, Canada Chapter 6 Multistage Stochastic Programming: A Scenario Tree Based Approach to Planning under Uncertainty ..................................................................................................................................97 Boris Defourny, University of Liège, Belgium Damien Ernst, University of Liège, Belgium Louis Wehenkel, University of Liège, Belgium Chapter 7 Automatically Generated Explanations for Markov Decision Processes ...........................................144 Omar Zia Khan, University of Waterloo, Canada Pascal Poupart, University of Waterloo, Canada James P. Black, University of Waterloo, Canada Chapter 8 Dynamic LIMIDS ...............................................................................................................................164 Francisco J. Díez, UNED, Spain Marcel A. J. van Gerven, Radboud University Nijmegen, The Netherlands Chapter 9 Relational Representations and Traces for Efficient Reinforcement Learning ...................................190 Eduardo F. Morales, National Institute for Astrophysics, Optics and Electronics, México Julio H. Zaragoza, National Institute for Astrophysics, Optics and Electronics, México Section 3 Solutions Chapter 10 A Decision-Theoretic Tutor for Analogical Problem Solving ............................................................219 Kasia Muldner, Arizona State University, USA Cristina Conati, University of British Columbia, Canada Chapter 11 Dynamic Decision Networks Applications in Active Learning Simulators ........................................248 Julieta Noguez, Tecnológico de Monterrey, Mexico Karla Muñoz, University of Ulster, Northern Ireland Luis Neri, Tecnológico de Monterrey, Mexico Víctor Robledo-Rella, Tecnológico de Monterrey, Mexico Gerardo Aguilar, Tecnológico de Monterrey, Mexico Chapter 12 An Intelligent Assistant for Power Plant Operation and Training Based on Decision- Theoretic Planning ..............................................................................................................................271 Alberto Reyes, Instituto de Investigaciones Eléctricas, México Francisco Elizalde, Instituto de Investigaciones Eléctricas, México Chapter 13 POMDP Models for Assistive Technology .........................................................................................294 Jesse Hoey, University of Waterloo, Canada Pascal Poupart, University of Waterloo, Canada Craig Boutilier, University of Toronto, Canada Alex Mihailidis, University of Toronto, Canada Chapter 14 A Case Study of Applying Decision Theory in the Real World: POMDPs and Spoken Dialog Systems ......................................................................................................................315 Jason D. Williams, AT&T Labs, USA Chapter 15 Task Coordination for Service Robots Based on Multiple Markov Decision Processes ....................343 Elva Corona, National Institute for Astrophysics, Optics and Electronics, Mexico L. Enrique Sucar, National Institute for Astrophysics, Optics and Electronics, Mexico Chapter 16 Applications of DEC-MDPs in Multi-Robot Systems ........................................................................361 Aurélie Beynier, University Pierre and Marie Curie, France Abdel-Illah Mouaddib, University of Caen, France Compilation of References ...............................................................................................................385 About the Contributors ....................................................................................................................417 Index ...................................................................................................................................................424 Detailed Table of Contents Foreword ..............................................................................................................................................xii Preface .................................................................................................................................................xiii Acknowledgment ................................................................................................................................xiv Section 1 Fundamentals Chapter 1 Introduction .............................................................................................................................................1 L. Enrique Sucar, National Institute for Astrophysics, Optics and Electronics, Mexico Eduardo Morales, National Institute for Astrophysics, Optics and Electronics, Mexico Jesse Hoey, University of Waterloo, Canada This chapter gives a general introduction to decision-theoretic models in artificial intelligence and an overview of the book. Chapter 2 Introduction to Bayesian Networks and Influence Diagrams .................................................................9 Luis Enrique Sucar, National Institute for Astrophysics, Optics and Electronics, Mexico This chapter covers the fundamentals of probabilistic graphical models, in particular: (i) Bayesian networks, (ii) Dynamic Bayesian networks and (iii) Influence diagrams. For each it describes the rep- resentation and main inference techniques. For Bayesian networks and dynamic Bayesian networks it includes an overview of structure and parameter learning. Chapter 3 An Introduction to Fully and Partially Observable Markov Decision Processes .................................33 Pascal Poupart, University of Waterloo, Canada This chapter provides a gentle introduction to Markov decision processes as a framework for sequential decision making under uncertainty. It reviews fully and partially observable Markov decision processes, describes basic algorithms to find good policies and discusses modeling and computational issues that arise in practice. Chapter 4 An Introduction to Reinforcement Learning .........................................................................................63 Eduardo F. Morales, National Institute for Astrophysics, Optics and Electronics, México Julio H. Zaragoza, National Institute for Astrophysics, Optics and Electronics, México This chapter provides a concise and updated introduction to Reinforcement Learning from a machine learning prespective. It gives the require background to undersand the chapters related to reinforcement learning in this book, and includes an overview of some of the latest trends in the area. Section 2 Concepts Chapter 5 Inference Strategies for Solving Semi-Markov Decision Processes .....................................................82 Matthew Hoffman, University of British Columbia, Canada Nando de Freitas, University of British Columbia, Canada Semi-Markov decision processes are used to formulate many control problems and play a key role in hierarchical reinforcement learning. This chapter shows how to translate the decision making problem into a form that can instead be solved by inference and learning techniques. It establishes a formal con- nection between planning in semi-MDPs and inference in probabilistic graphical models. Chapter 6 Multistage Stochastic Programming: A Scenario Tree Based Approach to Planning under Uncertainty ..................................................................................................................................97 Boris Defourny, University of Liège, Belgium Damien Ernst, University of Liège, Belgium Louis Wehenkel, University of Liège, Belgium This chapter presents the multistage stochastic programming framework for sequential decision making under uncertainty. It describes the standard technique for solving approximately multistage stochastic problems, which is based on a discretization of the disturbance space called scenario tree. It also shows how supervised learning techniques can be used to evaluate reliably the quality of an approximation. Chapter 7 Automatically Generated Explanations for Markov Decision Processes ...........................................144 Omar Zia Khan, University of Waterloo, Canada Pascal Poupart, University of Waterloo, Canada James P. Black, University of Waterloo, Canada It presents a technique to explain policies for factored MDPs by populating a set of domain-independent templates, and a mechanism to determine a minimal set of templates that, viewed together, completely justify the policy. The technique is demonstrated using the problems of advising undergraduate students in their course selection and assisting people with dementia in completing the task of handwashing. Chapter 8 Dynamic LIMIDS ...............................................................................................................................164 Francisco J. Díez, UNED, Spain Marcel A. J. van Gerven, Radboud University Nijmegen, The Netherlands Dynamic limited-memory influence diagrams (DLIMIDs) are a new type of decision-support model. Its main difference with other models is the restriction of limited memory, which means that the decision maker must make a choice based only on recent observations. This chapter presents several algorithms for evaluating DLIMIDs, shows a real-world model for a medical problem, and compares DLIMIDs with related formalisms. Chapter 9 Relational Representations and Traces for Efficient Reinforcement Learning ...................................190 Eduardo F. Morales, National Institute for Astrophysics, Optics and Electronics, México Julio H. Zaragoza, National Institute for Astrophysics, Optics and Electronics, México This chapter introduces an approach for reinforcement learning based on a relational representation. The underlying idea is to represent states as set of first order relations, actions in terms of these relations, and policies over those generalized representations. The effectiveness of the approach is tested on a flight simulator and on a mobile robot. Section 3 Solutions Chapter 10 A Decision-Theoretic Tutor for Analogical Problem Solving ............................................................219 Kasia Muldner, Arizona State University, USA Cristina Conati, University of British Columbia, Canada A decision-theoretic tutor that helps students learn from Analogical Problem Solving is described. The tutor incorporates an innovative example-selection mechanism that tailors the choice of example to a given student. An empirical evaluation shows that this selection mechanism is more effective than standard selection approaches for fostering learning. Chapter 11 Dynamic Decision Networks Applications in Active Learning Simulators ........................................248 Julieta Noguez, Tecnológico de Monterrey, Mexico Karla Muñoz, University of Ulster, Northern Ireland Luis Neri, Tecnológico de Monterrey, Mexico Víctor Robledo-Rella, Tecnológico de Monterrey, Mexico Gerardo Aguilar, Tecnológico de Monterrey, Mexico This chapter describes how an intelligent tutor based on Dynamic Decision Networks is applied in an undergraduate Physics scenario, where the aim is to adapt the learning experience to suit the learners’ needs. It employs Probabilistic Relational Models to facilitate the construction of the model. With this representation, the tutor can be easily adapted to different experiments, domains, and student levels.
Description:The list of books you might like
