Algorithms and architectures of artificial intelligence

185 Pages·2007·2.48 MB·English

by  E. Tyugu

ALGORITHMS AND ARCHITECTURES OF ARTIFICIAL
INTELLIGENCE
Frontiers in Artificial Intelligence and
Applications
FAIA covers all aspects of theoretical and applied artificial intelligence research in the form of
monographs, doctoral dissertations, textbooks, handbooks and proceedings volumes. The FAIA
series contains several sub-series, including “Information Modelling and Knowledge Bases” and
“Knowledge-Based Intelligent Engineering Systems”. It also includes the biennial ECAI, the
European Conference on Artificial Intelligence, proceedings volumes, and other ECCAI – the
European Coordinating Committee on Artificial Intelligence – sponsored publications. An
editorial panel of internationally well-known scholars is appointed to provide a high quality
selection.
Series Editors:
J. Breuker, R. Dieng-Kuntz, N. Guarino, J.N. Kok, J. Liu, R. López de Mántaras,
R. Mizoguchi, M. Musen and N. Zhong
Volume 159
Recently published in this series
Vol. 158. R. Luckin et al. (Eds.), Artificial Intelligence in Education – Building Technology
Rich Learning Contexts That Work
Vol. 157. B. Goertzel and P. Wang (Eds.), Advances in Artificial General Intelligence:
Concepts, Architectures and Algorithms – Proceedings of the AGI Workshop 2006
Vol. 156. R.M. Colomb, Ontology and the Semantic Web
Vol. 155. O. Vasilecas et al. (Eds.), Databases and Information Systems IV – Selected Papers
from the Seventh International Baltic Conference DB&IS’2006
Vol. 154. M. Duží et al. (Eds.), Information Modelling and Knowledge Bases XVIII
Vol. 153. Y. Vogiazou, Design for Emergence – Collaborative Social Play with Online and
Location-Based Media
Vol. 152. T.M. van Engers (Ed.), Legal Knowledge and Information Systems – JURIX 2006:
The Nineteenth Annual Conference
Vol. 151. R. Mizoguchi et al. (Eds.), Learning by Effective Utilization of Technologies:
Facilitating Intercultural Understanding
Vol. 150. B. Bennett and C. Fellbaum (Eds.), Formal Ontology in Information Systems –
Proceedings of the Fourth International Conference (FOIS 2006)
Vol. 149. X.F. Zha and R.J. Howlett (Eds.), Integrated Intelligent Systems for Engineering
Design
Vol. 148. K. Kersting, An Inductive Logic Programming Approach to Statistical Relational
Learning
Vol. 147. H. Fujita and M. Mejri (Eds.), New Trends in Software Methodologies, Tools and
Techniques – Proceedings of the fifth SoMeT_06
ISSN 0922-6389
Algorithms and Architectures
of Artificial Intelligence
Enn Tyugu
Institute of Cybernetics, Tallinn University of Technology, Estonia
Amsterdam • Berlin • Oxford • Tokyo • Washington, DC
© 2007 The author and IOS Press.
All rights reserved. No part of this book may be reproduced, stored in a retrieval system,
or transmitted, in any form or by any means, without prior written permission from the publisher.
ISBN 978-1-58603-770-3
Library of Congress Control Number: 2007930585
Publisher
IOS Press
Nieuwe Hemweg 6B
1013 BG Amsterdam
Netherlands
fax: [phone]
e-mail: [email protected]
Distributor in the UK and Ireland Distributor in the USA and Canada
Gazelle Books Services Ltd. IOS Press, Inc.
White Cross Mills 4502 Rachael Manor Drive
Hightown Fairfax, VA 22032
Lancaster LA1 4XS USA
United Kingdom fax: [phone]
fax: [phone] e-mail: [email protected]
e-mail: [email protected]
LEGAL NOTICE
The publisher is not responsible for the use which might be made of the following information.
PRINTED IN THE NETHERLANDS
Algorithms and Architectures of Artificial Intelligence v
E. Tyugu
IOS Press, 2007
© 2007 The author and IOS Press. All rights reserved.
Foreword
This book gives an overview of methods developed in artificial intelligence for search,
learning, problem solving and decision-making. It gives an overview of algorithms and
architectures of artificial intelligence that have reached the degree of maturity when a
method can be presented as an algorithm, or when a well-defined architecture is known, e.g.
in neural nets and intelligent agents. It can be used as a handbook for a wide audience of
application developers who are interested in using artificial intelligence methods in their
software products. Parts of the text are rather independent, so that one can look into index
and go directly to a description of a method presented in the form of an abstract algorithm
or an architectural solution. The book can be used also as a textbook for a course in applied
artificial intelligence. It has been used as a course material for this kind of courses on
undergraduate and graduate levels at Tallinn University of Technology and Royal Institute
of Technology in Stockholm. Exercises on the subject are added at the end of each chapter.
Neither programming skills nor specific knowledge in computer science are expected from
the reader. However, some parts of the text will be fully understood by those who know the
terminology of computing well.
The algorithms developed in the field of artificial intelligence are often directly
applicable in solving various difficult problems and, therefore, they constitute a valuable
part of the artificial intelligence that can be used independently of the philosophical and
deep theoretical considerations related to the research in this area. Most of the algorithms
are presented in this book in a simple algorithmic language. Sometimes, e.g. when
describing neurons in artificial neural nets, it has been more expressive to use common
mathematical notations for presenting the functionality of entities. Taxonomy of algorithms
and methods concerned is discussed at the end of each chapter, summarizing the material
presented in it. This taxonomy can be a valuable aid in developing new algorithms by
elaborating the presented ones.
This page intentionally left blank
vii
Contents
Introduction............................................................................................................................1
Language of algorithms.........................................................................................................3
1. Knowledge Handling.........................................................................................................5
1.1. Knowledge and knowledge systems...........................................................................6
1.1.1. Abstract representation of knowledge systems...................................................6
1.1.2. Examples of deductive systems..........................................................................8
1.2. Brute force deduction and value propagation...........................................................11
1.3. Clausal calculi and resolution..................................................................................12
1.3.1. Language..........................................................................................................12
1.3.2. Inference rule - resolution................................................................................14
1.3.3. Resolution strategies.........................................................................................15
1.4. Pure Prolog...............................................................................................................15
1.5. Nonmonotonic theories............................................................................................17
1.6. Production rules.......................................................................................................19
1.7. Decision tables.........................................................................................................21
1.8. Rete algorithm..........................................................................................................22
1.9. Semantic networks...................................................................................................25
1.10. Frames....................................................................................................................27
1.11. Knowledge architecture..........................................................................................29
1.11.1. Hierarchical connection..................................................................................29
1.11.2. Semantic connection.......................................................................................30
1.11.3. Union..............................................................................................................31
1.11.4. Operational connection...................................................................................32
1.11.5. Examples of knowledge architectures............................................................32
1.11.6. Ontologies and knowledge systems................................................................34
1.12. Summary................................................................................................................35
1.13. Exercises................................................................................................................38
2. Search..............................................................................................................................41
2.1. Search problem.........................................................................................................42
2.2. Exhaustive search methods......................................................................................44
2.2.1. Breadth-first search..........................................................................................44
2.2.2. Depth-first search.............................................................................................46
2.2.3. Search on binary trees......................................................................................47
2.3. Heuristic search methods.........................................................................................48
2.3.1. Best-first search................................................................................................48
2.3.2. Beam search.....................................................................................................49
2.3.3. Hill-climbing....................................................................................................50
2.3.4. Constrained hill-climbing.................................................................................51
2.3.5. Search with backtracking.................................................................................52
2.3.6. Search on and-or trees......................................................................................53
2.3.7. Search with dependency-directed backtracing.................................................54
2.3.8. Branch-and-bound search.................................................................................55
2.3.9. Stochastic branch and bound search.................................................................56
2.3.10. Minimax search..............................................................................................57
2.3.11. Alpha-beta pruning.........................................................................................58
2.4. Specific search methods...........................................................................................60
2.4.1. A* algorithm....................................................................................................60
2.4.2. Unification........................................................................................................62
viii
2.4.3. Dictionary search..............................................................................................63
2.4.4. Simulated annealing.........................................................................................65
2.4.5. Discrete dynamic programming.......................................................................68
2.4.6. Viterby algorithms............................................................................................70
2.5. Forward search and backward search.......................................................................72
2.6. Hierarchy of search methods....................................................................................73
2.7. Exercises..................................................................................................................75
3. Learning and Decision Making........................................................................................79
3.1. Learning for adaptation............................................................................................80
3.1.1. Parametric learning...........................................................................................80
3.1.2. Adaptive automata............................................................................................81
3.2. Symbolic learning....................................................................................................84
3.2.1. Concept learning as search in a hypothesis space............................................84
3.2.2. Specific to general concept learning.................................................................86
3.2.3. General to specific concept learning................................................................88
3.2.4. Inductive inference...........................................................................................90
3.2.5. Learning with an oracle....................................................................................91
3.2.6. Inductive logic programming...........................................................................92
3.2.7. Learning by inverting resolution......................................................................94
3.3. Massively parallel learning in genetic algorithms....................................................98
3.4. Learning in neural nets...........................................................................................100
3.4.1. Perceptrons.....................................................................................................102
3.4.2. Hopfield nets..................................................................................................103
3.4.3. Hamming nets................................................................................................104
3.4.4. Comparator.....................................................................................................106
3.4.5. Carpenter-Grossberg classifier.......................................................................106
3.4.6. Kohonen's feature maps..................................................................................109
3.4.7. Bayesian networks..........................................................................................109
3.4.8. Taxonomy of neural nets................................................................................112
3.5. Data clustering.......................................................................................................113
3.5.1. Sequential leader clustering............................................................................114
3.5.2. K-means clustering.........................................................................................114
3.6. Specific learning algorithms...................................................................................116
3.6.1. Learning decision trees from examples..........................................................116
3.6.2. Learning productions from examples.............................................................119
3.6.3. Discovering regularities in monotonous systems...........................................120
3.6.4. Discovering relations and structure................................................................123
3.7. Summary................................................................................................................125
3.8. Exercises................................................................................................................127
4. Problem Solving and Planning.......................................................................................129
4.1. Constraint satisfaction problem..............................................................................130
4.2. Consistency algorithms..........................................................................................132
4.2.1. Binary consistency.........................................................................................132
4.2.2. Path consistency............................................................................................133
4.3. Propagation algorithms..........................................................................................134
4.3.1. Functional constraint networks.......................................................................134
4.3.2. Computational problems and value propagation............................................135
4.3.3. Equational problem-solver.............................................................................137
4.3.4. Minimizing an algorithm................................................................................139
4.3.5. Lattice of problems on functional constraint network....................................140
ix
4.3.6. Higher-order constraint propagation..............................................................140
4.4. Special algorithms of constraint solving................................................................145
4.4.1. Clustering of equations...................................................................................145
4.4.2. Interval propagation.......................................................................................148
4.5. Program synthesis..................................................................................................149
4.5.1. Deductive synthesis of programs....................................................................149
4.5.2. Inductive synthesis of programs.....................................................................150
4.5.3. Transformational synthesis of programs........................................................150
4.6. Structural synthesis of programs............................................................................151
4.7. Planning.................................................................................................................153
4.7.1. Scheduling......................................................................................................155
4.8. Intelligent agents....................................................................................................155
4.8.1. Agent architectures.........................................................................................156
4.8.2. Agent communication languages...................................................................159
4.8.3. Implementation of agents...............................................................................160
4.8.4. Reflection.......................................................................................................161
4.9. Exercises................................................................................................................163
References..........................................................................................................................167
Subject Index.....................................................................................................................169