Artificial Intelligence in the 21st Century. A Living Introduction

668 Pages·2016·5.73 MB·English

by  Stephen Lucci, Danny Kopec

st
A Living Introduction
Stephen Lucci Danny Kopec
T he City College of New York, CUNY Brooklyn College, CUNY
Arinti tfihcei A2l1 in tceelnltiguernyce
second edition
Mercury leArning And inforMAtion
Publisher: David Pallai
M L I
22841 Quicksilver Drive
Dulles, VA 20166
Preface
In 2006 Professor James Moor of the Philosophy Department at Dartmouth College asked me to
organize a computer games exhibit and competition at AI @ 50, a conference that celebrated the
50th Anniversary of the Dartmouth Summer Conference where John McCarthy coined the term
“Artiicial Intelligence.” A number of the original attendees of that Dartmouth Conference were
able to attend AI @ 50 including the late John McCarthy, Marvin Minsky, the late Oliver Selfridge,
and Ray Solomonoff. Professor Lucci also attended AI @ 50 and shortly thereafter we agreed to
collaborate on an AI text.
Perspective and Needs
Our view is that AI is comprised of PEOPLE, IDEAS, METHODS, MACHINES, and OUTCOMES.
First, it is people who make up AI. People have ideas and these ideas become methods. Those
ideas can be represented by algorithms, heuristics, procedures or systems that are the backbone of
computation; and inally we have the production of those machines (programs) which we can call
“outcomes.” Every outcome can be measured for its value, effectiveness, eficiency, etc.
We have found that existing AI books are often lacking in one or more of these areas. Without
people there is no AI. Hence we have decided that it is important to “showcase” the people who
have made AI successful through the human interest boxes which are sprinkled throughout the text.
From people come the ideas and the development of the methods that we present over the seventeen
chapters of this book. AI and computer science are relatively young ields, compared to other
sciences such as mathematics, physics, chemistry and biology. Yet, AI is a discipline that is truly
interdisciplinary, combining elements of many other ields. Machines/computers are the tools of
AI researchers allowing them to experiment, learn, and improve methods for problem-solving that
can be applied in many interesting domains that can be beneicial to humankind. And inally, not in
the least, there are the outcomes, measurable as the results of applying AI to various problems and
disciplines that remind us that AI must also be accountable. In a number of places in our text you
will ind discussions of the distinction between “performance” and “competence.” Both are needed,
as the ield of AI matures and advances.
To date, as faculty members who have taught from and observed the development of AI
texts, we have found that most of the available texts sometimes fall short in one or more of the
areas described above. The names and vast contributions of Turing, McCarthy, Minsky, Michie,
McLelland, Feigenbaum, Shortliffe, Lenat, Newell and Simon, Brooks, and many others should
be familiar to students. Yet, this is not a history book! We believe, however, that the discipline, as
interesting and widespread as it is, with its ininite potential, should justiiably be colored with the
fascinating ideas and work of the people who comprise it.
Furthermore, students need hands on experience with problem solving, i.e., students need to
get their hands “dirty” with the fundamentals of search techniques fully explained in Chapters
2 through 4, the methods of logic in Chapter 5, and the role of knowledge representation in AI
(Chapter 6). Chapter 7 sets the stage for learning about fuzzy logic (Chapter 8) and expert systems
(Chapter 9).
Advanced methods such as neural networks and genetic algorithms are thoroughly presented
in Chapters 11 and 12. And inally, advanced topics such as natural language processing, planning,
robotics and advanced computer games are covered in Chapters 13, 14, 15 and 16 respectively.
Chapter 17, Reprise, summarizes where you’ve been on your journey with us through AI, with a
view to the future.
The presentation is enhanced with several hundred fully worked examples, as well as over
300 igures and images, many in color. Students will also beneit from the signiicant number of
solutions to exercises that have been provided.
How to Use This Book
This text contains more material than can comfortably be covered in a one semester (45 contact
hours) course. The authors have taught the following courses using material that has led to the
st
development of AI in the 21 Century. Note that at CUNY, graduate courses often meet for three
hours per week for 15 weeks.
As a irst course in AI (graduate or undergraduate):
I A Brief History of AI: Uses and limitations of the subject. Application areas.
Chapter 1 6 contact hours
II Search Methodologies: State Space. Graphs. Generate and Test. Backtracking, Greedy
Search. Blind Search Methods – depth irst search breadth irst search and depth-irst iterative
deepening.
Chapter 2 3 hours
III Informed Search: Heuristics, Hill Climbing, Beam Search, Best First Search, Branch and
Bound based Search and A* Search; And/Or Trees.
Chapter 3 (Section 3.7.3 – Bidirectional Search is optional) 3 hours
IV Search Using Games: Game Trees and Minimax Evaluation. Elementary two-person games:
tic-tac-toe and nim. Minimax with Alpha-Beta Pruning.
C hapter 4 (Section 4.5 – Game Theory and The Iterated Prisoner’s Dilemma is optional).
3 hours.
V Logic in AI: Propositional Logic and Predicate Logic (FOPL); Uniication and Resolution in
FOPL. Converting a Predicate Expression to Clause Form.
Chapter 5 (Section 5.4 – Other Logics is optional) 6 hours
VI Knowledge Representation: Choices of representation; Semantic Nets, Frames, and Scripts.
Inheritance and Object-Oriented Programming.Production Systems; Agent Methods.
C hapter 6 (Sections 6.10, 6.11 – Association and More Recent Approaches are optional)
3 hours.
VII Production Systems: A rchitecture and Examples, Resolution Strategies. Conlict
Resolution Strategies. State Space Search – Data Driven and Goal
Driven Approaches, Cellular Automata (CA). One-dimensional
CA (Wolfram) and Two-Dimensional CA and The Game of Life
(Conway).
C hapter 7 (Section 7.6 Stochastic Processes and Markov Chains are optional) 3 hours
VIII Expert Systems (ES): An Introduction: Why ES? Characteristics and Architectures for
ES; Knowledge Engineering, Knowledge Acquisition and Classic Expert Systems; Newer
Systems Case-based Approaches. Chapter 9 (Sections 9.6, 9.7 and 9.8 are optional). 3 hours
IX Introduction to Neural Computing: Rudiments of Artiicial Neural Networks and and The
Perceptron Learning Rule.
Chapter 11 Sections 1.0, 11.1 and 11.3 only 3 hours
X Introduction to Evolutionary Computing - Genetic Algorithms:
Chapter 12 Sections 12.0 and 12.2 (only) 2 hours
XI Automated Planning: The Problem, Planning as Search; Means Ends Analysis (GPS)STRIPS;
Various planning algorithms and methods. More Modern Systems: Nonlin, Graphplan, etc.
Chapter 14 Sections 14.0, 14.1, 14.3.1, 14.3.12, 14.4.1 2 hours
XII Epilog: Accomplishments of the First 50 years of AI. Prospects for the Future -Where are we
going?
Chapter 17 2 hours
Midterm Exam 3 hours
Final Exam 3 hours
Two-Three Programming Assignments (one in Prolog)
One Term Paper
As a second course in AI
(AI-2, usually at the graduate level). Originally it was taught as a course in Neural Computing.
Artiicial neural networks (ANN) are often used for learning—e.g., in the distinction between pattern
classes; it therefore seemed natural to incorporate genetic algorithms (GA) into the curriculum. AI
systems are often required to justify their reasoning, especially characteristic of expert systems.
ANN’s are not especially proicient in this capacity. Fuzzy logic was added to ANN’s and fuzzy
ANN’s are often used in conjunction to remedy this deiciency.
Emergence, ant colony optimization, fractals, artiicial life and evolutionary computation
(beyond GA) found their way into this course as all of these perspectives are useful in solving
dificult problems. Many refer to such a rubric as “natural computing” because Mother Nature
provides inspiration for these approaches. A proposed syllabus for AI-2:
I Preliminaries: Elementary Concepts: Natural Computing, AI, A-Life, emergence, feedback,
agent top-down vs. bottom-up development. Supplementary material may be used here.
3 hours
II Search inspired by Mother Nature: Search and State Space Graphs. Hill Climbing and
its drawbacks. Simulated Annealing, Genetic Algorithms and Genetic Programming. Tabu
Search. Ant Colony Optimization.
Chapter 2 Sections 2.0, 2.1, and 2.1.1.
Chapter 3 Sections 3.0, 3.1, and 3.2
Chapter 12 (10 – 15 hours)
III Neural Networks: Artiicial neurons vs. their biological counterparts. McCulloch-Pitts
Neurons. The Perceptron Learning Rule and its limitations. The Delta Rule. Backpropagation,
Analyzing patterns and some guidelines for training. Discrete Hopield Networks. Application
Areas. Introduction to Machine Learning Chapter 10
Chapter 11 18 Hours
IV Fuzzy Sets and Fuzzy Logic: Crisp Sets vs. Fuzzy Sets. Membership functions. Fuzzy logic
and fuzzy inference systems.
Chapter 8 Sections 8.0 – 8.3 3 Hours
V Optional Topics: Selected from:
• Unsupervised Learning in ANN.
• Artificial Life including Cellular Automata
• Fractals and Complexity
• Immunocomputing
• Quantum Computing 2+ Hours
A 3-hour midterm and a 3-hour inal are given. There are 5–6 programming assignments and a term
paper.
Some alternative courses could easily be designed from the 17 Chapters we’ve developed.
A irst course could, for example include: Chapter 1 (Introduction/Overview) Chapter 2
(Uninformed Search), Chapter 3 (Intelligent Search), Chapter 4 (Search Using Games), Chapter 5
(Logic), Chapter 6 (Knowledge Representation) Chapter 7 (Production Systems), and Chapter 9
(Expert Systems).
A second course could consist of: Chapter 8 (Fuzzy Logic) Chapter 10 (Machine Learning:
Part 1) Chapter 11 (Neural Networks), Chapter 12 (Search Inspired by Mother Nature) and then one
or two of the special topics chapters from Chapter 13 (Natural Language Processing), Chapter 14
(Planning), Chapter 15 (Robotics), and Chapter 16 (Advanced Computer Games).
A Special Topics Course on Expert Systems might run: Chapter 1 (Introduction), Chapter 7
(Production Systems), Chapter 9 (Expert Systems) “spiced-up” with Chapter 12 (Search Inspired
by Mother Nature) and some supplemental papers / readings.
Stephen Lucci’s vast classroom experience teaching AI courses at City College, Brooklyn
College, and other CUNY schools, has often been lauded by students. Danny Kopec has considerable
research experience in Computer Chess (University of Edinburgh, Machine Intelligence Research
Unit), Intelligent Tutoring Systems (University of Maine, 1988–1992) and Computer Science
Education / Software Engineering / Medical Errors, Technological Mishaps and Problem Solving
(Brooklyn College from 1991 – present). The text represents a strong collaboration of our efforts.
You will occasionally hear two voices sharing their ideas and experience. The writing process itself
has often been joint, listening and adjusting to each other for knowledge, opinion, and style.
A Shared Vision
Writing this book has not been an overnight process. We also believe that our approaches, to writing
and development of material, while different in many respects are complementary.
We believe that the composite work should provide a strong foundation for anyone interested
in AI with suficient opportunity to accrue knowledge, experience, and competence in the methods
that deine the ield. We are fortunate that both the authors and the publisher, David Pallai, president
and founder of Mercury Learning and Information, have shared the same goals and vision for this
book. Fundamental to this effort has been the agreement that the book should be balanced with
theory and applications, accurate, pedagogically sound, and reasonably priced. The process has
required several years, but we are particularly appreciative to Mr. Pallai for seeing the potential in
our book and bringing it to fruition.
We hope that you will enjoy and learn from our efforts.
Preface to the Second Edition
Much time has passed since the publication of our irst edition. Artiicial Intelligence concepts,
methods, and systems are becoming more integrated into everyday activities. For example, at the
time when our irst edition was being developed, many automobiles were built with the capability
to parallel park themselves; it has now become commonplace to build cars with collision avoidance
systems. Technologies that were the fantasies of sci-i enthusiasts, e.g. drones and robots, are now
a reality; increasingly, they are becoming a part of our everyday existence. Phone apps, GPS
systems, and social networks, which were just surfacing in the irst decade of 21st Century, are
now pervasive. They include, optimal trafic routing, health advising, and personal attendants for
diverse aspects of our lives, each typically using some form of AI. Advances in Natural Language
Processing and Speech have dramatically changed how we interact with machines.
The Second Edition has a new Chapter 10 (Machine Learning: Part I) with presentation and
discussion of Decision Trees for machine learning. Hence Chapter 10, combined with Chapters 11
(Neural Approaches) and Chapter 12 (an introduction to genetic approaches) together provide a
foundation for further study. Chapter 13 (Natural Understanding) has a new Section (13.10) which
presents the theory, methods, and applications of speech understanding. A new section on Metaphor
in NLP is also included. Chapter 15 offers an overview of the ield of Robotics, including recent
applications. The end of this chapter, in conjunction with Chapter 17 (Reprise) provides a glimpse
of what the future may hold for us. New exercises have been added to many of the original chapters.
Instructor Resource Files (Available Upon Adoption)
The instructor iles contain the complete set of PowerPoint slides, solutions, sample exams, high
resolution color igures, and an instructor’s guide with teaching suggestions.
Online Resources
Digital versions of the text and all of the instructor and student materials are available at
www.authorcloudware.com, Vital Source, and many other e-vendors. Electronic review and desk
copies are also available.
Stephen Lucci Danny Kopec
The City College of New York Brooklyn College (The City University of New York)
NY, NY
November, 2015
Contents
Preface xvii
Acknowledgments xxiii
Credits xxvii
Part 1: Introduction 1
Chapter 1 Overview of Artificial Intelligence 3
1.0 Introduction
1.0.1 What is Artificial Intelligence? 4
1.0.2 What is Thinking? What is intelligence? 5
1.1 The Turing Test 7
1.1.1 Definition of the Turing Test 7
1.1.2 Controversies and Criticisms of the Turing Test 9
1.2 Strong AI Versus Weak AI 10
1.3 Heuristics 12
1.3.1 The Diagonal of a Rectangular Solid: Solving a Simpler, But Related Problem 12
1.3.2 The Water Jug Problem: Working Backward 13
1.4 Identifying Problems Suitable for AI 14
1.5 Applications and Methods 16
1.5.1 Search Algorithms and Puzzles 17
1.5.2 Two-Person Games 19
1.5.3 Automated Reasoning 19
1.5.4 Production Rules and Expert Systems 20
1.5.5 Cellular Automata 21
1.5.6 Neural Computation 22
1.5.7 Genetic Algorithms 23
1.5.8 Knowledge Representation 23
1.5.9 Uncertainty Reasoning 25
1.6 Early History of AI 25
1.6.1 Logicians and Logic Machines 27
1.7 Recent History of AI to the Present 29
1.7.1 Games 29
1.7.2 Expert Systems 30
1.7.3 Neural Computing 31
1.7.4 Evolutionary Computation 31
1.7.5 Natural Language Processing 32
1.7.6 Bioinformatics 34
1.8 AI In the New Millennium 34
1.9 Chapter Summary 36
PART II: FUNDAMENTALS 43
Chapter 2 Uninformed Search 45
2.0 Introduction: Search in Intelligent Systems 46
2.1 State-Space Graphs 47
2.1.1 The False Coin Problem 47
2.2 Generate-and-Test Paradigm 48
2.2.1 Backtracking 50
2.2.2 The Greedy Algorithm 53
2.2.3 The Traveling Salesperson Problem 55
2.3 Blind Search Algorithms 56
2.3.1 Depth First Search 57
2.3.2 Breadth First Search 58
2.4 Implementing and Comparing Blind Search Algorithms 60
2.4.1 Implementing A Depth First Search Solution 61
2.4.2 Implementing A Breadth First Search Solution 62
2.4.3 Measuring Problem-Solving Performance 62
2.4.4 Comparing Dfs and Bfs 64
2.5 Chapter Summary 67
Chapter 3 Informed Search 75
3.0 Introduction 75
3.1 Heuristics 77
3.2 Informed Search Algorithms (Part I) – Finding Any Solution 83
3.2.1 Hill Climbing 83
3.2.2 Steepest-Ascent Hill Climbing 84
3.3 The Best-First Search 86
3.4 The Beam Search 89
3.5 Additional Metrics for Search Algorithms 90
3.6 Informed Search (Part 2) – Finding An Optimal Solution 91
3.6.1. Branch and Bound 92
3.6.2 Branch and Bound With Underestimates 95
3.6.3 Branch and Bound With Dynamic Programming 98
3.6.4 The A* Search 99
3.7 Informed Search (Part 3) – Advanced Search Algorithms 100
3.7.1 Constraint Satisfaction Search 100
3.7.2 And/Or Trees 101
3.7.3 The Bidirectional Search 102
3.8 Chapter Summary 105
Chapter 4 Search Using Games 111
4.0 Introduction 112
4.1 Game Trees and Minimax Evaluation 112
4.1.1 Heuristic Evaluation 114
4.1.2 Minimax Evaluation of Game Trees 115
4.2 Minimax With Alpha-Beta Pruning 118
4.3 Variations and Improvements To Minimax 123
4.3.1 Negamax Algorithm 123
4.3.2 Progressive Deepening 125
4.3.3 Heuristic Continuation and the Horizon Effect 125
4.4 Games of Chance and the Expectiminimax Algorithm 125
4.5 Game Theory 127
4.5.1 The Iterated Prisoner’s Dilemma 129
4.6 Chapter Summary 129
Chapter 5 Logic in Artificial Intelligence 137
5.0 Introduction 138
5.1 Logic and Representation 139
5.2 Propositional Logic 140
5.2.1 Propositional Logic – Basics 140
5.2.2 Arguments in the Propositional Logic 144
5.2.3 Proving Arguments in the Propositional Logic Valid – A Second Approach 145
5.3 Predicate Logic – Introduction 147
5.3.1 Unification in the Predicate Logic 148
5.3.2 Resolution in the Predicate Logic 150
5.2.3 Converting A Predicate Expression to Clause Form 152
5.4 Several Other Logics 155
5.4.1 Second Order Logic 156
5.4.2 Non-Monotonic Logic 156
5.4.3 Fuzzy Logic 157
5.4.4 Modal Logic 157
5.5 Chapter Summary 158
Chapter 6 Knowledge Representation 167
6.0 Introduction 168
6.1 Graphical Sketches and the Human Window 171