Artificial Intelligence in the 21st Century : A Living Introduction

679 Pages·2015·9.242 MB·English

by  Stephen Lucci, Danny Kopec

A i
rtificiAl ntelligence
21st c
in the entury
A Living Introduction
s e
econd dition
Stephen Lucci Danny Kopec
The City College of New York, CUNY Brooklyn College, CUNY
M l i
ercury eArning And nforMAtion
Dulles, Virginia
Boston, Massachusetts
New Delhi
Copyright ©2016 by Mercury Learning and Information. All rights reserved.
This publication, portions of it, or any accompanying software may not be reproduced in any way, stored in a
retrieval system of any type, or transmitted by any means, media, electronic display or mechanical display,
including, but not limited to, photocopy, recording, Internet postings, or scanning, without prior permission
in writing from the publisher.
Publisher: David Pallai
Mercury Learning and Information
22841 Quicksilver Drive
Dulles, VA 20166
[email protected]
www.merclearning.com
[phone]
Stephen Lucci and Danny Kopec. ArtificiAl intelligence in the 21ST century: A Living Introduction 2/E
ISBN: 978-1-942270-00-3
The publisher recognizes and respects all marks used by companies, manufacturers, and developers as a means to
distinguish their products. All brand names and product names mentioned in this book are trademarks or service marks
of their respective companies. Any omission or misuse (of any kind) of service marks or trademarks, etc. is not an
attempt to infringe on the property of others.
Library of Congress Control Number: 2015934535
151617321 Printed in the United States of America
This book is printed on acid-free paper.
Our titles are available for adoption, license, or bulk purchase by institutions, corporations, etc.
For additional information, please contact the Customer Service Dept. at 800-232-0223 (toll free). Digital versions of
our titles are available at: www.authorcloudware.com and other e-vendors. All companion iles are available by writing
to the publisher at [email protected].
The sole obligation of mercury Learning and Inform to the purchaser is to replace the book and/or disc, based on
defective materials or faulty workmanship, but not based on the operation or functionality of the product.
To the memory of my parents,
Magdalena and Vladimir Kopec,
Who set the stage for me.
Danny Kopec
To the memory of my parents,
Louis and Connie Lucci,
Both of whom always encouraged my education.
Stephen Lucci
LICENSE, DISCLAIMER OF LIABILITY, AND LIMITED WARRANTY
By purchasing or using this book (the “Work”), you agree that this license grants per-
mission to use the contents contained herein, but does not give you the right of ownership
to any of the textual content in the book or ownership to any of the information or products
This license does not permit uploading of the Work onto the Internet or on a
network (of any kind) without the written consent of the Publisher. Duplication or dissemi-
nation of any text, code, simulations, images, etc. contained herein is limited to and subject
to licensing terms for the respective products, and permission must be obtained from the
Publisher or the owner of the content, etc., in order to reproduce or network any portion of
the textual material (in any media) that is contained in the Work.
Mercury Learning And Information LLC (“MLI” or “the Publisher”) and anyone in-
volved in the creation, writing, or production of the companion disc, accompanying algo-
rithms, code, or computer programs (“the software”), and any accompanying Web site or
software of the Work, cannot and do not warrant the performance or results that might be
obtained by using the contents of the Work. The author, developers, and the Publisher have
used their best efforts to insure the accuracy and functionality of the textual material and/or
programs contained in this package; we, however, make no warranty of any kind, express
or implied, regarding the performance of these contents or programs. The Work is sold “as
is” without warranty (except for defective materials used in manufacturing the book or due
The author, developers, and the publisher of any accompanying content, and anyone
involved in the composition, production, and manufacturing of this work will not be liable
for damages of any kind arising out of the use of (or the inability to use) the algorithms,
source code, computer programs, or textual material contained in this publication. This
includes, but is not limited to, loss of revenue or proit, or other incidental, physical, or
consequential damages arising out of the use of this Work.
The sole remedy in the event of a claim of any kind is expressly limited to replacement
of the book, and only at the discretion of the Publisher. The use of “implied warranty” and
certain “exclusions” vary from state to state, and might not apply to the purchaser of this
Preface
In 2006 Professor James Moor of the Philosophy Department at Dartmouth College asked me to
organize a computer games exhibit and competition at AI @ 50, a conference that celebrated the
50th Anniversary of the Dartmouth Summer Conference where John McCarthy coined the term
“Artiicial Intelligence.” A number of the original attendees of that Dartmouth Conference were
able to attend AI @ 50 including the late John McCarthy, Marvin Minsky, the late Oliver Selfridge,
and Ray Solomonoff. Professor Lucci also attended AI @ 50 and shortly thereafter we agreed to
collaborate on an AI text.
Perspective and Needs
Our view is that AI is comprised of PEOPLE, IDEAS, METHODS, MACHINES, and OUTCOMES.
First, it is people who make up AI. People have ideas and these ideas become methods. Those
ideas can be represented by algorithms, heuristics, procedures or systems that are the backbone of
computation; and inally we have the production of those machines (programs) which we can call
“outcomes.” Every outcome can be measured for its value, effectiveness, eficiency, etc.
We have found that existing AI books are often lacking in one or more of these areas. Without
people there is no AI. Hence we have decided that it is important to “showcase” the people who
have made AI successful through the human interest boxes which are sprinkled throughout the text.
From people come the ideas and the development of the methods that we present over the seventeen
chapters of this book. AI and computer science are relatively young ields, compared to other
sciences such as mathematics, physics, chemistry and biology. Yet, AI is a discipline that is truly
interdisciplinary, combining elements of many other ields. Machines/computers are the tools of
AI researchers allowing them to experiment, learn, and improve methods for problem-solving that
can be applied in many interesting domains that can be beneicial to humankind. And inally, not in
the least, there are the outcomes, measurable as the results of applying AI to various problems and
disciplines that remind us that AI must also be accountable. In a number of places in our text you
will ind discussions of the distinction between “performance” and “competence.” Both are needed,
as the ield of AI matures and advances.
To date, as faculty members who have taught from and observed the development of AI
texts, we have found that most of the available texts sometimes fall short in one or more of the
areas described above. The names and vast contributions of Turing, McCarthy, Minsky, Michie,
xviii ■ Preface
McLelland, Feigenbaum, Shortliffe, Lenat, Newell and Simon, Brooks, and many others should
be familiar to students. Yet, this is not a history book! We believe, however, that the discipline, as
interesting and widespread as it is, with its ininite potential, should justiiably be colored with the
fascinating ideas and work of the people who comprise it.
Furthermore, students need hands on experience with problem solving, i.e., students need to
get their hands “dirty” with the fundamentals of search techniques fully explained in Chapters
2 through 4, the methods of logic in Chapter 5, and the role of knowledge representation in AI
(Chapter 6). Chapter 7 sets the stage for learning about fuzzy logic (Chapter 8) and expert systems
(Chapter 9).
Advanced methods such as neural networks and genetic algorithms are thoroughly presented
in Chapters 11 and 12. And inally, advanced topics such as natural language processing, planning,
robotics and advanced computer games are covered in Chapters 13, 14, 15 and 16 respectively.
Chapter 17, Reprise, summarizes where you’ve been on your journey with us through AI, with a
view to the future.
The presentation is enhanced with several hundred fully worked examples, as well as over
300 igures and images, many in color. Students will also beneit from the signiicant number of
solutions to exercises that have been provided.
How to Use This Book
This text contains more material than can comfortably be covered in a one semester (45 contact
hours) course. The authors have taught the following courses using material that has led to the
development of AI in the 21st Century. Note that at CUNY, graduate courses often meet for three
hours per week for 15 weeks.
As a irst course in AI (graduate or undergraduate):
I A Brief History of AI: Uses and limitations of the subject. Application areas.
Chapter 1 6 contact hours
II Search Methodologies: State Space. Graphs. Generate and Test. Backtracking, Greedy
Search. Blind Search Methods – depth irst search breadth irst search and depth-irst iterative
deepening.
Chapter 2 3 hours
III Informed Search: Heuristics, Hill Climbing, Beam Search, Best First Search, Branch and
Bound based Search and A* Search; And/Or Trees.
Chapter 3 (Section 3.7.3 – Bidirectional Search is optional) 3 hours
IV Search Using Games: Game Trees and Minimax Evaluation. Elementary two-person games:
tic-tac-toe and nim. Minimax with Alpha-Beta Pruning.
C hapter 4 (Section 4.5 – Game Theory and The Iterated Prisoner’s Dilemma is optional).
3 hours.
V Logic in AI: Propositional Logic and Predicate Logic (FOPL); Uniication and Resolution in
FOPL. Converting a Predicate Expression to Clause Form.
Chapter 5 (Section 5.4 – Other Logics is optional) 6 hours
VI Knowledge Representation: Choices of representation; Semantic Nets, Frames, and Scripts.
Inheritance and Object-Oriented Programming.Production Systems; Agent Methods.
C hapter 6 (Sections 6.10, 6.11 – Association and More Recent Approaches are optional)
3 hours.
Preface ■ xix
VII Production Systems: Architecture and Examples, Resolution Strategies. Conlict
Resolution Strategies. State Space Search – Data Driven and Goal
Driven Approaches, Cellular Automata (CA). One-dimensional
CA (Wolfram) and Two-Dimensional CA and The Game of Life
(Conway).
C hapter 7 (Section 7.6 Stochastic Processes and Markov Chains are optional) 3 hours
VIII Expert Systems (ES): An Introduction: Why ES? Characteristics and Architectures for
ES; Knowledge Engineering, Knowledge Acquisition and Classic Expert Systems; Newer
Systems Case-based Approaches. Chapter 9 (Sections 9.6, 9.7 and 9.8 are optional). 3 hours
IX Introduction to Neural Computing: Rudiments of Artiicial Neural Networks and and The
Perceptron Learning Rule.
Chapter 11 Sections 1.0, 11.1 and 11.3 only 3 hours
X Introduction to Evolutionary Computing - Genetic Algorithms:
Chapter 12 Sections 12.0 and 12.2 (only) 2 hours
XI Automated Planning: The Problem, Planning as Search; Means Ends Analysis (GPS)STRIPS;
Various planning algorithms and methods. More Modern Systems: Nonlin, Graphplan, etc.
Chapter 14 Sections 14.0, 14.1, 14.3.1, 14.3.12, 14.4.1 2 hours
XII Epilog: Accomplishments of the First 50 years of AI. Prospects for the Future -Where are we
going?
Chapter 17 2 hours
Midterm Exam 3 hours
Final Exam 3 hours
Two-Three Programming Assignments (one in Prolog)
One Term Paper
As a second course in AI
(AI-2, usually at the graduate level). Originally it was taught as a course in Neural Computing.
Artiicial neural networks (ANN) are often used for learning—e.g., in the distinction between pattern
classes; it therefore seemed natural to incorporate genetic algorithms (GA) into the curriculum. AI
systems are often required to justify their reasoning, especially characteristic of expert systems.
ANN’s are not especially proicient in this capacity. Fuzzy logic was added to ANN’s and fuzzy
ANN’s are often used in conjunction to remedy this deiciency.
Emergence, ant colony optimization, fractals, artiicial life and evolutionary computation
(beyond GA) found their way into this course as all of these perspectives are useful in solving
dificult problems. Many refer to such a rubric as “natural computing” because Mother Nature
provides inspiration for these approaches. A proposed syllabus for AI-2:
I Preliminaries: Elementary Concepts: Natural Computing, AI, A-Life, emergence, feedback,
agent top-down vs. bottom-up development. Supplementary material may be used here.
3 hours
II Search inspired by Mother Nature: Search and State Space Graphs. Hill Climbing and
its drawbacks. Simulated Annealing, Genetic Algorithms and Genetic Programming. Tabu
Search. Ant Colony Optimization.
xx ■ Preface
Chapter 2 Sections 2.0, 2.1, and 2.1.1.
Chapter 3 Sections 3.0, 3.1, and 3.2
Chapter 12 (10 – 15 hours)
III Neural Networks: Artiicial neurons vs. their biological counterparts. McCulloch-Pitts
Neurons. The Perceptron Learning Rule and its limitations. The Delta Rule. Backpropagation,
Analyzing patterns and some guidelines for training. Discrete Hopield Networks. Application
Areas. Introduction to Machine Learning Chapter 10
Chapter 11 18 Hours
IV Fuzzy Sets and Fuzzy Logic: Crisp Sets vs. Fuzzy Sets. Membership functions. Fuzzy logic
and fuzzy inference systems.
Chapter 8 Sections 8.0 – 8.3 3 Hours
V Optional Topics: Selected from:
• Unsupervised Learning in ANN.
• Artificial Life including Cellular Automata
• Fractals and Complexity
• Immunocomputing
• Quantum Computing 2+ Hours
A 3-hour midterm and a 3-hour inal are given. There are 5–6 programming assignments and a term
paper.
Some alternative courses could easily be designed from the 17 Chapters we’ve developed.
A irst course could, for example include: Chapter 1 (Introduction/Overview) Chapter 2
(Uninformed Search), Chapter 3 (Intelligent Search), Chapter 4 (Search Using Games), Chapter 5
(Logic), Chapter 6 (Knowledge Representation) Chapter 7 (Production Systems), and Chapter 9
(Expert Systems).
A second course could consist of: Chapter 8 (Fuzzy Logic) Chapter 10 (Machine Learning:
Part 1) Chapter 11 (Neural Networks), Chapter 12 (Search Inspired by Mother Nature) and then one
or two of the special topics chapters from Chapter 13 (Natural Language Processing), Chapter 14
(Planning), Chapter 15 (Robotics), and Chapter 16 (Advanced Computer Games).
A Special Topics Course on Expert Systems might run: Chapter 1 (Introduction), Chapter 7
(Production Systems), Chapter 9 (Expert Systems) “spiced-up” with Chapter 12 (Search Inspired
by Mother Nature) and some supplemental papers / readings.
Stephen Lucci’s vast classroom experience teaching AI courses at City College, Brooklyn
College, and other CUNY schools, has often been lauded by students. Danny Kopec has considerable
research experience in Computer Chess (University of Edinburgh, Machine Intelligence Research
Unit), Intelligent Tutoring Systems (University of Maine, 1988–1992) and Computer Science
Education / Software Engineering / Medical Errors, Technological Mishaps and Problem Solving
(Brooklyn College from 1991 – present). The text represents a strong collaboration of our efforts.
You will occasionally hear two voices sharing their ideas and experience. The writing process itself
has often been joint, listening and adjusting to each other for knowledge, opinion, and style.
A Shared Vision
Writing this book has not been an overnight process. We also believe that our approaches, to writing
and development of material, while different in many respects are complementary.
Preface ■ xxi
We believe that the composite work should provide a strong foundation for anyone interested
in AI with suficient opportunity to accrue knowledge, experience, and competence in the methods
that deine the ield. We are fortunate that both the authors and the publisher, David Pallai, president
and founder of Mercury Learning and Information, have shared the same goals and vision for this
book. Fundamental to this effort has been the agreement that the book should be balanced with
theory and applications, accurate, pedagogically sound, and reasonably priced. The process has
required several years, but we are particularly appreciative to Mr. Pallai for seeing the potential in
our book and bringing it to fruition.
We hope that you will enjoy and learn from our efforts.
Preface to the Second Edition
Much time has passed since the publication of our irst edition. Artiicial Intelligence concepts,
methods, and systems are becoming more integrated into everyday activities. For example, at the
time when our irst edition was being developed, many automobiles were built with the capability
to parallel park themselves; it has now become commonplace to build cars with collision avoidance
systems. Technologies that were the fantasies of sci-i enthusiasts, e.g. drones and robots, are now
a reality; increasingly, they are becoming a part of our everyday existence. Phone apps, GPS
systems, and social networks, which were just surfacing in the irst decade of 21st Century, are
now pervasive. They include, optimal trafic routing, health advising, and personal attendants for
diverse aspects of our lives, each typically using some form of AI. Advances in Natural Language
Processing and Speech have dramatically changed how we interact with machines.
The Second Edition has a new Chapter 10 (Machine Learning: Part I) with presentation and
discussion of Decision Trees for machine learning. Hence Chapter 10, combined with Chapters 11
(Neural Approaches) and Chapter 12 (an introduction to genetic approaches) together provide a
foundation for further study. Chapter 13 (Natural Understanding) has a new Section (13.10) which
presents the theory, methods, and applications of speech understanding. A new section on Metaphor
in NLP is also included. Chapter 15 offers an overview of the ield of Robotics, including recent
applications. The end of this chapter, in conjunction with Chapter 17 (Reprise) provides a glimpse
of what the future may hold for us. New exercises have been added to many of the original chapters.
Instructor Resource Files (Available Upon Adoption)
The instructor iles contain the complete set of PowerPoint slides, solutions, sample exams, high
resolution color igures, and an instructor’s guide with teaching suggestions.
Online Resources
Digital versions of the text and all of the instructor and student materials are available at
www.authorcloudware.com, Vital Source, and many other e-vendors. Electronic review and desk
copies are also available.
Stephen Lucci Danny Kopec
The City College of New York Brooklyn College (The City University of New York)
NY, NY
November, 2015
Contents
Preface xvii
Acknowledgments xxiii
Credits xxvii
Part 1: Introduction 1
Chapter 1 Overview of Artificial Intelligence 3
1.0 Introduction
1.0.1 What is Artificial Intelligence? 4
1.0.2 What is Thinking? What is intelligence? 5
1.1 The Turing Test 7
1.1.1 Definition of the Turing Test 7
1.1.2 Controversies and Criticisms of the Turing Test 9
1.2 Strong AI Versus Weak AI 10
1.3 Heuristics 12
1.3.1 The Diagonal of a Rectangular Solid: Solving a Simpler, But Related Problem 12
1.3.2 The Water Jug Problem: Working Backward 13
1.4 Identifying Problems Suitable for AI 14
1.5 Applications and Methods 16
1.5.1 Search Algorithms and Puzzles 17
1.5.2 Two-Person Games 19
1.5.3 Automated Reasoning 19
1.5.4 Production Rules and Expert Systems 20
1.5.5 Cellular Automata 21
1.5.6 Neural Computation 22
1.5.7 Genetic Algorithms 23
1.5.8 Knowledge Representation 23
1.5.9 Uncertainty Reasoning 25