The quest for artificial intelligence : a history of ideas and achievements

707 Pages·2009·8.185 MB·English

by  Nils J Nilsson

0.0
THE QUEST FOR ARTIFICIAL INTELLIGENCE
A HISTORY OF IDEAS AND ACHIEVEMENTS
Web Version
Print version published by Cambridge University Press
http://www.cambridge.org/us/0521122937
1
Nils J. Nilsson
Stanford University
0
For Grace McConnell Abbott,
my wife and best friend
2
0.0
Contents
I Beginnings 17
1 Dreams and Dreamers 19
2 Clues 27
2.1 From Philosophy and Logic . . . . . . . . . . . . . . . . . . . . . 27
2.2 From Life Itself . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.2.1 Neurons and the Brain . . . . . . . . . . . . . . . . . . . . 34
2.2.2 Psychology and Cognitive Science . . . . . . . . . . . . . 37
2.2.3 Evolution . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.2.4 Development and Maturation . . . . . . . . . . . . . . . . 45
2.2.5 Bionics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.3 From Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
2.3.1 Automata, Sensing, and Feedback . . . . . . . . . . . . . 46
2.3.2 Statistics and Probability . . . . . . . . . . . . . . . . . . 52
2.3.3 The Computer . . . . . . . . . . . . . . . . . . . . . . . . 53
II Early Explorations: 1950s and 1960s 71
3 Gatherings 73
3.1 Session on Learning Machines . . . . . . . . . . . . . . . . . . . . 73
3.2 The Dartmouth Summer Project . . . . . . . . . . . . . . . . . . 77 3
3.3 Mechanization of Thought Processes . . . . . . . . . . . . . . . . 81
4 Pattern Recognition 89
0 CONTENTS
4.1 Character Recognition . . . . . . . . . . . . . . . . . . . . . . . . 90
4.2 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.2.1 Perceptrons . . . . . . . . . . . . . . . . . . . . . . . . . . 92
4.2.2 ADALINES and MADALINES . . . . . . . . . . . . . . . 98
4.2.3 The MINOS Systems at SRI . . . . . . . . . . . . . . . . 98
4.3 Statistical Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 102
4.4 Applications of Pattern Recognition to Aerial Reconnaissance . . 105
5 Early Heuristic Programs 113
5.1 The Logic Theorist and Heuristic Search . . . . . . . . . . . . . . 113
5.2 Proving Theorems in Geometry . . . . . . . . . . . . . . . . . . . 118
5.3 The General Problem Solver . . . . . . . . . . . . . . . . . . . . . 121
5.4 Game-Playing Programs . . . . . . . . . . . . . . . . . . . . . . . 123
6 Semantic Representations 131
6.1 Solving Geometric Analogy Problems. . . . . . . . . . . . . . . . 131
6.2 Storing Information and Answering Questions . . . . . . . . . . . 134
6.3 Semantic Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 136
7 Natural Language Processing 141
7.1 Linguistic Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
7.2 Machine Translation . . . . . . . . . . . . . . . . . . . . . . . . . 146
7.3 Question Answering . . . . . . . . . . . . . . . . . . . . . . . . . 150
8 1960s’ Infrastructure 155
8.1 Programming Languages . . . . . . . . . . . . . . . . . . . . . . . 155
8.2 Early AI Laboratories . . . . . . . . . . . . . . . . . . . . . . . . 157
8.3 Research Support . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
8.4 All Dressed Up and Places to Go . . . . . . . . . . . . . . . . . . 163
III Efflorescence: Mid-1960s to Mid-1970s 167
9 Computer Vision 169
9.1 Hints from Biology . . . . . . . . . . . . . . . . . . . . . . . . . . 171
4
0.0 CONTENTS
9.2 Recognizing Faces . . . . . . . . . . . . . . . . . . . . . . . . . . 172
9.3 Computer Vision of Three-Dimensional Solid Objects . . . . . . 173
9.3.1 An Early Vision System . . . . . . . . . . . . . . . . . . . 173
9.3.2 The “Summer Vision Project” . . . . . . . . . . . . . . . 175
9.3.3 Image Filtering . . . . . . . . . . . . . . . . . . . . . . . . 176
9.3.4 Processing Line Drawings . . . . . . . . . . . . . . . . . . 181
10 “Hand–Eye” Research 189
10.1 At MIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
10.2 At Stanford . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
10.3 In Japan . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
10.4 Edinburgh’s “FREDDY” . . . . . . . . . . . . . . . . . . . . . . . 193
11 Knowledge Representation and Reasoning 199
11.1 Deductions in Symbolic Logic . . . . . . . . . . . . . . . . . . . . 200
11.2 The Situation Calculus . . . . . . . . . . . . . . . . . . . . . . . . 202
11.3 Logic Programming . . . . . . . . . . . . . . . . . . . . . . . . . 203
11.4 Semantic Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 205
11.5 Scripts and Frames . . . . . . . . . . . . . . . . . . . . . . . . . . 207
12 Mobile Robots 213
12.1 Shakey, the SRI Robot . . . . . . . . . . . . . . . . . . . . . . . . 213
12.1.1 A∗: A New Heuristic Search Method . . . . . . . . . . . . 216
12.1.2 Robust Action Execution . . . . . . . . . . . . . . . . . . 221
12.1.3 STRIPS: A New Planning Method . . . . . . . . . . . . . 222
12.1.4 Learning and Executing Plans . . . . . . . . . . . . . . . 224
12.1.5 Shakey’s Vision Routines . . . . . . . . . . . . . . . . . . 224
12.1.6 Some Experiments with Shakey . . . . . . . . . . . . . . . 228
12.1.7 Shakey Runs into Funding Troubles . . . . . . . . . . . . 229
12.2 The Stanford Cart . . . . . . . . . . . . . . . . . . . . . . . . . . 231
5
13 Progress in Natural Language Processing 237
13.1 Machine Translation . . . . . . . . . . . . . . . . . . . . . . . . . 237
13.2 Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
0 CONTENTS
13.2.1 SHRDLU . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
13.2.2 LUNAR . . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
13.2.3 Augmented Transition Networks . . . . . . . . . . . . . . 244
13.2.4 GUS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
14 Game Playing 251
15 The Dendral Project 255
16 Conferences, Books, and Funding 261
IV Applications and Specializations: 1970s to Early
1980s 265
17 Speech Recognition and Understanding Systems 267
17.1 Speech Processing . . . . . . . . . . . . . . . . . . . . . . . . . . 267
17.2 The Speech Understanding Study Group . . . . . . . . . . . . . . 270
17.3 The DARPA Speech Understanding Research Program . . . . . . 271
17.3.1 Work at BBN . . . . . . . . . . . . . . . . . . . . . . . . . 271
17.3.2 Work at CMU . . . . . . . . . . . . . . . . . . . . . . . . 272
17.3.3 Summary and Impact of the SUR Program . . . . . . . . 280
17.4 Subsequent Work in Speech Recognition . . . . . . . . . . . . . . 281
18 Consulting Systems 285
18.1 The SRI Computer-Based Consultant . . . . . . . . . . . . . . . 285
18.2 Expert Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
18.2.1 MYCIN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
18.2.2 PROSPECTOR . . . . . . . . . . . . . . . . . . . . . . . . . 295
18.2.3 Other Expert Systems . . . . . . . . . . . . . . . . . . . . 300
18.2.4 Expert Companies . . . . . . . . . . . . . . . . . . . . . . 303
19 Understanding Queries and Signals 309
19.1 The Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
19.2 Natural Language Access to Computer Systems . . . . . . . . . . 313
19.2.1 LIFER . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
6
0.0 CONTENTS
19.2.2 CHAT-80 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
19.2.3 Transportable Natural Language Query Systems . . . . . 318
19.3 HASP/SIAP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 319
20 Progress in Computer Vision 327
20.1 Beyond Line-Finding . . . . . . . . . . . . . . . . . . . . . . . . . 327
20.1.1 Shape from Shading . . . . . . . . . . . . . . . . . . . . . 327
20.1.2 The 21-D Sketch . . . . . . . . . . . . . . . . . . . . . . . 329
2
20.1.3 Intrinsic Images. . . . . . . . . . . . . . . . . . . . . . . . 329
20.2 Finding Objects in Scenes . . . . . . . . . . . . . . . . . . . . . . 333
20.2.1 Reasoning about Scenes . . . . . . . . . . . . . . . . . . . 333
20.2.2 Using Templates and Models . . . . . . . . . . . . . . . . 335
20.3 DARPA’s Image Understanding Program . . . . . . . . . . . . . 338
21 Boomtimes 343
V “New-Generation” Projects 347
22 The Japanese Create a Stir 349
22.1 The Fifth-Generation Computer Systems Project . . . . . . . . . 349
22.2 Some Impacts of the Japanese Project . . . . . . . . . . . . . . . 354
22.2.1 The Microelectronics and Computer Technology Corpo-
ration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
22.2.2 The Alvey Program . . . . . . . . . . . . . . . . . . . . . 355
22.2.3 ESPRIT . . . . . . . . . . . . . . . . . . . . . . . . . . . . 355
23 DARPA’s Strategic Computing Program 359
23.1 The Strategic Computing Plan . . . . . . . . . . . . . . . . . . . 359
23.2 Major Projects . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362
23.2.1 The Pilot’s Associate. . . . . . . . . . . . . . . . . . . . . 363
23.2.2 Battle Management Systems . . . . . . . . . . . . . . . . 364
7
23.2.3 Autonomous Vehicles . . . . . . . . . . . . . . . . . . . . 366
23.3 AI Technology Base . . . . . . . . . . . . . . . . . . . . . . . . . 369
23.3.1 Computer Vision . . . . . . . . . . . . . . . . . . . . . . . 370
0 CONTENTS
23.3.2 Speech Recognition and Natural Language Processing . . 370
23.3.3 Expert Systems . . . . . . . . . . . . . . . . . . . . . . . . 372
23.4 Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 373
VI Entr’acte 379
24 Speed Bumps 381
24.1 Opinions from Various Onlookers . . . . . . . . . . . . . . . . . . 381
24.1.1 The Mind Is Not a Machine . . . . . . . . . . . . . . . . . 381
24.1.2 The Mind Is Not a Computer . . . . . . . . . . . . . . . . 383
24.1.3 Differences between Brains and Computers . . . . . . . . 392
24.1.4 But Should We? . . . . . . . . . . . . . . . . . . . . . . . 393
24.1.5 Other Opinions . . . . . . . . . . . . . . . . . . . . . . . . 398
24.2 Problems of Scale . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
24.2.1 The Combinatorial Explosion . . . . . . . . . . . . . . . . 399
24.2.2 Complexity Theory . . . . . . . . . . . . . . . . . . . . . . 401
24.2.3 A Sober Assessment . . . . . . . . . . . . . . . . . . . . . 402
24.3 Acknowledged Shortcomings . . . . . . . . . . . . . . . . . . . . . 406
24.4 The “AI Winter” . . . . . . . . . . . . . . . . . . . . . . . . . . . 408
25 Controversies and Alternative Paradigms 413
25.1 About Logic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 413
25.2 Uncertainty . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414
25.3 “Kludginess” . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 416
25.4 About Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
25.4.1 Behavior-Based Robots . . . . . . . . . . . . . . . . . . . 417
25.4.2 Teleo-Reactive Programs . . . . . . . . . . . . . . . . . . 419
25.5 Brain-Style Computation . . . . . . . . . . . . . . . . . . . . . . 423
25.5.1 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . 423
25.5.2 Dynamical Processes . . . . . . . . . . . . . . . . . . . . . 424
25.6 Simulating Evolution . . . . . . . . . . . . . . . . . . . . . . . . . 425
25.7 Scaling Back AI’s Goals . . . . . . . . . . . . . . . . . . . . . . . 429
8
0.0 CONTENTS
VII The Growing Armamentarium: From the 1980s
Onward 433
26 Reasoning and Representation 435
26.1 Nonmonotonic or Defeasible Reasoning . . . . . . . . . . . . . . . 435
26.2 Qualitative Reasoning . . . . . . . . . . . . . . . . . . . . . . . . 439
26.3 Semantic Networks . . . . . . . . . . . . . . . . . . . . . . . . . . 441
26.3.1 Description Logics . . . . . . . . . . . . . . . . . . . . . . 441
26.3.2 WordNet . . . . . . . . . . . . . . . . . . . . . . . . . . . 444
26.3.3 Cyc . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 446
27 Other Approaches to Reasoning and Representation 455
27.1 Solving Constraint Satisfaction Problems . . . . . . . . . . . . . 455
27.2 Solving Problems Using Propositional Logic . . . . . . . . . . . . 460
27.2.1 Systematic Methods . . . . . . . . . . . . . . . . . . . . . 461
27.2.2 Local Search Methods . . . . . . . . . . . . . . . . . . . . 463
27.2.3 Applications of SAT Solvers . . . . . . . . . . . . . . . . . 466
27.3 Representing Text as Vectors . . . . . . . . . . . . . . . . . . . . 466
27.4 Latent Semantic Analysis . . . . . . . . . . . . . . . . . . . . . . 469
28 Bayesian Networks 475
28.1 Representing Probabilities in Networks . . . . . . . . . . . . . . . 475
28.2 Automatic Construction of Bayesian Networks . . . . . . . . . . 482
28.3 Probabilistic Relational Models . . . . . . . . . . . . . . . . . . . 486
28.4 Temporal Bayesian Networks . . . . . . . . . . . . . . . . . . . . 488
29 Machine Learning 495
29.1 Memory-Based Learning . . . . . . . . . . . . . . . . . . . . . . . 496
29.2 Case-Based Reasoning . . . . . . . . . . . . . . . . . . . . . . . . 498
29.3 Decision Trees. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 500
29.3.1 Data Mining and Decision Trees . . . . . . . . . . . . . . 500
9
29.3.2 Constructing Decision Trees . . . . . . . . . . . . . . . . . 502
29.4 Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . 507
29.4.1 The Backprop Algorithm . . . . . . . . . . . . . . . . . . 508
0 CONTENTS
29.4.2 NETtalk . . . . . . . . . . . . . . . . . . . . . . . . . . . . 509
29.4.3 ALVINN. . . . . . . . . . . . . . . . . . . . . . . . . . . . 510
29.5 Unsupervised Learning . . . . . . . . . . . . . . . . . . . . . . . . 513
29.6 Reinforcement Learning . . . . . . . . . . . . . . . . . . . . . . . 515
29.6.1 Learning Optimal Policies . . . . . . . . . . . . . . . . . . 515
29.6.2 TD-GAMMON . . . . . . . . . . . . . . . . . . . . . . . . . 522
29.6.3 Other Applications . . . . . . . . . . . . . . . . . . . . . . 523
29.7 Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524
30 Natural Languages and Natural Scenes 533
30.1 Natural Language Processing . . . . . . . . . . . . . . . . . . . . 533
30.1.1 Grammars and Parsing Algorithms . . . . . . . . . . . . . 534
30.1.2 Statistical NLP . . . . . . . . . . . . . . . . . . . . . . . . 535
30.2 Computer Vision . . . . . . . . . . . . . . . . . . . . . . . . . . . 539
30.2.1 Recovering Surface and Depth Information . . . . . . . . 541
30.2.2 Tracking Moving Objects . . . . . . . . . . . . . . . . . . 544
30.2.3 Hierarchical Models . . . . . . . . . . . . . . . . . . . . . 548
30.2.4 Image Grammars . . . . . . . . . . . . . . . . . . . . . . . 555
31 Intelligent System Architectures 561
31.1 Computational Architectures . . . . . . . . . . . . . . . . . . . . 563
31.1.1 Three-Layer Architectures . . . . . . . . . . . . . . . . . . 563
31.1.2 Multilayered Architectures . . . . . . . . . . . . . . . . . 563
31.1.3 The BDI Architecture . . . . . . . . . . . . . . . . . . . . 569
31.1.4 Architectures for Groups of Agents . . . . . . . . . . . . . 572
31.2 Cognitive Architectures . . . . . . . . . . . . . . . . . . . . . . . 576
31.2.1 Production Systems . . . . . . . . . . . . . . . . . . . . . 576
31.2.2 ACT-R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578
31.2.3 SOAR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 581
VIII Modern AI: Today and Tomorrow 589
32 Extraordinary Achievements 591
10