STANFORD ARTIFICIAL INTELLIGENCE PROJECT MEMO AIM-143. COMPUTER SClENCi ...

90 Pages·1998·6.63 MB·English

by  Stanford University Libraries

STANFORD
ARTIFICIAL INTELLIGENCE PROJECT
AIM-143.
COMPUTER SClENCi DEPARTMENT
. . !
CS-209
PROJECT TECHNICAL REPORT --
BY
John McCarthy, Arthur Samuel, and the Artificial Intelligence Project
/
EdwardIeigenbaum, Joshua and the
Heuristic Programming Project Staff
1971
ARPA Order No. 457
COMPUTER SCIENCE DEPARTMENT
UNIVERSITY
.

’
LI
I
ARTIFICIAL
STAWORD INTELLIGENCE PHOJECT MARCH 1971
MEMO AIM-143
L
Project ‘TechnIcal Report
i
i John McCarthy, Arthur Samuel, ano the Artlflclal lntelllgence Project
L and
Eakard Felgenbaum, Joshua Lederberg the Heuristic Programming
Project Staff,
i
ABSTRACT: An OvervIew IS presented of current research at Stanford
L
In artlflcial lntelllgsnce and heurlstk Prcgram~ing, This report Is
IarselY the text of a pr@posal to the Advanced Research ProJects
Agency for fiscal Years 1972-3,
f
L
part
i The research reported here was supported In by the Advanced
Research Projects Agency of the Office of the Secretary of Defense
under Contract SD-183 and In part by the Natbnal Institutes of
i Mental Health under Grant PHS MH 066-44-08,
i
The views and conclusions contalned ln thjs document are_ those of the
i auth.ors and should not be Interpreted as necessarily representjng the
i off I-clal ~01 lclesl either expressed or lmpl led, of the Advanced
Rese:arch ProJects Agency or the U,S, Government,
1
Federal
Reproduced In the USA, Avallable from the Claarlnghouse for
Sclentiflc and Technlcal InformatIon, S~rlngfleldr Vlrglnla 22151,
$,95,
Price: full size copy, $3,001 mlcrof(ahe copy,
i

L
Table of Contents
Page
1, Artlflclrl InteIllgence ProJect too*oo~o~#@o.~o~o~to~~. 1
L 1,l Analysis of Algorithms ,~~ccr~cr,coro~oor,~,rr,rctrro( 3
1,2 M a c h l n o Translation ,c)rrr~or*tr*oteocc*~ rrtrr(rtr~o 7
L IF3 Interactlon with the Physlcal World cte.o.~*co~r*r~~ 11
1,3,5, Hand-Eye r(r*~rrrrr)rrr~~rrr@*~*~~~o . . •o**o~o~,~~~o 11
1,3,2 S p e e c h Reoagnltion rr~~*oror*o*r~~trrcct~~* 16
1 1,4 Heuristics ~((~tt@(#~te~tt*~t~~*tt~o~~~t*~~tt~~~~~* 17
L 1,4,1. Machine Learnlng ~)~*~t~~tt~t~ttoo~~***~~~o~~~~~~ 17
1,4,2 Automatic Deductlon c~~r*,*r,~*ttt~#c,,tttvo 17
1,5 Mathematical Theory of Computatlon r))trr~*,cttrtt@ 20
1,5,1 Recent Research O,~Ott*O~,*~ •*~*0~~~*..**~~*9$@ 20
I? 1,5,2 P r o p o s e d R e s e a r c h t~)*r,*tt,rttc,rr.,ttt~~ 21
1,6 Representation Theory ,r(rrr~.c~(r***t**e*~ l *~tt~torr~ 23
I 1,7 Computer SlmuIatlon of Belief Systems ~*t~~oo***o~t.~*)~ 24
L 1,8 Faclllties (),(to,ror(rtttrt~tt l tt**~oe~*e~***ttt(~ 29
,2 heurlstlc Programming Project ~~*t~,~oo*r*tooo**(*tr~, 32
1 2,l Introduction *~~to~o*t~ttt~~~oto*t/~t(~t~@t*to)*t()@ 32
I 2,2 Change-of Project Name .~tt~~:t~.~~*~~oo*t*~~~~t~~~t~~o 32
2,3 Proposed Work for New Contract Period ~~~ett(t~o~~~~~~ 33
2,4 laolrotsiH
Synopsis ~~~t.~~o~#t~to~~e~o~o~~tm~o~~(~ 38
-I 2,5 Views of Others Concerning Thls Research m.,r*c(e 39
2,6 Review of Work of the Current Period @~~~*oo~o*~t~~~ 40
as noltacllppA ot
2,7 Heuristic DENDRAL Chemistry:
1 Possible NIH Support ~**~t~~~~otot*~~oro~toto~~*ot~*) 41
c
2,8 Computer Facllltles ~~*tto~o@~*otrtB~*t***~rt~~r))rr 42
2,9 Budgetary Note Concsrnlng Computer Time ~.ct,~:c 43
! 2,10 Budgetary Note Concerning Personnel *~~~t((oo*ro~)o 43
i \
,3
B u d g e t *~~**~~~tm*~te~oo*t~to*t~D*~~o~~o~*tte(~*~**~)*( 47
3,1 Summary of Budgets for Contlnuatlon of SD-183 (FY 1972) 47
3,2 Summary of Budgets for Contlnuatlon of SO-183 (FY 1973) 48
3,3 Artlflclal Intelligence Budget ~a*t(~.o~~*otoooro*~(t~ 49
3,4 Heuristl c Programmlng Budget *~~~~~o~~~tooo~*s~t*~~t~ 51
i a
t- 4. Cognizant Personnel ~~~tt~tt~eto**~t*to*roorrt*c,pttr~~coor, 52
i Appendices
c
A, i Fubllcatlons of ProJect Members emtt~otoo*oo@~t*to~ot~t A-l
L 0” Artif l.alc1 InteIIlgence, Memos ttoo*o~e,o~~t~oo,(~t~t~o D-1
d
I 1
i

L
1, Artlflclal Intelligence Project
L
Artlflclnl Intel Ilgence 1s the experlmental and theoretical study of
perceptual and Intel (ectuai processes using computers, Its ultimate
processes
goa I Is to understand these well enough to make a computer
L perceive, understand and act In ways now OnlY possible for humans,
This understanding Is at present In a very prellmlnary state.
Nevertheless, progress In ldentlfylng and dupltcatlng Intellectual
mechanisms 1s being made and the range of problems that computers can
!
be n;ade to solve ls Increaslng, The understandlng so far achieved has
Important potential practical awllcatlons, The development of these
appllcatlons
is worth undertaklng,
I
The Stanford Artlflclal Intelligence Project Is concerned wlth both
the central problems of artlflclal lntelllsence and some related
L subflelds of computer science, The proposed structure of the Project
Is given In Figure 1, The scopes of some oontlnulng actlvltles have
been mod(f and two new research areas have been added: Analysis
of Algorithms and Machine Translation,
i
1 Flwre 1, Structure of the Stanford Artlflclal
Intelligence Project
1
---~-~c~IsII”II-----~-~~~~~~~~~~~~-~~~-~-~~~-~-~----“~-~ - - - - -
I I ! I I I I
I I I I I I
L
I -I-w- I CIICIIC- I 9-9-9 I WmwIII I LI-**I- I’"9-V-W I *
I l Interadtlon I I l Heurlstlcs I I I Mathematical I
I I with the I I I I 1 I Theory of I I
L I IPhysl~al World1 I I I I I Computa.t .ion I
I I -I-m--------“- I I I --cI-~~~-~zI I I I -,--LZLII-“LI7I I
I Binford,Feldman i Luckham,Samue( l Floyd, Knuth
I Kay, Samue I I Manna, McCarthy I
f
a I I I
i-
I I I . I
l’--,-,m ,,,,,1,L- -I-I-*-* I r-w-w- c--r-r-----
IAnalysIs I lMechanlcal I IPepresentatlon l I Models of I
i I-of I ITranslatbnl I Theory I I Cognltlve I
JAlgorlthmsl l I I I 1 Processes 1
I I
I ----w-I-L- I ---- 11 ---m-l I -LII-~-~~--~-~- I lL,1,,--J
i
Knuth Schank,Wllks McCarthy Colby
McCarthy
i
c
VqAnal~SIs of Atgorlthms" Is headed by Professor Donald Knuth and
dlrected to an understanding Of t h e quantltatlve behavior of
properties
particular algorithms, The of many algorithms that are of
central Importance to computer science are known Only In a
qualltatlve or crudely quantltatlve Way, Knuth and his group are
seuqinhcet
employing analytIcal to de_epen our knowledge of thls area,
be approached anew
The problem of machine translation will from two
sihT
dlrectlons: artlflclal Intel (Igence and llnsulstlcs, small
propose
project wll( Involve representatives of both dlsclpllnes who
to test their ideas lnltlally on a restrlcted formal language,
Interaction wlth the Physical World" Includes contlnulng projects On
computer VlSlOn and control, as well as speech recognltlon research.
Ourlng Prof, Feldman's sabbatloal I eave# (academic year 1970-71)
responslbll Ity for Hand-Eye research has passed to Drs, Thomas
Binford and Alan Kay, Work on speech recognltlon was curtailed with
but SI continuing nI the
the departure of Professor Reddy area Of
syntax-dlrected recognition,
Work on Heuristl?% continues In the areas of machine learning and
autonatlc deductlon, Board games such as Checkers and Go are the
primary test vehicles for Ideas In machine IearnIng, Theorem-
provlng Is the current obJectlve of our research On automatic
,noItcuded
,John McCarthy's Representation Theory work Will continue on
esplstemologlcal problems (I (em choosing a sultable representation
that ebircsed
for situations and the rules how sltuatlons change),
Research In Mathematical Theory of Computatfon Is expanding somewhat,
partly through other sources of SuPPort, A practical goal of this
work I S to replace certain time-consuming and uncertain program
debugging processes with formal proofs of the correctness of programs,
Thp work on Models of Cognltlve Processes shown In Figure 1 Is an
tceJorp tI
afflllated not Included In this proposal. wIII be
supported by the Natlonnl Instltutss Of Mental Health under Grant
MH06645-10,
Subsequent sections cover the proposed research in SOmeWhat more
detallr
1J ANALYSIS OF ALGORITHMS (Dona(d Knuth)
*~Analysls of algorlthmsn is a field of study directed to an
understanding of the behavior of particular algorithms, Two kinds of
problems are usua(ly Investigated:
A, Quantltatfve analysis of an ajgorlthm, nI this case the
a0g I Is usually to determlne the running time and/or memory space
requirements of a given algorithm, The determination of running time
can be done In an essentially machlne-independent manner by
exoresslng the algorithm In some machIne- Independent language (not
necessarl lY a formal language) and counting the number of times each
c a se
step Is executed, Usually these cwJqts lnolude a "worst
analySis" (the maxlmum number of times that the step can be
performed, taken over some specified set of inputs to the algorlthm)l
a "best case ana(ysW (the minlmum number of times), and a VyPical
case analysW (the average number of tlmes for a given distribution
of Inputs), It Is ln fact desirable to have complete information
about the dlstrlbutjon of the number of times, for a known
dlstrlbutlon of the I nput, whenever this can be worked Out, A
typlcal example of such an analysis is presented ln detal l In Cl, pp.
95~$92,
*flamltpV nI
B, Determlnatlon ot algorithms, this case the
goal is usually to find the "best posslbW algorlthm In a given
olass of algorithms, We set up some deflnltlan of ;';st possible"
which reflects, as realIstIcallY as possible, pertinent
characteristics of the hardware which IS to be associated with the
algorithm, A typIcal example of this sort of analuslst applied to
the problem of computing x*n with the fewest mu)tlpllcat[ons when n
‘IS fixed, Is discussed ln detal I In C2, PP, 401-4183,
Analyses of type A are usually employed when comparlng two different
algorithms that do the same job, to see which Is more sultable on a
some
particular oomputer for particular type of input data, Since
there Is usually more than one yaw to solve a problem, analyses of
'this type nac be very helpful in decldlng which of several algorithms
should be chosen. Occaslom( (Y type A analyses are also lncoroorated
rof example, “spectral ”tset
into the algorithms themselves! the
type
aigorlthm C2, pp, 93-96) carries out one of Iteration until it
fin-as that the data has been transformed enough to let another type
of :Iterat/on complete the Job In a reasonable amount of tjme,
epyt
It rr;ay seem that type B analyses are far superlor to A analyses;
we will have found the VmsV algorithm once and for al 1, Instead of
performing type A analyses of all algorithms in the class, But this
Is only true to a llmlted extent, since slight changes In the
poss.1 h 1s”
deflnltlon of "best aan slgnlflcantly affect which
algorithm Is beat, For example, X931 cannot be calculated (startIne
with the value of x) in fewer than 9 multlplicatlons, but lt oan be
evaluated wlth only 6 arithmetic operations If dlvlslon 1s allowed,
veYuyJK
and Kokovkln4chsrbak w. proved that the Gaussian
3
el lrr,lnatlon method for matrix InversIon uses the mlnlmum number of
arithmetic operations , provided that whole rows are always operated
h as
on at a time; but Strassen C41 recently discovered that
substantlal(y fewer operations are needed If the row restrlctlon Is
dropped,
Another problem with epyt B ana‘iyses 1s that, even When a simple
deflnltlon of "best possible" Is postulated, the determlnatlon of an
aptlwal algorlthn Is exceadlngly djfflcu(t, FQr example, the
basla
foIlowIng problems are among those not yet comp(ets(y resolved:
(a) The mlnjmum number of multlpllcatlons to compute xqn
glven x with n flxed,
cltemhtlra operatlons to etupmoc a
(b) The mlnlmum number of
general polynomial a(n)Vn + (,) + a(l) + a(0)t given xn fop fixed
values of the coeffjclents,
of steps needed to
(cl The mlnlmum number multiply two 0 I ven
binary nmblt numbers,
(d) The minimum number of steps needed to recognize whether a
giver string belongs to a given contextnfree language,
steps
(a) The mlnlmum number of needed to multlply two glven
n'x n matrIces, when n Is known,
(f) The mlnlmum number Of comparjson steps needed to sort n
elements,
Asy!rptotlc solutlorls are known for proolems (aI and u?, and the
solution to (b) Is known wlthln 1 or 2 operations, for “almost all”
polynomla)s; but In cases (~1, (d)r and (e) the known upper and lower
bounds for the desired quantltles are far apart, No a8ymptotlc
snosirapmoc
solution to problem (f) Is known when slmultaneous are
al low& The evidence In case (aI suggest strongly that the exact
a&her as a function of n has no simple form wh I ch wl I I ever be
dIscovered without exhaustive trial-and-error search, Furthermore,
the slmpllfled deflnltlons of "best posslbleq~ often fai 1 to represent
auff
lclently real lstlc sltuatlonsl for example, Items need not be
d:etros at la ,I
by means of comparisons they can be sorted by using
bjt InspectIon or by using ldentltles IIke
)bra(nlm = (a + b -
lb-a()/2
,deredisnoc other
If only the number of comparfsons Is tmPortant
charactetlstlcs of the aortlng problem (e,g,, the logical complex)ty
of the program and- fo eht atad s.tructures) are Ignored, Therefore
although type B analyses are extremely lnterestlng, type A analyses
pay
more often off In practlae,