Natural and Artificial Intelligence. Misconceptions About Brains and Neural Networks

671 Pages·1992·39.304 MB·English

by  A. de Callataÿ (Auth.)

New, Expanded Edition
NATURAL AND
ARTIFICIAL INTELLIGENCE
Misconceptions about Brains and Neural Networks
Armand M. de Callatay
Member of
IBM European Systems Research Institute
La Hulpe, Belgium
NORTH-HOLLAND
AMSTERDAM · LONDON · NEW YORK · TOKYO
NORTH-HOLLAND
ELSEVIER SCIENCE PUBLISHERS B.V.
Sara Burgerhartstraat 25
P.O. Box 211,1000 AE Amsterdam, The Netherlands
Title of original edition:
NATURAL AND ARTIFICIAL INTELLIGENCE
Processor Systems Compared to the Human Brain
This New, Expanded Edition:
ISBN: 0444 89081 5 (Hardbound)
ISBN: 0 444 89502 7 (Paperback)
© 1992 Elsevier Science Publishers B.V. All rights reserved.
No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by
any means, electronic, mechanical, photocopying, recording or otherwise, without the prior written permis-
sion of the publisher, Elsevier Science Publishers B.V, Copyright & Permissions Department, P.O. Box 521,
1000 AM Amsterdam, The Netherlands.
Special regulations for readers in the U.S.A. - This publication has been registered with the Copyright
Clearance Center Inc. (CCC), Salem, Massachusetts. Information can be obtained from the CCC about con-
ditions under which photocopies of parts of this publication may be made in the U.S.A. All other copyright
questions, including photocopying outside of the U.S.A., should be referred to the copyright owner, Elsevier
Science Publishers B.V.
No responsibility is assumed by the publisher for any injury and/or damage to persons or property as a
matter of products liability, negligence or otherwise, or from any use or operation of any methods, products,
instructions or ideas contained in the material herein.
This book is printed on acid-free paper.
Printed in The Netherlands
PI
How to Read the Book.
The summary of the book, preceded by a summary of the**new prologue, in-
troduces the topics.
This book describes mechanisms which are best explained by diagrams.
Readers who think visually could first look at the figures (a list of abbreviations
gives the acronyms).
Specialists should first read the new prologue in which warnings against
misconceptions are listed by research domain.
Others might first read the introduction for which a deep knowledge on
brains and computers is not needed.
Pages 27 to 455 are reproductions of the 1986 edition with the original
numbering for pages and figures. The reference numbers in the prologue are
preceded by the Utter P.
A few technical terms were ambiguous and have been changed in the ex-
panded edition (developed in the glossary):
• Nucleus is replaced by group.
• Gate is replaced by latch.
• Information retrieval is replaced by document retrieval.
• Behavior rule is replaced by invariant rule.
The appendices and the glossary have been completed and updated.
The history of the research and acknowledgements are given in appendix 3.
The preface of the first edition is put there.
The expanded version of this book was produced using the "ITC New Baskerville® " family
of Type I font programs on an IBM® mainframe computer using application software packages
that support Adobe's Postscript® page description language. IBM is a trade mark of International
Business Machines Corporation. Adobe, Postscript, and Type 1 are trademarks of Adobe Systems
Incorporated. ITC New Baskerville is a trademark of International Typeface Corporation.
Summary P3
Summary of the Expanded Sections
Brain architecture. Modelling brains is like designing a processing
plant. The engineer must connect many devices in an efficient way.
The brain designer has a large choice of components in the machinery
of regulation systems, instrumentation, neural networks, and artificial
intelligence. The design task is to reproduce natural intelligence in a robot
by integrating these devices. The book topic is "large scale architecture"
as defined in Minsky (1990).
An unusual brain model. At conferences, I sometimes hear: "See that
man: he really believes there are little switches in the brain". I explain here
why there is no alternative to the memorization by irreversible switches
(latches).
Implementing the switches requires a radically new view of the
brain. I am not the first to suggest most of the mechanisms shown here.
I may be the first to integrate all these unlikely features as the necessary
components of a model. Careful scientists may believe I like to imagine
and put together the most original, challenging, improbable, new ideas.
They would be right, but it also turns out that these mechanisms are
all required because each one cannot work without the others.
The book principles are opposed to the current views on brains. I
think those are misconceptions which have prevented researchers from
developing a fully fledged model of the brain.
Most new features of my solution were unlikely when I advanced
them. Several predictions of the model are now supported by plausible
biological mappings.
The impossible and the improbable. Discovering how brains work
is a detective job. Sherlock Holmes said: "When you have eliminated the
impossible, whatever remains, although improbable, must be the truth. " (Conan
Doyle, 1884). This is the method here.
The first version of the book described an improbable brain model
but did not show why the other paths were impossible. These explana-
tions might have helped researchers not to repeat the errors of the past.
The added prologue now describes the misconceptions in the 1960's
and after.
The aims of modeling. The purpose of a model is to prepare ex-
periments and formal theories for falsifications or existence proofs.
Models spread ideas. Many views on brains discussed to-day concern
topics introduced in my book and papers. They are sometimes pre-
sented under other names. I have listed these terms in an appendix
discussing other models and in the glossary.
P4 Summary
Misconceptions about brains and neural networks. In the prologue,
I try to shake the foundations of many current ideas.
• Should thought have an unifying principle?
• Should neuronal networks be homogenous?
• Are all-or-none switches biologically impossible?
• Must memory disappear in biological organisms continuously modi-
fied?
• Is the plasticity of neuronal maps incompatible with computer mod-
els?
• Are the neuronal computations mostly based on inhibition and
excitation?
• Is the topographical organization of brains necessary everywhere?
• Do we forget our previous habits and beliefs after having changed
them?
• Are our behaviors frequently perturbed by noise?
• Is the large variance of our movements incompatible with digital sys-
tems?
• Does machine behavior have less variance than that of animals?
• Does adaptive learning exclude irreversible memorization?
• Is symbolic computation incompatible with approximate reasoning?
• Is combinatorial explosion of cases unavoidable in symbolic systems?
• Is information distribution more reliable than redundancy?
• Do Lashley's experiments prove that brain memory must be distrib-
uted?
• Is addition of learned events impossible in conventional neural net-
works?
• Are grand-mother neurons impossible in the real nervous system?
• Are Gestalt phenomena incompatible with symbolic processing?
• Are attractor neural networks more reliable than grand-mother net-
works?
• Is instant learning rare?
• Are generalized rules necessary for reasoning?
• Is AI a rigid logic method?
• Is rote learning incompatible with understanding?
• Does intelligent classification need a teacher?
• Can classification be a continuous operation?
• Can a rhythmic system be non-oscillatory?
• Is the variance of reaction time incompatible with a rhythmic control?
• Does natural intelligence infer instead of deciding?
• Is making decisions a complex algorithm?
• Do decisions depend on the way the quantum wave collapses?
Summary P5
• Must we find mathematical formulae explaining thought?
• Is document retrieval a complex algorithm?
• Is reasoning by analogy a complex algorithm?
• Is intention a mental concept not implementable in hardware?
• Is consciousness not understandable as a mechanism?
• Do the limitations of expert systems prove that AI is an inadequate
model?
• Do Gödel's and Turing's limitations of mathematics and computers
prevent natural intelligence by machines?
• Are the primary elements of brain knowledge permanent things?
• Must we first find how similarities are computed?
• Is long-term memorization preceded by short-term memorization?
• Can parsing be a continuous operation?
• Is human behavior not stereotyped?
• Is spontaneous human behavior frequently efficient?
• Is human behavior optimized?
• Is it impossible that simple mechanisms combined produce intelli-
gence?
I explain in the book why the response to all these questions might
be "no".
Words in different disciplines have different meanings and cannot
be generalized in the same way. Clarifying the vocabulary will wipe away
most misconceptions. I do not claim to have the right terminology, but
I try to make clear the facts behind them.
Each discipline is mainly characterized by its methods. The methods
used here are not those of psychology, biology, or computer science.
Large scale architecture of mechanisms is an engineering method hav-
ing its own rules.
The parts of the model which were not described in the previous
edition are:
• the rôle of the basal ganglia for choosing and managing prepared
plans combined with the choice of a sensible behavior (appendix 1),
• a more formal description of symbolic neural networks by applicative
logic and reasoning by analogy (deCallatay, 1987c).
• a biological mapping which describes logical impulses by calcium im-
pulses (seen as calcium flashes), enabling control by modulators act-
ing on NMDA channels, and synchronization control by the recently
observed ß oscillations.
P6 Summary (1986)
Summary of the Book (1986)
The following theses are suggested:
How does the mind work? It works according to the laws of thought
already described in philosophy and rediscovered by scientists in Arti-
ficial Intelligence (AI). The knowledge represented by the "Physical
Symbols" of Newell and Simon corresponds to the Mental World of
Plato. Logic Programming is based on the syllogisms of Aristotle and
the "Mathesis" of Leibniz.
How are data stored in brain? Artificial intelligence represents data
by virtual pointers between symbols. In connectionist architectures,
these virtual relations are replaced by actual wires with potential gate
connections. The brain has a similar architecture.
How does the mental world connect with the physical world? De-
vices with servo-mechanisms and analog circuits provide an interface
between the symbolic processor and the physical world. Symbol ma-
nipulation must be phasic and indeed many fast rhythms have been
discovered in the central nervous system.
What does the brain learn? It learns "behavior rules": how to gen-
erate from experience an action producing an intended result in a
given context.
How does the brain learn? By opening gates to build new con-
nections between elements simultaneously activated; spine résorption
plays the role of an all-or-none switch. These links store events in the
form of production rules. They represent behavior by the body.
How can memory be permanent in a living organism where proteins
are frequently destroyed or added? By a homeostasis which maintains
spine shape.
How does the brain retrieve data? By pattern matching, the main
operation of AI programs: a group of neurons works as a recording
content addressable memory (RCAM) and executes an algorithm of
information retrieval. A concept is deduced from its properties as a
document is retrieved from its keywords. Fuzzy pattern matching is au-
tomatically done in the RCAMs.
How does the brain generate consistent behavior? It would appear
that applicative, functional or logic programming enforces the single
assignment rule which does not modify data and guarantees that de-
duced actions are "true", i.e. in conformity with possible events.
Learning mainly adds behavior episodes instead of modifying previous
knowledge. Irreversibility of spine résorption has the same property.
Simple rules induced from these complex episodes are axioms for the
brain inference machine. Deductions based on behavior functions and
Book summary P7
events explain such features of natural intelligence as insights, common
sense, and Piaget's developmental stages.
How does the brain make efficient decisions? The stimulus-response
(SR) model permits pleasure search and pain avoidance, but is too
simple to explain the complexities of behavior. Hence I modify the SR
model so that "stimulus" is replaced by "integrated result" and "re-
sponse" by "intention". The "decision rules", corresponding to the
"extra-pyramidal system", require only 1% of the human brain's volume.
As in Piaget's sensory-motor scheme, behavior rules are "assimilated",
and decision rules are "accommodated".
How can the brain, with its loose circuits, be more reliable than ro-
bots controlled by computer? Additive memory allows storage of re-
dundant data without logical complications and performance
degradation.
How can the brain, with a cycle time of some milliseconds, be faster
than large computers? The memory is integrated in distributed micro-
processors working in parallel.
What programs provide effective parallelism? Special algorithms
executable on new architectures. A mind rule is to think of one item
at a time: because this object of thought is a structure in the knowledge
network, the rule becomes: "process one item at a time in each category
of knowledge". Instead of sending knowledge to the computer central
processing unit, processing authorization is given to knowledge. Paral-
lel processing is thus automatically safe in a database machine because
knowledge is distributed among independent processors, each of which
can activate only one item.
How does coherent action arise from distributed processes? By hi-
erarchical control. As in a corporation, top intentions are distributed
and executed by departments which provide summary reports of their
activity. Each department works with the same logical methods, but with
different knowledge. My brain model includes many types of hierarchy,
each with rhythmic coordination of message exchange.
How can we prevent combinatorial explosion? When symbols are
combined, a result is classified in no more categories than there are
cells. The combinatorial explosion problem is replaced by a categori-
zation problem. Categorization algorithms like those allowed by the
Piaget's sensory-motor scheme are studied.
How are programs built? In the proposed computer architecture,
the operating systems and the learning programs are the result of pre-
wired hardware connections between components. Some networks, de-
signed by natural selection, parse temporal strings like a cascaded
augmented transition network, answer queries like a production-rule
system, or resolve problems like a modified Prolog interpreter. My
P8 Book summary
brain model is a hardware program which matches some of what is
known of brain neuroanatomy. It can be built with simple rules of
neurogenesis. This "innate" program spontaneously learns from expe-
rience and, using these learned rules, may deduce which behavior
achieves a given intention.
My hybrid model shows how different theories of brain functions
are integrated and how different kinds of storage and learning methods
cooperate.
The cerebral cortex stores and retrieves behavior rules. The hippo-
campal formation retrieves the episodes most similar to a new event and
enables the recording of this new event in neocortical areas well-
connected with the areas that had recorded the former episodes. The
cerebellum inner nuclei use servo-mechanisms to stop movements or
ensure equilibrium. To offset the long computation delay of neural
circuits, the feedback is computed by the cerebellar cortex which fore-
casts movement results. Some brain diseases are analogous to hardware
dysfunctions. To some degree, my model posits neurophysiological
functions for which there are no evidence.
How can the theories in my model be tested? Architecture consist-
ency may be verified with simulations of robot controllers more com-
plex than those made so far. Biological mappings may be checked by
designing experiments proving that the model analogies do not corre-
spond to brain operations.
P23
LIST OF ILLUSTRATIONS
Figure PI. Brain models: P27.
Figure P2. Separation of recognition and adaptive response. P30.
Figure P3. Separation of feature extraction and categorization. P31.
Figure P4. Brain model overview. P32.
Figure P5. Main control loops. P34.
Figure P6. Representations of neural networks P38.
Figure P7. Types of neural networks. P40.
Figure P8. Estimation of present value. P41.
Figure P9. Analog computers and neural networks. P42.
Figure P10. Principles of neural networks. P44.
Figure Pll. Self organization of neural networks. P49.
Figure P12. Entropy of information. P51.
Figure PI3. Knowledge and information. P53.
Figure PI4. Short and long term memories. P54.
Figure PI5. Attractor and circular neural networks. P57.
Figure PI 6. Processing in a digital neural layer. P60.
Figure PI7. Code conversion with several outputs. P61.
Figure PI8. Code conversion in two layers. P62.
Figure P19. Classifier. P64.
Figure P20. Chandelier cell. P66.
Figure P21. Computation in neural and neuronal networks. P76.
Figure P22. Receptors and channels. P77.
Figure P23. Stability of a synaptic plate. P78.
Figure P24. Transmitters in cerebellum. P79.
Figure P25. History of the size of a floating synapse: P81.
Figure P26. Risks of false recognitions. P83.
Figure P27. Synaptic size regulated by homeostasis. P87.
Figure P28. Neuronal mode of firing. P89.
Figure P29. Information transfer. P91.
Figure P30. Calcium flash. P93.
Figure P31. Central neuromodulators. P95.
Figure P32. Synchronization of remote cortical areas. P97.
Figure P33. Authorization and suppression. PI00.
Figure P34. Principles of servo controllers. PI03.
Figure P35. Dynamic response of a thermostat. P104.
Figure P36. Controllers based on qualitative physics. PI07.
Figure P37. A problem for large fast animals. PI08.
Figure P38. Modes of muscular control. PI 10.
Figure P39. Computation of temporal data. PI 12.
Figure P40. Control of limbs. PI 14.
Figure P41. Modules automatically processed in parallel. PI26.
Figure P42. Addition of an intermediate layer. PI 29.
Figure P43. Analog and symbolic controllers. PI 31.
Figure P44. Circular rhythmic oriented network of processors systems PI36.
Figure P45. Representation of stacked variables in NNs. P140.
Figure P46. Resolution in symbolic neural networks. P141.
Figure P47. Timing of actions, sensations and brain activities. PI46.
Figure 1. Three types of computation. 29.
Figure 2. Computation principles integrated in the model: 30.
Figure 3. Three main stages of processing. 31.
Figure 4. Software and hardware pointers: 32.
Figure 5. Schematic neuron: 35.
Figure 6. Connections between neurons: 36.