Natural Intelligence in Artificial Creatures

284 Pages·2004·1.3 MB·English

Natural Intelligence
in Artificial Creatures
––––––––––
Christian Balkenius
Lund University Cognitive Studies 37
1995
First edition
©1995 byChristianBalkenius
Allrights reserved.Nopartofthispublicationmaybereproduced,stored inaretrievalsys-
tem,ortransmitted,inanyformorbyanymeans, electronic,mechanical,photocopying,re-
cording orotherwisewithout theprior permissionofthecopyrightowner, except byare-
viewer whomayquote brief passages inareview.
Printed inSweden byLunds Offset
ISBN 91-628-1599-7
ISSN 1101-8453
ISRN LUHFDA/HFKO--1004--SE
Lund UniversityCognitiveStudies 37
Tomyfriends and enemies

Table of Contents
1. Introduction 9
2. Biological Learning 15
3. Design Principles 47
4. Reactive Behavior 79
5. Modeling Learning 127
6. Motivation and Emotion 161
7. Perception 185
8. Spatial Orientation 207
9. Cognition 227
10. Conclusion 245
Appendices 261

Chapter 1
Introduction
It is the goal of several sciences to construct models of behavior and cognition.
Two fundamental questions for all such endeavors are: What mechanisms are re-
quired to support cognitiveprocessesin an animal or arobot. How do such mech-
anisms interact with each other? This book is an attempt to study these questions
within the field ofbehavior-basedsystems and artificialneural networks.
The overall task will be to construct complete, artificial nervous systems for
simulated artificial creatures. This enterprise will take as its starting point studies
made of biological systems within ethology and animal learning theory. We will
also consider many ideas from neurobiology and psychology, as well as from be-
havior-basedroboticsand control theory. All these areas have valuableinsightsto
contributetothe understandingofcognition.
Ethology has stressed the importance of innate fixed-action patterns, or in-
stincts, in the explanation of behavior. Another significant contribution is the de-
mandthatbehaviorshould bestudied inthenatural habitat ofananimal. Thisleads
toaview ofbehaviorandcognitionwhich isvery differentfrom theonesuggested
by animal learning theory. This latter theory attempts to understand the basis of
learning by observing the behavior of animals in laboratory experiments. It sug-
gests aview of cognition that is complementary to that offered by ethology, since
it stresses the role of learning rather than innate mechanisms. However, as more
empiricaldata become available,the models inboth ethologyand animal learning
theory graduallyconvergeonwhat maybecome asubstantiallymore unified theo-
ryofanimal learningand behavior.
ChristianBalkenius,1995, NaturalIntelligenceinArtificialCreatures,LundUniversityCognitiveStudies37,1995.
10 – Introduction
Other important insights about the mechanisms of cognition are offered by
neurobiology and psychology. The goal of neurobiology is to uncover the neuro-
physiological mechanisms of the nervous system from the neural level and up.
Much research within this tradition investigates the properties of individual neu-
rons, however, but here we will mainly consider models at a system level. Such
models are closer topsychologywhere anumber ofrelevantmodels can befound.
Inthis book, wewill considerresults from both thebehavioralandcognitivetradi-
tions inpsychology.Althoughwedonot adhere tothe views ofthe behavioralpo-
sition, much oftheterminologywewill useoriginatesfrom that tradition.Howev-
er, we will mainly look at phenomena typically studied in the more cognitive
approach,such asexpectancy,categorization,planningand problem solving.
From the engineering sciences we will borrow ideas from both behavior-based
roboticsandcontrol theory. Inmanyrespects,behavior-basedroboticsisthecoun-
terpart of ethology within robotics. Many of the models within this area are very
similar to those proposed in ethology. The important difference is, of course, that
behavior-basedroboticsattemptstobuild working robots, and isnot anattempt to
study biological systems. A number of concepts from control theory will also be
used in this book. The most important one is the view of an animal as engaged in
closed-loopinteractionwith the environment.
Somewhere in the middle of these fields is cognitive science with the ambition
to cut across the boundaries of these more traditional approaches (Norman 1990).
The present book is such an attempt to combine ideas from all these different are-
as.
This book hasthree goals. Thefirst istoidentifythesystems requiredinacom-
plete, artificial creature. We will argue that such a creature requires a large set of
interacting systems. Some of these are fixed, while others must include different
types of learning mechanisms. Our main task will be to identify these systems,
rather than give any final solutionstotheir operation.Wewill, however,take care
toconstructfully working miniaturemodels ofall the proposedsystems.
The second goal istoinvestigatehow the differentsystems should interactwith
each other to make the overall behavior of the creature consistent. Many different
models have been proposed in the various areas we will consider, and our attempt
will be to make an inventory of these different mechanisms. Again, we will pro-
pose anumber offully worked out mechanisms.
Finally, we want to map out the way for more cognitive abilities, such as plan-
ning and problem solving. We believe that an overall emphasis on the concept of
expectationswill promote atransitiontosuch abilities.
Intaking adesign perspectivetoanimal behaviorandlearning,wewillconsider
howtoconstructsystems thatproduce sensiblecoherentbehaviorrather thantryto
explain behavioral data. If successful, this approach should give us insights about
why real animals are constructedasthey are. This requiresthat the componentsof
the model are developedtoalevel where they can successfullyoperate together.
Introduction – 11
The model proposedhere will be based on alarge set of findingswithin animal
learning theory, but our goal is not to settle any disputes about animal learning or
behavior. Instead, the aim of this book is to construct a set of mechanisms that
reflect those found in biological systems. The goal is, thus, to find a consistent
model of a complete creature. Since the model we will propose is computational,
consistencywillalways betheprime condition,andagreementwithempiricaldata
only a secondary requirement. Of course, this does not mean that we will ignore
empirical data, but it will not be ultimately constraining. The creatures developed
will, infact, bemostly based onempiricalfindings,althoughitisnecessarytosim-
plify many details inorder toget the overall system tofunction.
Eventhough itwould obviouslybeinterestingtotrytoemulate neurophysiolog-
ical and behavioral data more closely, the current knowledge of the brain makes
suchanendeavorverydifficult,evenforaveryrestrictedsub-system.Toconstruct
an entirely realistic model of a complete nervous system based on our current
knowledge is clearly impossible. The proposed model can, thus, be compared to
real nervous systems on a functional level only. We believe, however, that the
functionalsub-systemswepropose must have parallelsinreal nervous systems.A
completemodel ofacreaturecan, therefore,beofgreat use intwo areas.
The first is in the study of biological systems where it can be used both to sug-
gest mechanisms to look for, and to give an understanding of the number of sys-
tems interacting with each other. We hope the model proposed in this book will
give the overall picture that isoften missing when specificabilitiesorsystems are
discussed. It should be kept in mind, however, that this book deals primarily with
artificialcreatures,andassuch, itcannot giveusanydirect model ofanyparticular
real animal. Such questionsare better handled byempiricalstudies.
The second area where the model can be used is within autonomous systems.
Since the model is detailed enough to be implemented in a computer, it can also
potentially be adapted for robotic control. This would very likely require many
changes within low level aspects of the model, but the overall structure would be
the same. The presented model can, thus, be seen as a framework for an autono-
mous agent.
Since itisthe overall picture that isour interest,wewill try touse asfew math-
ematicalconceptsaspossible,inorder tomakethetextmorecomprehensible.For-
mal specifications of all systems are given in the appendices, although there is
little formal treatmentofthemodel. Suchananalysiswould, ofcourse, beinterest-
ing, but is not the primary goal of this book. The reported simulations will, thus,
have toserve both asexamples,and asproof ofthe performanceofthe system.
Chapter 2presentsanoverviewofthe differentproblemsthat have tobesolved
inorder toconstructamodel ofacompletecreature.The emphasiswill beonvar-
ious results from animal learningtheory. The goal ofthis chapter istoshow that a
general learning system is not realistic from a biological perspective. We will ar-
gue that biological systems use many interacting systems for different abilities,
12 – Introduction
and the conclusion will be that it is necessary to take this into account if we want
toproduce anartificialsystem withsimilar abilities.Thischapter is,thus, intended
both as a presentation of the biological background and as an attempt to set the
goal for the model wewill bedevelopinginthe remainingchapters.
Chapter 3 gives a background to the design principles that will be used in the
construction of the model. We will briefly review a number of ideas from behav-
ior–basedroboticsand discuss how they can beused toconstrainthe design ofar-
tificial creatures. It is argued that the basic building block for artificial creatures
should be the behavior module, which represents a particular mapping from sen-
sors to effectors, that is, aparticular control strategy. It is suggested that behavior
modules can be combined into hierarchies called engagement modules, each of
which controlsoneparticulartaskofthecreature.Wealsointroducethetypeofar-
tificial neural network that is used for the artificial nervous systems of our crea-
tures. The chapter concludes with a concrete example of an artificial creature
which illustrates how neural networks can be used to control a simple body in a
simulatedenvironment.
Chapter 4 initiates the development of the model. We present a taxonomy of
different reactive behaviors and a number of elementary components that can be
used to construct them. We first discuss the directedness of behavior and identify
four general categories of behavior. Appetitive behavior is directed toward an at-
tractiveobject orsituation.Aversivebehaviorisdirectedaway from negativesitu-
ations. Exploratory behavior is directed toward stimuli that are novel in the envi-
ronment. Finally, we describe a class of neutral behaviors relating to objects that
are neither appetitivenor aversive.This classificationisastep away from asingle
hedonic dimension and it gives aricher framework for understanding reactive be-
havior. It becomes possible to distinguish between active avoidance used for es-
cape, passive avoidance used to inhibit inappropriate behavior, and neutral avoid-
ance used to negotiate obstacles. The new classification also captures the
difference between exploratory and appetitive behavior in a natural way. We
finally present a number of ways in which behavior modules can be coordinated
both sequentially and in parallel. The chapter concludes with an example of an
elementaryreactiverepertoirefor our model creature.
Chapter 5 discusses how adaptation can be included within and between
engagementmodules tocoordinatewhich behaviormodules should beactivatedor
inhibited. Starting from the two classical types of learning: instrumental and
classical conditioning, we present a new real-time model of conditioning that can
beused forboth types oflearning.Themodel combinesmany propertiesofearlier
two-process models of conditioning (Mowrer 1960, Gray 1975, Klopf 1988), but
has the additional ability to distinguish between appetitive, aversive, neutral and
unknown situations. It can, thus, select between the different types of behaviors
described in chapter 4. The model also shares many properties with other rein-