The Handbook of Brain Theory and Neural Networks

1309 Pages·1998·11.42 MB·English

by  Michael A. Arbib

The Handbook of
Brain Theory
and Neural Networks
This Page Intentionally Left Blank
The Handbook of
Brain Theory
and Neural Networks
Second Edition
EDITED BY
Michael A. Arbib
EDITORIAL ADVISORY BOARD
Shun-ichiAmari(cid:127)JohnBarnden(cid:127)AndrewBarto(cid:127)RonaldCalabrese
AvisCohen(cid:127)Joaqu´ınFuster(cid:127)StephenGrossberg(cid:127)JohnHertz
MarcJeannerod(cid:127)MitsuoKawato(cid:127)ChristofKoch(cid:127)WolfgangMaass
JamesMcClelland(cid:127)KennethMiller(cid:127)TerrenceSejnowski
NoelSharkey(cid:127)DeLiangWang
EDITORIAL ASSISTANT
PrudenceH.Arbib
ABradfordBook
THE MIT PRESS
Cambridge,Massachusetts
London,England
(cid:1)2003MassachusettsInstituteofTechnology
Allrightsreserved.Nopartofthisbookmaybereproducedinanyformbyanyelectronicor
mechanicalmeans(includingphotocopying,recording,orinformationstorageandretrieval)
withoutpermissioninwritingfromthepublisher.
ThisbookwassetinTimesRomanbyImpressionsBookandJournalServices,Inc.,Madison,
Wisconsin,andwasprintedandboundintheUnitedStatesofAmerica.
LibraryofCongressCataloging-in-PublicationData
Thehandbookofbraintheoryandneuralnetworks/MichaelA.Arbib,
editor—2nded.
p. cm.
“ABradfordbook.”
Includesbibliographicalreferencesandindex.
ISBN0–262–01197–2
1.Neuralnetworks(Neurobiology)—Handbooks,manuals,etc.
2.Neuralnetworks(Computerscience)—Handbooks,manuals,etc.
I.Arbib,MichaelA.
QP363.3.H36 2002
612.8(cid:2)2—dc21 2002038664
CIP
Contents
PrefacetotheSecondEdition ix II.3. Brain,Behavior,andCognition 31
PrefacetotheFirstEdition xi NeuroethologyandEvolution 31
HowtoUseThisBook xv MammalianBrainRegions 34
CognitiveNeuroscience 37
Part I: Background: The Elements of
II.4. Psychology,Linguistics,and
Brain Theory and Neural Networks 1 ArtificialIntelligence 40
Psychology 40
HowtoUsePartI 3
LinguisticsandSpeechProcessing 42
I.1. IntroducingtheNeuron 3
ArtificialIntelligence 44
TheDiversityofReceptors 4
II.5. BiologicalNeuronsandNetworks 47
BasicPropertiesofNeurons 4
BiologicalNeuronsandSynapses 47
ReceptorsandEffectors 7
NeuralPlasticity 49
NeuralModels 7
NeuralCoding 52
MoreDetailedPropertiesofNeurons 9
BiologicalNetworks 54
I.2. LevelsandStylesofAnalysis 10
II.6. DynamicsandLearninginArtificialNetworks 55
AHistoricalFragment 10
DynamicSystems 55
Brains,Machines,andMinds 11
LearninginArtificialNetworks 58
LevelsofAnalysis 12
ComputabilityandComplexity 64
SchemaTheory 13
II.7. SensorySystems 65
I.3. DynamicsandAdaptationinNeuralNetworks 15 Vision 65
DynamicSystems 15 OtherSensorySystems 70
Continuous-TimeSystems 15 II.8. MotorSystems 71
Discrete-TimeSystems 16 RoboticsandControlTheory 71
Stability,LimitCycles,andChaos 16 MotorPatternGenerators 73
HopfieldNets 17 MammalianMotorControl 74
AdaptationinDynamicSystems 18 II.9. Applications,Implementations,andAnalysis 77
AdaptiveControl 18 Applications 77
PatternRecognition 18 ImplementationandAnalysis 78
AssociativeMemory 19
LearningRules 19 Part III: Articles 81
HebbianPlasticityandNetwork
ThearticlesinPartIIIarearrangedalphabeticallybytitle.
Self-Organization 19
Toretrievearticlesbyauthor,turntothecontributorslist,
Perceptrons 20
whichbeginsonpage1241.
NetworkComplexity 20
GradientDescentandCreditAssignment 21
ActionMonitoringandForwardControlof
Backpropagation 21
Movements 83
ACautionaryNote 22
Activity-DependentRegulationofNeuronal
Envoi 23
Conductances 85
Part II: Road Maps: A Guided Tour of AdaptiveResonanceTheory 87
AdaptiveSpikeCoding 90
Brain Theory and Neural Networks 25
Amplification,Attenuation,andIntegration 94
HowtoUsePartII 27 AnalogNeuralNets:ComputationalPower 97
II.1. TheMeta-Map 27 AnalogVLSIImplementationsofNeuralNetworks 101
II.2. GroundingModelsofNeuronsandNetworks 29 Analogy-BasedReasoningandMetaphor 106
GroundingModelsofNeurons 29 ArmandHandMovementControl 110
GroundingModelsofNetworks 31 ArtificalIntelligenceandNeuralNetworks 113
vi Contents
AssociativeNetworks 117 DendriticLearning 320
AuditoryCortex 122 DendriticProcessing 324
AuditoryPeripheryandCochlearNucleus 127 DendriticSpines 332
AuditorySceneAnalysis 132 DevelopmentofRetinotectalMaps 335
AxonalModeling 135 DevelopmentalDisorders 339
AxonalPathFinding 140 DiffusionModelsofNeuronActivity 343
Backpropagation:GeneralPrinciples 144 DigitalVLSIforNeuralNetworks 349
BasalGanglia 147 DirectionalSelectivity 353
BayesianMethodsandNeuralNetworks 151 DissociationsBetweenVisualProcessingModes 358
BayesianNetworks 157 Dopamine,Rolesof 361
BiologicallyInspiredRobotics 160 DynamicLinkArchitecture 365
BiophysicalMechanismsinNeuronalModeling 164
DynamicRemapping 368
BiophysicalMosaicoftheNeuron 170
DynamicsandBifurcationinNeuralNets 372
BrainSignalAnalysis 175
DynamicsofAssociationandRecall 377
Brain-ComputerInterfaces 178
Echolocation:CochleotopicandComputationalMaps
CanonicalNeuralModels 181
381
CerebellumandConditioning 187
EEGandMEGAnalysis 387
CerebellumandMotorControl 190
Electrolocation 391
Cerebellum:NeuralPlasticity 196
EmbodiedCognition 395
ChainsofOscillatorsinMotorandSensorySystems 201
EmotionalCircuits 398
ChaosinBiologicalSystems 205
EnergyFunctionalsforNeuralNetworks 402
ChaosinNeuralSystems 208
EnsembleLearning 405
CognitiveDevelopment 212
EquilibriumPointHypothesis 409
CognitiveMaps 216
Event-RelatedPotentials 412
CognitiveModeling:PsychologyandConnectionism 219
EvolutionandLearninginNeuralNetworks 415
CollectiveBehaviorofCoupledOscillators 223
EvolutionofArtificialNeuralNetworks 418
CollicularVisuomotorTransformationsforGaze
EvolutionofGeneticNetworks 421
Control 226
EvolutionoftheAncestralVertebrateBrain 426
ColorPerception 230
Eye-HandCoordinationinReachingMovements 431
CommandNeuronsandCommandSystems 233
FaceRecognition:NeurophysiologyandNeural
CompetitiveLearning 238
Technology 434
CompetitiveQueuingforPlanningandSerial
FaceRecognition:PsychologyandConnectionism 438
Performance 241
FastVisualProcessing 441
CompositionalityinNeuralSystems 244
FeatureAnalysis 444
ComputingwithAttractors 248
Filtering,Adaptive 449
ConceptLearning 252
Forecasting 453
Conditioning 256
GaborWaveletsandStatisticalPatternRecognition 457
ConnectionistandSymbolicRepresentations 260
GaitTransitions 463
Consciousness,NeuralModelsof 263
GaussianProcesses 466
ConstituencyandRecursioninLanguage 267
ContourandSurfacePerception 271 GeneralizationandRegularizationinNonlinearLearning
ConvolutionalNetworksforImages,Speech,andTime Systems 470
Series 276 GENESISSimulationSystem 475
CooperativePhenomena 279 GeometricalPrinciplesinMotorControl 476
CorticalHebbianModules 285 GlobalVisualPatternExtraction 482
CorticalMemory 290 GraphicalModels:ParameterLearning 486
CorticalPopulationDynamicsandPsychophysics 294 GraphicalModels:ProbabilisticInference 490
CovarianceStructuralEquationModeling 300 GraphicalModels:StructureLearning 496
CrustaceanStomatogastricSystem 304 GraspingMovements:VisuomotorTransformations 501
DataClusteringandLearning 308 Habituation 504
DatabasesforNeuroscience 312 Half-CenterOscillatorsUnderlyingRhythmic
DecisionSupportSystemsandExpertSystems 316 Movements 507
Contents vii
HebbianLearningandNeuronalRegulation 511 MotorPrimitives 701
HebbianSynapticPlasticity 515 MotorTheoriesofPerception 705
HelmholtzMachinesandSleep-WakeLearning 522 MultiagentSystems 707
HemisphericInteractionsandSpecialization 525 MuscleModels 711
HiddenMarkovModels 528 Neocognitron:AModelforVisualPattern
HippocampalRhythmGeneration 533 Recognition 715
Hippocampus:SpatialModels 539 Neocortex:BasicNeuronTypes 719
HybridConnectionist/SymbolicSystems 543 Neocortex:ChemicalandElectricalSynapses 725
IdentificationandControl 547 NeuralAutomataandAnalogComputationalComplexity
ImagingtheGrammaticalBrain 551 729
ImagingtheMotorBrain 556 NeuroanatomyinaComputationalPerspective 733
ImagingtheVisualBrain 562 Neuroethology,Computational 737
Imitation 566 Neuroinformatics 741
IndependentComponentAnalysis 569 Neurolinguistics 745
InformationTheoryandVisualPlasticity 575 NeurologicalandPsychiatricDisorders 751
Integrate-and-FireNeuronsandNetworks 577 NeuromanifoldsandInformationGeometry 754
InvertebrateModelsofLearning:Aplysiaand NeuromodulationinInvertebrateNervousSystems 757
Hermissenda 581 NeuromodulationinMammalianNervousSystems 761
IonChannels:KeystoNeuronalSpecialization 585 NeuromorphicVLSICircuitsandSystems 765
KalmanFiltering:NeuralImplications 590 NEURONSimulationEnvironment 769
LaminarCorticalArchitectureinVisualPerception 594 NeuropsychologicalImpairments 773
LanguageAcquisition 600 Neurosimulation:ToolsandResources 776
LanguageEvolutionandChange 604 NMDAReceptors:Synaptic,Cellular,andNetwork
LanguageEvolution:TheMirrorSystemHypothesis 606 Models 781
LanguageProcessing 612 NSLNeuralSimulationLanguage 784
LayeredComputationinNeuralNetworks 616 ObjectRecognition 788
LearningandGeneralization:TheoreticalBounds 619 ObjectRecognition,Neurophysiology 792
LearningandStatisticalInference 624 ObjectStructure,VisualProcessing 797
LearningNetworkTopology 628 OcularDominanceandOrientationColumns 801
LearningVectorQuantization 631 OlfactoryBulb 806
LesionedNetworksasModelsofNeuropsychological OlfactoryCortex 810
Deficits 635 OptimalSensoryEncoding 815
LimbGeometry,NeuralControl 638 OptimalityTheoryinLinguistics 819
LocalizedVersusDistributedRepresentations 643 Optimization,Neural 822
Locomotion,Invertebrate 646 OptimizationPrinciplesinMotorControl 827
Locomotion,Vertebrate 649 OrientationSelectivity 831
LocustFlight:ComponentsandMechanismsinthe OscillatoryandBurstingPropertiesofNeurons 835
Motor 654 PACLearningandNeuralNetworks 840
MarkovRandomFieldModelsinImageProcessing 657 PainNetworks 843
Memory-BasedReasoning 661 PastTenseLearning 848
MinimumDescriptionLengthAnalysis 662 PatternFormation,Biological 851
ModelValidation 666 PatternFormation,Neural 859
ModularandHierarchicalLearningSystems 669 PatternRecognition 864
MotionPerception:ElementaryMechanisms 672 PerceptionofThree-DimensionalStructure 868
MotionPerception:Navigation 676 Perceptrons,Adalines,andBackpropagation 871
Motivation 680 PerspectiveonNeuronModelComplexity 877
MotoneuronRecruitment 683 Phase-PlaneAnalysisofNeuralNets 881
MotorControl,BiologicalandTheoretical 686 PhilosophicalIssuesinBrainTheoryand
MotorCortex:CodingandDecodingofDirectional Connectionism 886
Operations 690 PhotonicImplementationsofNeurobiologicallyInspired
MotorPatternGeneration 696 Networks 889
viii Contents
PopulationCodes 893 SpeechProduction 1072
Post-HebbianLearningAlgorithms 898 SpeechRecognitionTechnology 1076
PotentialFieldsandNeuralNetworks 901 SpikingNeurons,Computationwith 1080
PrefrontalCortexinTemporalOrganizationofAction SpinalCordofLamprey:GenerationofLocomotor
905 Patterns 1084
PrincipalComponentAnalysis 910 StatisticalMechanicsofGeneralization 1087
ProbabilisticRegularizationMethodsforLow-Level StatisticalMechanicsofNeuralNetworks 1090
Vision 913 StatisticalMechanicsofOn-lineLearningand
ProgrammableNeurocomputingSystems 916 Generalization 1095
Prosthetics,MotorControl 919 StatisticalParametricMappingofCorticalActivity
Prosthetics,Neural 923 Patterns 1098
Prosthetics,SensorySystems 926 StereoCorrespondence 1104
PursuitEyeMovements 929 StochasticApproximationandEfficientLearning 1108
Q-LearningforRobots 934 StochasticResonance 1112
RadialBasisFunctionNetworks 937 StructuredConnectionistModels 1116
RateCodingandSignalProcessing 941 SupportVectorMachines 1119
ReachingMovements:ImplicationsforComputational SynapticInteractions 1126
Models 945 SynapticNoiseandChaosinVertebrateNeurons 1130
ReactiveRoboticSystems 949 SynapticTransmission 1133
Reading 951 Synchronization,BindingandExpectancy 1136
RecurrentNetworks:LearningAlgorithms 955 SynfireChains 1143
RecurrentNetworks:NeurophysiologicalModeling 960 SyntheticFunctionalBrainMapping 1146
ReinforcementLearning 963 SystematicityofGeneralizationsinConnectionist
ReinforcementLearninginMotorControl 968 Networks 1151
RespiratoryRhythmGeneration 972 TemporalDynamicsofBiologicalSynapses 1156
Retina 975 TemporalIntegrationinRecurrentMicrocircuits 1159
RobotArmControl 979 TemporalPatternProcessing 1163
RobotLearning 983 TemporalSequences:LearningandGlobalAnalysis 1167
RobotNavigation 987 TensorVotingandVisualSegmentation 1171
RodentHeadDirectionSystem 990 Thalamus 1176
SchemaTheory 993 UniversalApproximators 1180
ScratchReflex 999 UnsupervisedLearningwithGlobalObjective
Self-OrganizationandtheBrain 1002 Functions 1183
Self-OrganizingFeatureMaps 1005 Vapnik-ChervonenkisDimensionofNeuralNetworks
SemanticNetworks 1010 1188
SensorFusion 1014 Vestibulo-OcularReflex 1192
SensorimotorInteractionsandCentralPattern VisualAttention 1196
Generators 1016 VisualCortex:AnatomicalStructureandModelsof
SensorimotorLearning 1020 Function 1202
SensoryCodingandInformationTransmission 1023 VisualCourseControlinFlies 1205
SequenceLearning 1027 VisualScenePerception,Neurophysiology 1210
Short-TermMemory 1030 VisualSceneSegmentation 1215
SiliconNeurons 1034 VisuomotorCoordinationinFrogandToad 1219
SimulatedAnnealingandBoltzmannMachines 1039 VisuomotorCoordinationinSalamander 1225
Single-CellModels 1044 Winner-Take-AllNetworks 1228
SleepOscillations 1049 Ying-YangLearning 1231
SomatosensorySystem 1053
Somatotopy:PlasticityofSensoryMaps 1057
SoundLocalizationandBinauralProcessing 1061 EditorialAdvisoryBoard 1239
SparseCodinginthePrimateCortex 1064 Contributors 1241
SpeechProcessing:Psycholinguistics 1068 SubjectIndex 1255
Preface to the Second Edition
Like the first edition, which it replaces, this volume is inspired by two great questions:
“Howdoesthebrainwork?”and“Howcanwebuildintelligentmachines?”Asinthefirst
edition, the heart of the book is a set of close to 300 articles in Part III which cover the
wholespectrumofBrainTheoryandNeuralNetworks.Tohelpreadersorientthemselves
withrespecttothiscornucopia,IhavewrittenPartItoprovidetheelementarybackground
on the modeling of both brains and biological and artificial neural networks, and Part II
toprovideaseriesofroadmapstohelpreadersinterestedinaparticulartopicsteerthrough
thePartIIIarticlesonthattopic.Moreonthemotivationandstructureofthebookcanbe
foundinthePrefacetotheFirstEdition,whichisreproducedafterthis.Ialsorecommend
readingthesection“HowtoUseThisBook”—onereaderofthefirsteditionwhodidnot
do so failed to realize that the articles in Part III were in alphabetical order, or that the
Contributorslistletsonelocateeacharticlewrittenbyagivenauthor.
The reader new to the study of Brain Theory and Neural Networks willfind it wiseto
readPartIfororientationbeforejumpingintoPartIII,whereasmoreexperiencedreaders
will find most of Part I familiar. Many readers will simply turn to articles in Part III of
particular interest at a given time. However, to help readers who seek amoresystematic
view of a particular subfield of Brain Theory and Neural Networks, Part II provides 22
Road Maps, each providing an essay linking most of the articles on a given topic. (Isay
“most”becausethethresholdissubjectivefordecidingwhenaparticulararticlehasmore
than a minor mention of the topic in a Road Map.) The Road Maps are organized into8
groupsinPartIIasfollows:
GroundingModelsofNeuronsandNetworks
GroundingModelsofNeurons
GroundingModelsofNetworks
Brain,Behavior,andCognition
NeuroethologyandEvolution
MammalianBrainRegions
CognitiveNeuroscience
Psychology,Linguistics,andArtificialIntelligence
Psychology
LinguisticsandSpeechProcessing
ArtificialIntelligence
BiologicalNeuronsandNetworks
BiologicalNeuronsandSynapses
NeuralPlasticity
NeuralCoding
BiologicalNetworks
DynamicsandLearninginArtificialNetworks
DynamicSystems
LearninginArtificialNetworks
ComputabilityandComplexity
SensorySystems
Vision
OtherSensorySystems
MotorSystems
RoboticsandControlTheory
MotorPatternGenerators
MammalianMotorControl