Neural Networks and Intellect: Using Model-Based Concepts

496 Pages·2000·4.21 MB·English

by  Leonid I. Perlovsky

Neural Networks and Intellect:
Using Model-Based Concepts
Leonid I. Perlovsky
OXFORD UNIVERSITY PRESS
N N I
EURAL ETWORKS AND NTELLECT
This page intentionally left blank
N N I
EURAL ETWORKS AND NTELLECT
Using Model-Based Concepts
Leonid I. Perlovsky
•
New York Oxford
OXFORD UNIVERSITY PRESS
2001
OxfordUniversityPress
Oxford NewYork
Athens Auckland Bangkok Bogota´ BuenosAires Calcutta
CapeTown Chennai DaresSalaam Delhi Florence HongKong Istanbul
Karachi KualaLumpur Madrid Melbourne MexicoCity Mumbai
Nairobi Paris Sa˜oPaulo Shanghai Singapore Taipei Tokyo Toronto Warsaw
andassociatedcompaniesin
Berlin Ibadan
Copyright©2001byOxfordUniversityPress,Inc.
PublishedbyOxfordUniversityPress,Inc.,
198MadisonAvenue,NewYork,NewYork,10016
http://www.oup-usa.org
OxfordisaregisteredtrademarkofOxfordUniversityPress
Allrightsreserved.Nopartofthispublicationmaybereproduced,
storedinaretrievalsystem,ortransmitted,inanyformorbyanymeans,
electronic,mechanical,photocopying,recording,orotherwise,
withoutthepriorpermissionofOxfordUniversityPress.
LibraryofCongressCataloging-in-PublicationData
Perlovsky,LeonidI.
Neuralnetworksandintellect:usingmodel-basedconcepts/LeonidI.Perlovsky.
p. cm.
Includesbibliographicalreferencesandindex.
ISBN0-19-511162-1
1.Neuralnetworks(computerscience) 2.Mathematicalmodels. I.Title.
QA76.87.P435 2000
006.3'2—dc21 00-026297
Printing(lastdigit):987654321
PrintedintheUnitedStatesofAmerica
onacid-freepaper
CONDENSED TABLE OF CONTENTS
PREFACE xix
PART ONE: OVERVIEW: 2300 YEARS OF PHILOSOPHY, 100 YEARS
OF MATHEMATICAL LOGIC, AND 50 YEARS OF COMPUTATIONAL
INTELLIGENCE
1 Introduction: Concepts of Intelligence 3
2 Mathematical Concepts of Mind 51
3 Mathematical versus Metaphysical Concepts of Mind 125
PART TWO: MODELING FIELD THEORY: NEW MATHEMATICAL
THEORY OF INTELLIGENCE WITH ENGINEERING APPLICATIONS
4 Modeling Field Theory 153
5 MLANS: Maximum Likelihood Adaptive Neural System for Grouping
and Recognition 206
6 Einsteinian Neural Network 263
7 Prediction, Tracking, and Dynamic Models 289
8 Quantum Modeling Field Theory (QMFT) 321
9 Fundamental Limitations on Learning 329
10 Intelligent System Organization: MFT, Genetic Algorithms,
and Kant 356
PART THREE: FUTURISTIC DIRECTIONS: FUN STUFF:
MIND—PHYSICS + MATHEMATICS + CONJECTURES
11 Gödel Theorems, Mind, and Machine 383
12 Toward Physics of Consciousness 391
LISTOFSYMBOLS 425
DEFINITIONS 429
BIBLIOGRAPHY 447
INDEX 461
v
This page intentionally left blank
CONTENTS
PREFACE xix
PART ONE: OVERVIEW: 2300 YEARS OF PHILOSOPHY, 100 YEARS
OF MATHEMATICAL LOGIC, AND 50 YEARS OF COMPUTATIONAL
INTELLIGENCE
1 Introduction: Concepts of Intelligence 3
1.1 CONCEPTSOFINTELLIGENCEINMATHEMATICS,PSYCHOLOGY,
ANDPHILOSOPHY 3
1.1.1 WhatIsIntelligence? 3
1.1.2 Plato,Occam,andNeuralNetworks 4
1.1.3 Rule-BasedArtificialIntelligence,Complexity,and
Aristotle 6
1.1.4 Philosophyvs.ArchitectureofIntelligentTracker 8
1.1.5 Summary 12
1.2 PROBABILITY,HYPOTHESISCHOICE,PATTERNRECOGNITION,
ANDCOMPLEXITY 13
1.2.1 Prerequisite:BasicNotionsoftheTheoryofProbability 13
1.2.2 ClassicalHypothesesChoiceParadigmsandDefinitions 20
1.2.3 PatternRecognition 22
1.2.4 APrioriInformationandAdaptation 24
1.2.5 MathematicalFormulationofModel-BasedRecognition 27
1.2.6 ConundrumofCombinatorialComplexity 29
1.3 PREDICTION,TRACKING,ANDDYNAMICMODELS 29
1.3.1 LinearRegression 30
1.3.2 RegressionasanExpectation 32
1.3.3 Autoregression 33
1.3.4 Tracking 35
vii
viii Contents
1.3.5 AssociationProblem 37
1.4 PREVIEW:INTELLIGENCE,INTERNALMODEL,SYMBOL,EMOTIONS,
ANDCONSCIOUSNESS 42
Notes 45
BibliographicalNotes 46
Problems 47
2 Mathematical Concepts of Mind 51
2.1 COMPLEXITY,ARISTOTLE,ANDFUZZYLOGIC 52
2.1.1 ConundrumofCombinatorialComplexity 52
2.1.2 Adaptivity,Apriority,andComplexity 53
2.1.3 FuzzyLogicandComplexity 55
2.2 NEARESTNEIGHBORSANDDEGENERATEGEOMETRIES 58
2.2.1 TheNearestNeighborConcept 58
2.2.2 MathematicalFormulation 59
2.2.3 WhatConstitutesSimpleandComplexClassification
Problems? 59
2.2.4 DegenerateGeometryofClassificationSpaces 60
2.3 GRADIENTLEARNING,BACKPROPAGATION,ANDFEEDFORWARD
NEURALNETWORKS 62
2.3.1 ConceptofDiscriminatingSurfacesandGradient
Learning 62
2.3.2 MathematicalFormulation 64
2.3.3 LearningDisability 67
2.4 RULE-BASEDARTIFICIALINTELLIGENCE 68
2.4.1 Minsky,Apriority,andAdaptivity 68
2.4.2 SoarProductionSystem 70
2.5 CONCEPTOFINTERNALMODEL 73
2.5.1 Prolegomena:Parametricvs.NonparametricEstimation 73
2.5.2 Model-BasedVision(MBV) 74
2.5.3 AdaptivityandMBV 75
2.6 ABDUCTIVEREASONING 76
2.6.1 Deduction,Induction,andAbduction 76
2.6.2 AbductiveReasoningTreesandBayesianNetworks 77
2.7 STATISTICALLEARNINGTHEORYANDSUPPORT
VECTORMACHINES 79
2.7.1 ModelComplexity:RiskMinimizationvs.PDFEstimation 79
Contents ix
2.7.2 ConsistencyofERMandVCDimension 81
2.7.3 SupportVectorMachines(SVM) 82
2.8 AIDEBATESPASTANDFUTURE 85
2.8.1 ArgumentsandDisagreements:AnOverview 85
2.8.2 CanaMachineThink? 87
2.8.3 Rule-BasedAIvs.Connectivism 90
2.8.4 EmergingDebates 91
2.9 SOCIETYOFMIND 94
2.9.1 SocietyofAgents 94
2.9.2 TypesofAgents 95
2.9.3 FramesandUnityofApperception 96
2.9.4 LimitationsandWhatIsNext 96
2.10 SENSORFUSIONANDJDLMODEL 97
2.10.1 SensorFusionandOriginsofJDLModel 97
2.10.2 Definitions,Issues,andTypesofFusionProblems 98
2.10.3 SensorFusionLevels 99
2.10.4 HierarchyofJDLModelOrganization 100
2.11 HIERARCHICALORGANIZATION 100
2.12 SEMIOTICS 104
2.13 EVOLUTIONARYCOMPUTATION,GENETICALGORITHMS,
ANDCAS 106
2.13.1 ComplexAdaptiveSystems(CAS) 107
2.13.2 CAS:Complexityvs.Fuzziness 109
2.14 NEURALFIELDTHEORIES 110
2.14.1 Grossberg’sMethod:PhysicsofMind 110
2.14.2 ARTNeuralNetwork 111
2.14.3 IllusionsandAPrioriContentsofVision 114
2.14.4 MotorCoordinationandSensorimotorControl 115
2.14.5 EmotionsandLearning 116
2.14.6 QuantumNeurodynamics 118
2.14.7 ModelingFieldTheory 119
2.15 INTELLIGENCE,LEARNING,ANDCOMPUTABILITY 120
2.15.1 Computability:Turingvs.Physics 120
2.15.2 ComputationalMethodsofIntelligence:Summary 121
Notes 121
BibliographicalNotes 122
Problems 124