Allocation of Computational Resources in the Nervous System

116 Pages·2006·1.07 MB·English

Allocation of Computational Resources
in the Nervous System
Adissertation
submittedforthedegreeof
DoctorofPhilosophy
by
SantiagoJaramillo
Supervisor: BarakA.Pearlmutter
DepartmentofElectronicEngineering
NationalUniversityofIreland,Maynooth
October2006
Contents
Summary v
Acknowledgements vi
1 Introduction 1
1.1 Resourcesandallocation . . . . . . . . . . . . . . . . . . . . . 2
1.2 Motivation: theneuralcode . . . . . . . . . . . . . . . . . . . 3
1.3 Historicalcontext . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.4 Remainingquestions . . . . . . . . . . . . . . . . . . . . . . . 7
1.5 Themodelingapproach . . . . . . . . . . . . . . . . . . . . . 7
1.6 Dissertationoutline . . . . . . . . . . . . . . . . . . . . . . . . 8
2 Behavioralmeasurementsoforientingsystems 10
2.1 Covertattentionimprovesperformance . . . . . . . . . . . . 10
2.2 Dividedattention . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.3 Whyhavecovertorientingifwecanmoveourbodies? . . . 13
2.4 Saliency . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2.5 Measurementsofovertorienting . . . . . . . . . . . . . . . . 14
2.6 Modelingorientingphenomena . . . . . . . . . . . . . . . . . 15
3 Resourceallocationinmulti-layerperceptronnetworks 21
3.1 Optimalcodingundernon-uniformrelevance . . . . . . . . 21
3.2 Feedforwardnetworkwithmodulatoryinput . . . . . . . . . 23
3.3 Spatially-drivenallocation . . . . . . . . . . . . . . . . . . . . 24
3.3.1 Simulationparameters . . . . . . . . . . . . . . . . . . 24
3.3.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
3.3.3 Discussionandlimitations. . . . . . . . . . . . . . . . 26
3.4 Non-spatiallydrivenallocation(feature-based) . . . . . . . . 28
3.4.1 Simulationparameters . . . . . . . . . . . . . . . . . . 28
3.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
3.4.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 30
i
CONTENTS
3.5 Concludingremarks . . . . . . . . . . . . . . . . . . . . . . . 31
4 Attentionalmodulationofneuronalactivity 32
4.1 BOLD/fMRIandPETmeasurements . . . . . . . . . . . . . . 32
4.2 Modulationofbrainrhythmsandsynchrony . . . . . . . . . 33
4.3 Modulationofevokedpotentialsandevokedfields . . . . . 35
4.4 Firingratemodulation . . . . . . . . . . . . . . . . . . . . . . 36
4.4.1 Primaryvisualcortex: V1 . . . . . . . . . . . . . . . . 37
4.4.2 V2andtheventralstream: V4andIT . . . . . . . . . 38
4.4.3 Thedorsalstream: MTandMST . . . . . . . . . . . . 40
4.4.4 Otherareasselectivetovisualstimuli . . . . . . . . . 40
4.4.5 Non-visualmodalities . . . . . . . . . . . . . . . . . . 41
4.5 Microstimulationstudies . . . . . . . . . . . . . . . . . . . . . 41
4.6 Computationalmodels . . . . . . . . . . . . . . . . . . . . . . 42
5 Activitymodulationemergesfromoptimalcodingprinciples 45
5.1 Attentionaldependenceofpreferredstimuli . . . . . . . . . 46
5.2 Attention-dependentmodulationofactivity . . . . . . . . . . 47
5.3 Magnitudeofmodulationdependsonthesystem’scapacity 51
5.4 Modulationoforientationtuningcurves . . . . . . . . . . . . 52
5.5 Discussionandlimitationsofthemodel . . . . . . . . . . . . 54
5.6 Concludingremarks . . . . . . . . . . . . . . . . . . . . . . . 56
6 Neuralmechanismsforresourceallocation 58
6.1 Howdoattentionalsignalsentersensoryprocessingcircuits? 59
6.1.1 Changesinsynapticstrengths . . . . . . . . . . . . . 59
6.1.2 Dendriticcomputation . . . . . . . . . . . . . . . . . . 60
6.1.3 Modulatoryandsensorysignalsareindistinguishable 60
6.2 Theoriginofattentionalcontrolsignals . . . . . . . . . . . . 61
6.3 Attentionandreceptivefieldformation . . . . . . . . . . . . 63
6.3.1 Receptivefieldformation . . . . . . . . . . . . . . . . 63
6.3.2 Sparsityconstraintsandredundancyreduction . . . 64
6.3.3 Non-uniformrelevanceproduceslocalizedRFs . . . 66
7 Allocationofresourcesduringactiveperception 67
7.1 Perceptionisanactiveprocess. . . . . . . . . . . . . . . . . . 67
7.2 Optimalstrategiesforsimultaneoustracking . . . . . . . . . 69
7.3 Analysisofatwo-channelsystem . . . . . . . . . . . . . . . . 69
7.3.1 Fixedallocation . . . . . . . . . . . . . . . . . . . . . . 72
7.3.2 Dynamicallocation . . . . . . . . . . . . . . . . . . . . 73
7.3.3 Comparingfixedvs.dynamicstrategies . . . . . . . . 75
7.3.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . 76
7.4 Concludingremarks . . . . . . . . . . . . . . . . . . . . . . . 78
ii
CONTENTS
8 Psychophysicsofactiveperception 79
8.1 Experimentalmethodology . . . . . . . . . . . . . . . . . . . 80
8.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
8.2.1 Changingstrategiesaccordingtotargetdynamics . . 82
8.2.2 Changingstrategiesaccordingtonoiselevel . . . . . 83
8.3 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
8.4 Concludingremarks . . . . . . . . . . . . . . . . . . . . . . . 85
9 Conclusions 86
9.1 Contributionsandimplications . . . . . . . . . . . . . . . . . 87
9.2 Futurework . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
9.3 Finalremarks . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
APPENDIX 92
A.1 Exampleofpatternsusedinsimulations . . . . . . . . . . . . 92
A.2 Backpropagationofmodifiederror . . . . . . . . . . . . . . . 93
A.3 Additionalstudies . . . . . . . . . . . . . . . . . . . . . . . . 94
A.3.1 BrightnessIllusionsasOptimalPercepts . . . . . . . 94
A.3.2 BlindSourceSeparationofMEGdata . . . . . . . . . 94
A.3.3 Time-frequencyanalysisofMEGdata . . . . . . . . . 95
iii
List of Figures
2.1 Covertattentionimprovesperformance . . . . . . . . . . . . 12
2.2 Traditionalmodelofvisualattention . . . . . . . . . . . . . . 16
3.1 Inputsduringoptimizationoftheconnectionstrengths . . . 25
3.2 ResourceallocationinMLPs . . . . . . . . . . . . . . . . . . . 26
3.3 Feature-basedallocation . . . . . . . . . . . . . . . . . . . . . 29
4.1 Retinotopyofvisualattention . . . . . . . . . . . . . . . . . . 34
5.1 Preferredstimulidependedonattentionalstate . . . . . . . . 46
5.2 Activityofmodelunitswasmodulatedbyattention . . . . . 47
5.3 ActivitymodulationmatchedmeasurementsfromareaMST 48
5.4 Allmodelunitsshowedsimilaractivitymodulation . . . . . 49
5.5 ActivitymodulationmatchedmeasurementsfromareaV4 . 51
5.6 Attentionalmodulationdependedonthesystem’scapacity . 53
5.7 Attentionalmodulationoftuningcurves. . . . . . . . . . . . 53
6.1 Neuralmechanismsforresourceallocation . . . . . . . . . . 59
6.2 Allocationmechanismsinafeedforwardnetwork . . . . . . 62
6.3 Non-uniformrelevanceproducedlocalizedpreferredstimuli 65
7.1 TwochannelMarkovmodel . . . . . . . . . . . . . . . . . . . 69
7.2 Averageerrorsforfixedallocationstrategies . . . . . . . . . 72
7.3 Optimalfixedallocation . . . . . . . . . . . . . . . . . . . . . 73
7.4 Averageerrorsfordynamicallocationstrategies . . . . . . . 74
7.5 Exampleofdynamicallocation . . . . . . . . . . . . . . . . . 75
7.6 Comparingfixedvs.dynamicallocation . . . . . . . . . . . . 76
8.1 Eye-trackingsystemanddisplay . . . . . . . . . . . . . . . . 80
8.2 Eye-movementsduringsimultaneoustracking . . . . . . . . 82
8.3 Averagescores . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
8.4 Strategiesfordifferentdynamics . . . . . . . . . . . . . . . . 84
8.5 Strategiesfordifferentnoiselevels . . . . . . . . . . . . . . . 85
iv
Summary
The nervous systemintegratespast information togetherwith predictions
about the future in order to produce rewarding actions for the organism.
This dissertationfocuses on the resources underlying thesecomputations,
andthetask-dependentallocation oftheseresources. Wepresentevidence
that principles from optimal coding and optimal estimation account for
overt and covert orienting phenomena, as observed from both behavioral
experimentsandneuronalrecordings.
First,wereviewbehavioralmeasurementsrelatedtoselectiveattention
and discussmodelsthat account for thesedata. Weshowthatreallocation
ofresourcesemergesasanaturalpropertyofsystemsthatencodetheirin-
putsefficientlyundernon-uniformconstraints. Wecontinuebydiscussing
theattentionalmodulationofneuronalactivity,andshowthat: (1)Modula-
tionofcodingstrategiesdoesnotrequirespecialmechanisms: itispossible
to obtain dramatic modulation even when signals informing the system
aboutfidelityrequirementsenterthesysteminafashionindistinguishable
from sensory signals. (2) Optimal coding under non-uniform fidelity re-
quirementsis sufficient toaccount for thefiring rate modulationobserved
duringselectiveattentionexperiments. (3)Theresponseofasingleneuron
cannotbewellcharacterizedbymeasurementsofattentionalmodulationof
only a single sensorystimulus. (4) The magnitude of the activity modula-
tiondependsonthecapacityoftheneuralcircuit. Alaterchapterdiscusses
the neural mechanisms for resource allocation, and the relation between
attentionalmechanismsandreceptivefieldformation.
The remainder of the dissertation focuses on overt orienting phenom-
ena and active perception. We present a theoretical analysis of the alloca-
tion of resources during state estimation of multiple targets with different
uncertainties,togetherwitheye-trackingexperimentsthatconfirmourpre-
dictions.
We finish by discussing the implications oftheseresults to our current
understandingoforientingphenomenaandtheneuralcode.
v
Acknowledgements
Firstandforemost,IwouldliketothankmysupervisorBarakPearlmutter
for his guidance and supportthroughouttheseyears. I also wish tothank
the faculty, administrators, and fellow students at the Hamilton Institute,
NUIMaynooth,forsignificanthelpduringtheexecutionofthisresearch.
Special thanks are extended to Rodolfo Llina´s and the people at the
CenterforNeuromagnetism,NewYorkUniversitySchoolofMedicine. Be-
ing part of the neuroscience community at NYU,and learning from many
outstandingresearchersatthisinstitution,werebothextraordinaryexperi-
ences.
The courses at the Santa Fe Institutefor Complex Systems, the Marine
Biological Laboratory in Woodshole, and the Frankfurt Institute for Ad-
vanceStudiesconstitutedasignificantelementofmyeducation. Iamgrate-
ful to organizers, faculty and students who made possible these amazing
courses.
I also owe a great deal to all my friends who shared part of their lives
with me during my years as a graduate student. Albuquerque, New York
CityandMaynoothwouldnothavebeenthesamewithoutyouall.
I would like to express my gratitude to all those who take part of the
Open Source community. My work was considerably dependent on your
contributions, and it would not have been as enjoyable without the tools
thecommunitycontinuestoprovide.
Finally, I wish to thank my family for their unconditional supportand
understanding.
vi
Declaration
I hereby certify that this material, which I now submit for assessment on
the program study leading to the award of Doctor of Philosophy in Elec-
tronic Engineering is entirely my own work and has not been taken from
theworkofotherssaveandtotheextentthatsuchworkhasbeencitedand
acknowledgedwithinthetextofmywork.
Signed: IDNo.: 62134141
Date:
vii
1
Chapter
Introduction
Summary
This chapter presents an outline of the contents of this disser-
tation, explains why orienting phenomena and resource allo-
cation were selected as research topics, and describes the ap-
proachfollowedtoinvestigatethem. Abriefhistoryofrelevant
work is also included, highlighting landmarks that will be dis-
cussedindetailinsubsequentchapters.
Understanding how the signals in the nervous system subserve behavior
is one of the main quests of 21st century science. The work presented in
this dissertation investigates various hypothesesconcerning how the ner-
vous systemencodesinformation about our perceptionsand actions. This
explorationusesofacombinationoftheoretical,computationalandexper-
imentalapproaches. Ourinterestis,forthemostpart,onquestionsrelated
tothehumannervoussystem,butinourattempttoachieveabetterunder-
standingofneuralprocessinganimalmodelshaveproventobeofextreme
importanceandwillbeusedfrequentlyinourdiscussion. Webeginbyde-
scribing the main concepts and terminology that will be used throughout
thisdissertation.
1
Chapter1. Introduction
1.1 Resources and allocation
The term computational resources refers here to the ability of the nervous
system to transform signals. The concept resource is used to emphasize
that there is a certain amount of computation that can be doneper unit of
time,andacertainamountofinformationthatcanflowthroughthesystem,
limited for example by the number of receptors and noise level. Energy,
timeandthenumberofsensoryreceptors,connectionsandmusclesarethe
physical quantities underlying these resources. Computational resources
are to be foundat all stagesofprocessing,fromsensorytomotorsystems,
including“intermediate”stepsinvolvedinperformingmentalcalculations
orrecalling memories. Allocation referstotheprocessofdistributing these
resources in order to represent, transform and transmit different signals
involvedinperceptionandaction.
Asanexampleofallocationofresources,letusfocusontheopticnerve
of humans in the context of information transmission. Composed of ap-
proximately1.2 million fiberspereye,itsinformation capacity can beesti-
mated, assuming independenceand given statistics of the firing rates and
noise characteristics, to be almost 10 Mbps (Koch et al., 2006). The signals
going through these axons depend on the position of the eyes, implying
that the nervous system is selecting the content that will be transmitted
tothe nextstageofprocessing,or equivalently, allocating transmission re-
sources to certain features of a visual stimulus. This process, when move-
mentofthesensorstakesplace,iscalledovertallocation. Incontrast,covert
allocation does not require physical movement (except perhaps for ions
moving across membranes making the signal transmission possible), and
occursforexamplewhentheretinaadaptstodifferentintensitylevelsand
contrastssoastoemploybetterthedynamicrangeoftheretinalcells(Bac-
cusandMeister,2002). Inthisdissertation,wewill focusmainly in alloca-
tionphenomenarelatedtoorientingattention,bothcovertlyandovertly.
Orienting phenomenacan be voluntary or automatic. It is voluntary, for
example,whenwetrytofindafriendinacrowd,orwhenwefocusonone
conversation in a noisy environment filtering out other sounds that reach
ourears(usually knownasthecocktail party effect). Orientingis automatic,
forinstance,whenourfocuschangesafterhearingsomeonesayourname.
These two types of phenomena are usually referred to as endogenous (task
2