Computational Neuroscience of Drug Addiction

356 Pages·2012·7.19 MB·English

by  Vladimir L. Tsibulsky

Springer Series in Computational Neuroscience
Volume 10
SeriesEditors
AlainDestexhe
CNRS
Gif-sur-Yvette
France
RomainBrette
ÉcoleNormaleSupérieure
Paris
France
Forfurthervolumes:
www.springer.com/series/8164
sdfsdf
Boris Gutkin (cid:2) Serge H. Ahmed
Editors
Computational
Neuroscience
of Drug Addiction
Editors
BorisGutkin SergeH.Ahmed
GroupforNeuralTheory CNRSUMR5293IMN
LNC,DEC,ENS UniversitéVictorSegalen
75005Paris 33076Bordeaux
France France
[email protected] [email protected]
ISBN978-1-4614-0750-8 e-ISBN978-1-4614-0751-5
DOI10.1007/978-1-4614-0751-5
SpringerNewYorkDordrechtHeidelbergLondon
LibraryofCongressControlNumber:2011940849
©SpringerScience+BusinessMedia,LLC2012
Allrightsreserved.Thisworkmaynotbetranslatedorcopiedinwholeorinpartwithoutthewritten
permission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York,
NY10013,USA),exceptforbriefexcerptsinconnectionwithreviewsorscholarlyanalysis.Usein
connectionwithanyformofinformationstorageandretrieval,electronicadaptation,computersoftware,
orbysimilarordissimilarmethodologynowknownorhereafterdevelopedisforbidden.
Theuseinthispublicationoftradenames,trademarks,servicemarks,andsimilarterms,eveniftheyare
notidentifiedassuch,isnottobetakenasanexpressionofopinionastowhetherornottheyaresubject
toproprietaryrights.
OrganizationofComputationalNeuroscience:www.cnsorg.org
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Foreword
One only has to read the newspapers to understand the extent to which addictions
areamongthescourgesoftheday.Theyleadtowastageandwastingofinnumerable
individuallives,andahugecosttothebodypolitic,withgargantuansumsofillicit
money supporting edifices of corruption. The blame for other modern solecisms,
suchasburgeoningobesity,isincreasinglybeinglaidatthesamedoor.
From the perspective of neuroscience, addictions present a critical challenge.
Substanceswithatleastinitiallyrelativelyimmediateeffectsonmoreorlesswell-
definedsetsofreceptors,have,insomeindividuals,apanoplyofphysiologicaland
psychological consequences that unravel over the course of years. Understanding
eachdomainofinquirybyitself,andthelinksbetweenthem,iscriticalforunder-
standingthecourseofaddictions,andinthelongerrun,conceivingmoreeffective
optionsforpalliationorevencure.Althoughthereisanearoverwhelmingvolume
ofdata,thecomplexitiesofthesubjectmeanthattherearealsomanyapparentin-
consistenciesandcontradictions.
The understanding of addiction rests on analyses over multiple scales. For in-
stance,wenotonlyhavetounderstandtheprogressiveeffectsoflong-termdruguse
on receptor characteristics and density, we must also grasp the changes this leads
to in the neurons concerned, and then in the dynamical operation and information
processingofthecircuitsandsystemsthoseneuronscomprise.Equally,wehaveto
understand how the effects on decision-making play out in terms of the economic
choicesmadebycomplex,human,decision-makers.
The need to tie together phenomena at these multiple scales is a critical force
leading to the current book’s focus on theoretical ideas. Indeed research in addic-
tionisaparadigmaticexampleofmodernsystemsbiology.Thetaskofprovidinga
formalscaffoldingforunderstandingthelinksacrosslevelsofinquiry,isthetopic
ofoneofthreewingsoftheoreticalneuroscience.Inthetop-downdirection,thisis
acaseofaformalscientificreduction,explainingphenomenaobservedatonescale
bymechanisticmodelsbuiltfromcomponentsthatliveatfinerscales.Thesecom-
ponentsarecharacterizedeitherbydescriptivemodels,orarethemselvesexplained
bymodelsatyetfinerscales.Buildingandprovingsuchmulti-scalemodelsisaper-
fectly normal role for mathematical and (increasingly) simulation-based modeling
v
vi Foreword
inanaturalscience.Onemightonlycavilthat,comparedwithsomeofitscousins,
experimentalneurosciencehassometimesseemedalittletardyinplayingball.
Chapter4isperhapsthepurestexampleofthissortofanalysis—providing,for
instance, a formal way of resolving the apparent conflict between in vivo and in
vitrodataastowhethernicotine’smainactionontheactivityofdopaminergiccells
(believedtobekeytothedrug’saddictivepotential)isdirectorindirect.However,
many of the other chapters also contain elements of this modus operandi too, ap-
pliedatdifferentlevels.Forinstance,thesophisticatedagentmodelingofChap.11
reminds us about the complexity of interactions between addicts and the environ-
ment which facilitates and hinders their addiction. Rich patterns of positive and
negativefeedbackemerge.Thus,itispossibletoexamineandpredicttheeffectsof
makingmanipulationsatsinglepointsinthenexusofinteractions—woewould,for
instance,betidethepolicy-makerwhoattemptstointervenetoosimplisticallyina
systemthatissufficientlynon-linearastobechaotic.
However, the chapters of the book also attest to the power of a second wing of
theoreticalneuroscience.Thisistheconceptthatbrainsmustsolvecomputational,
information-processingandcontrol-theoreticproblemsassociatedwithsurvivingin
a sometimesnasty andbrutish world. Decision-making,in its fullest sense, is per-
hapsthecriticalcompetenceforsurvival.Theideaistostartfromanunderstanding
ofthevariouswaysthatsystemsofanyarbitrarysortcan(learnto)makegoodde-
cisions in the face of rewards and punishments. This topic lies at the intersection
of economics, statistics, control theory, operations research and computer science.
Theresultingcomputationalandalgorithmicinsightsprovideafoundationfor,and
constraints on, how humans and other animals actuallymake choices. Further, the
sub-components of these models provide a parameterization of failure—points at
whichaddictivesubstancescanexerttheirintricatelymaligneffects.
Somewhatalternativeversionsofthesenormative,andapproximatelynormative,
notionsareapparentinmanyoftheremainingchapters,differingaccordingtothe
degree of abstraction, the intensity of focus on the computational level versus as-
pects of the algorithms and implementations of those computations, and also the
extent to which the complexity of the neural substrate is included. There are also
manydifferentwaystoformalizecontroltheoreticproblems,intermsof(a)thena-
tureoftheworld(andthepossibleinternalmodelthereof);(b)thegoalsofcontrol,
forinstancethehomeostaticintentofkeepingvariableswithinappropriatebounds
asopposedtofindingoptimalsolutionsinthelightofcostsandreturns;alongwith
(c)assumptionsaboutthepossiblesolutions.
TherelativelypurercontroltheoreticapproachesdiscussedinChaps.1,2,3lie
nearer one end of the spectrum. They pay particular attention to one of the cen-
tralideasincontroltheory,namelyfeedbackasawayofkeepingsystemsinorder.
Althoughonecanimaginecomputational-levelrenditions,thisismostlyanalgorith-
micidea,beingratherdivorcedfrompossiblecomputationalunderpinningsinterms
ofsuchthingsasbeinganoptimalwayofpreventingdivergencefromasuitableop-
eratingpointaccordingtoajustifiablecostfunction.Drugscanknocksystemsoutof
kilter,andsoinspireimmediateorpredictivefeedbacktocorrectthestate;thenega-
tiveeffectsofwithdrawalcanalsoleadtoacorrectivepolicyofself-administration.
Foreword vii
Inthesechapters,wecanseesomethingofthepowerofquantitativelycharacteriz-
ingthedynamicaleffectsofdrugsinsystemswithsubstantialadaptation.
ThebehavioraleconomicmodelingofChap.10hasasomewhatsimilarquality.
Here,thealiquotsofcomputationalanalysisconcerntherelationshipbetweenprice
anddemandandtheeffectoftemporaldelayonvalue.Thesethenplayoutthrough
psychologicaldataandalgorithmicrealizationsofthesedata.
The chapters based on modern reinforcement learning (RL) ideas (parts or all
of Chaps. 5, 6, 7, 8) span the other end of the spectrum. As will become apparent
inreviewingthesechapters,reinforcementlearning,whichcomprisesformsofop-
timal,adaptivecontrol,hasbecomeadominanttheoreticalparadigmformodeling
human and animal value-based decision-making. In some ways, it has taken over
thisrolefromcybernetics,itsclosecontrol-theoreticcousin,whichhistoricallyex-
erted a strong influence over systems thinking in areas in which value has played
alesserpart.Aspectsoftheactivityofdopaminergiccellsofferedoneoftheearly
strongtiesbetweentheoryandexperiment;itisonlysurprisinglyrecentlythatad-
diction,withitsmanifoldinvolvementofdopamine,hasbecomeatargetforthissort
ofeffort.
Different RL approaches, such as model-based methods (which make choices
bybuildingandsearchingmodelsofthedecision-makingdomain)andmodel-free
methods (which attempt to make the same optimal choices, but by learning how
to favor actions from experience, without building models) are suggested as be-
ingrealizedinstructurallyandfunctionally-segregatedpartsofthebrain,andhave
eachbeenprovidedwithnormativerationales.Thevariouschaptersalsospeaktoa
fecundcollaborationbetweentheoryandexperiment,forinstancewinklingoutpo-
tentially suboptimal interactions between the systems, and roles for evolutionarily
prespecifiedcontrolintheformofPavlovianinfluences.
These chapters take rather different perspectives on the overall problem of ad-
diction,evenifgenerallyadoptingarathercommonlanguage.Indeed,althoughthis
language is powerful, uniting as it does information-processing notions with psy-
chologicalandanatomical,physiologicalandpharmacologicaldata,itisfairtosay
thatitrevealsratherthanreducesthecomplexityoftheindividualsystems,acom-
plexity which is then hugely magnified in the way the systems interact. However,
themultipleinfluencesofdrugsactingoverthesystemsandtheirinteractionsallat
diversetimescales,canatleastbelaidbare,andreasonedabout,giventhisformal
vocabulary.
Chapter9comesfromanother,subtlydifferent,traditionofnormativemodeling,
associatedoriginallymorewithunsupervisedlearning(orprobabilisticmodelingof
the statistical structure of the environment with no consideration of valence) than
RL,andindeedcomestosuggestaratherdifferent(and,currentlyratherrestricted)
waythatdrugscanact.Nevertheless,therearetiesboththeRLmodelsandindeed
thecontroltheoreticones—forinstance,Chap.3’sdiscussionofthelimitsofhome-
ostasis is nicely recapitulated in Chap. 9’s notion of itinerant policies, for which
fixedpointsareananathema.
Finally, note a critical lesson from the diversity of the chapters. Chapter 4’s
whole contribution concerns important (but still doubtless not comprehensive) de-
tailsaboutthecomplexeffectsovertimeofoneparticulardrugofaddictiononone
viii Foreword
key circuit. Consider how these effects would be rendered in the abstract, impres-
sionistic, terms of pretty much all the other chapters. These latter models simply
lackthesophisticationtocapturethedetails—but,sadly,withoutbeingabletopro-
videaguaranteethatinthecomplexsystemstheydomodel,omittingthedetailsis
benign.
If one regards the diversity as a mark of adolescent effervescence in this field
of computational addiction; the fact that such a fascinating book is possible is a
markofimpendingmaturity.Theoreticalideasandmathematicalandcomputational
models are rapidly becoming deeply embedded in the field as a whole, and, most
critically,areprovidingnew,andmorepowerful,waystoconceiveofthemultiple,
interacting, problems wrought by addictive substances. Providing the guarantees
mentioned above, and indeed a more systematic tying together of all the different
levels and types of investigation across different forms and causes of addiction, is
thechallengeforearlyadulthood.
Acknowledgements
IamverygratefultotheGatsbyCharitableFoundationforfunding.
London,UK PeterDayan
Acknowledgements
TheeditorswouldliketoacknowledgethesupportfromtheFrenchResearchCoun-
cil (CNRS) (BSG and SHA), the Université Victor Segalen-Bordeaux (SHA), the
Conseil Régional d’Aquitaine (SHA), Ecole Normale Superieure and INSERM
(BSG), the French National Research Agency (ANR) (BSG and SHA), NERF
(BSG) and Fondation Pierre-Gilles de Genes. Boris Gutkin would like to thank
Prof. J.P. Changeux for launching into the field of addiction research and his con-
tinuingsupport.
ix