Decision Making, Affect, and Learning: Attention and Performance XXIII

576 Pages·2011·3.76 MB·English

by  Mauricio R. Delgado

Decision Making, Affect,
and Learning
Attention and Performance
Attention and Performance XIV: Synergies in Experimental Psychology, Artificial
Intelligence, and Cognitive Neuroscience.
Edited by David E.Meyer and Sylvan Kornblum, 1993
Attention and Performance XV: Conscious and Nonconscious Information Processing.
Edited by Carlo Umiltà and Morris Moscovitch, 1994
Attention and Performance XVI: Information Integration in Perception and Action.
Edited by Toshio Inui and James L. McClelland, 1996
Attention and Performance XVII: Cognitive Regulation of Performance: Interaction of
Theory and Application.
Edited by Daniel Gopher and Asher Koriat, 1998
Attention and Performance XVIII: Control of Cognitive Processes.
Edited by Stephen Monsell and Jon Driver, 2000
Attention and Performance XIX: Common Mechanisms in Perception and Action.
Edited by Wolfgang Prinz and Bernhard Hommel, 2002
Attention and Performance XX: Functional Neuroimaging of Visual Cognition.
Edited by Nancy Kanwisher and John Duncan, 2003
Attention and Performance XXI: Processes of Change in Brain and Cognitive Development.
Edited by Yuko Munakata and Mark Johnson, 2006
Attention and Performance XXII: Sensorimotor Foundations of Higher Cognition.
Edited by Patrick Haggard, Yves Rossetti, and Mitsuo Kawato, 2007
Attention and performance XXIII: Decision Making, Affect, and Learning.
Edited by Mauricio R. Delgado, Elizabeth A. Phelps, and Trevor W. Robbins, 2011
Decision Making,
Affect, and Learning
Attention and
Performance XXIII

Edited by
Mauricio R. Delgado
Department of Psychology
Rutgers University
Newark, NJ USA
Elizabeth A. Phelps
Department of Psychology and
Center for Neural Science
New York University
New York, NY USA
Trevor W. Robbins
Department of Experimental Psychology
University of Cambridge
Cambridge, UK
1
1

Great Clarendon Street, Oxford OX2 6DP
Oxford University Press is a department of the University of Oxford.
It furthers the University’s objective of excellence in research, scholarship,
and education by publishing worldwide in
Oxford New York
Auckland Cape Town Dar es Salaam Hong Kong Karachi
Kuala Lumpur Madrid Melbourne Mexico City Nairobi
New Delhi Shanghai Taipei Toronto
With offi ces in
Argentina Austria Brazil Chile Czech Republic France Greece
Guatemala Hungary Italy Japan Poland Portugal Singapore
South Korea Switzerland Thailand Turkey Ukraine Vietnam
Oxford is a registered trade mark of Oxford University Press
in the UK and in certain other countries
Published in the United States
by Oxford University Press Inc., New York
© The International Association for the study of Attention and Performance, 2011
Materials (Chapters 19 and 21) in this book prepared by authors as part of their offi cial duties as U.S. government
employees are not covered by copyright.
The moral rights of the authors have been asserted
Database right Oxford University Press (maker)
First published 2011
All rights reserved. No part of this publication may be reproduced,
stored in a retrieval system, or transmitted, in any form or by any means,
without the prior permission in writing of Oxford University Press,
or as expressly permitted by law, or under terms agreed with the appropriate
reprographics rights organization. Enquiries concerning reproduction
outside the scope of the above should be sent to the Rights Department,
Oxford University Press, at the address above
You must not circulate this book in any other binding or cover
and you must impose the same condition on any acquirer
British Library Cataloging in Publication Data
Data available
Library of Congress Cataloging in Publication Data
Library of Congress Control Number: 2010942088
Typeset in Minion by Glyph International, Bangalore, India
Printed in Asia
on acid-free paper through
Asia Pacifi c Offset
ISBN 978–0–19–960043–4
10 9 8 7 6 5 4 3 2 1
Whilst every effort has been made to ensure that the contents of this book are as complete, accurate and up-to-date
as possible at the date of writing, Oxford University Press is not able to give any guarantee or assurance that such is
the case. Readers are urged to take appropriately qualifi ed medical advice in all cases. The information in this book is
intended to be useful to the general reader, but should not be used as a means of self-diagnosis or for the prescription
of medication.
Preface
The study of decision making is a truly multi-disciplinary endeavor, subsuming investi-
gations in disciplines as disparate as behavioral and cognitive neuroscience, economics,
ethology, experimental psychology, mathematical modeling, psychiatry and psychophar-
macology. Most of these approaches are represented in this volume, which grew out of a
Workshop conference convened in Stowe, Vermont on July 13-17th 2008, as part of the
Attention and Performance (A&P) series, enabling us to convey the vibrancy and excite-
ment of this rapidly developing field.
I n order to understand the complexities of efficient decision making leading to adap-
tive outcomes, we believe it is necessary to pursue a theoretically-motivated, behavioral
economic approach to decision making that takes into account the individual heuristic
biases and idiosyncrasies of affective processing of the individual human operator. The
neural analysis of affect also encompasses neurochemically defined ‘reward’ and ‘punish-
ment’ systems which drive the fundamental processes of associative learning through
which experience comes to influence decision making. There is no doubt that the consid-
erable advances made in the neuroscience of reinforcement learning and affective process-
ing described here have contributed enormously to our evolving understanding of the
neural basis of decision making. The organization of this book reflects four interacting
facets described below. Three of them begin with special Tutorial Essays that set the scene
for the succeeding chapters.
F ollowing a comprehensive and valuable Tutorial Review by Daw on the computa-
tional methods now available for both learning and neural modeling, Section 1 provides
four essays on the interface between behavioral economics and basic psychological proc-
esses. Camerer directly poses the question of how conditioning and emotional processes
bear on economic decisions. McClure and van den Bos ask how a well-known typical
violation of economic rationality (‘the winner’s curse’) can be explained through an
interaction between modeling reinforcement theory and neuroscience. Chater and Vlaev
ponder the psychological factors that contribute to value instability through a combina-
tion of modeling reinforcement learning and cognitive choice. Kalentscher and Pennartz
provide obvious problems for decision-making principles based on value by considering
phenomena and mechanisms of intransitive choice.
Section II on neural mechanisms of decision making begins with a salutary Tutorial
Review in which Hayden and Platt illustrate the difficulties of bringing together studies
on decision making under uncertainty using behavioral studies with single-unit record-
ing in monkeys and fMRI in human volunteers. The rest of this Section nonetheless
emphasizes the considerable advances that have been made in identifying neural corre-
lates of decision making processes. Thus, Venkatraman et al use functional magnetic
resonance imaging in studies of risky decision making to show how specific neural
vi PREFACE
systems in the cerebral cortex support variability in human choice and strategy, for exam-
ple, the use of compensatory trade-offs and simplifications of decision rules. Doya and
colleagues use mathematical models of decision making, in order to inform their human
fMRI studies of temporal discounting, as affected by dietary manipulations of the central
serotonin system. Sakagami and colleagues also combine human fMRI and monkey elec-
trophysiology to study neural mechanisms underlying value-based choice. The last two
chapters of this Section focus on the central role of the orbitofrontal cortex in decision
making. Thus, Clarke and Roberts examine the role of the orbitofrontal cortex in co-
ordinating behavioral and autonomic activity in choice situations in marmoset monkeys,
its interactions with other systems such as the amygdala and striatum, and its neuro-
chemical modulation, for example, by serotonin. Walton and colleagues compare the
functions of the anterior cingulate cortex and orbitofrontal cortex in choice behavior
over time, mainly on the basis of lesion studies in monkeys, but substantiated by studies
using human fMRI. These contributions confirm that the orbitofrontal cortex has impor-
tant functions in decision-making cognition which are being illuminated by studies in
experimental animals, as well as in humans.
The chapters in Section III have a slightly different theme, being especially implicated
in underlying striatal-based reward and aversive processes. Roesch and Schoenbaum’s
Tutorial Review surveys their electrophysiological investigations of reinforcement learn-
ing, in terms of possible interactions of the rat orbitofrontal cortex in outcome processing
with the amygdala, and also with attentional and prediction error mechanisms of the
dopaminergic mid-brain. Four subsequent chapters focus on the basal ganglia, directly
addressing the issues of how these structures (somewhat implicitly) mediate appetitive
and aversive learning (Pessiglione et al) and are modulated by cognitive control centers
that regulate affect and decision making (Martin and Delgado). Through studies of nor-
mal volunteers and patients with Parkinson’s disease, Cools examines the contribution of
striatal and prefrontal dopamine modulation to component processes for human deci-
sion making, using probabilistic reversal learning as the main behavioral choice para-
digm. Wimmer and Shohamy also investigate the striatal role in learning in humans via
Parkinson’s disease but pose the question that the possible importance of dopamine-
dependent memory processing in hippocampus may previously have been underesti-
mated. It is instructive that Niewehnhuis and Jepma’s chapter concerns instead the role
of a different ascending chemical neurotransmitter system, noradrenaline, in modulating
attentional processes relevant to the ‘explore or exploit’ model. Typical of the multidisci-
plinarity in evidence at this Workshop, they show how a theoretical approach fed by
behavioral electrophysiological data from monkeys can be used to make predictions
about psychopharmacological manipulations of the noradrenergic system in studies of
human decision making and performance. Finally, utilizing both mathematical modeling
and functional neuroimaging in humans, Paulus focuses on the role of the insular cortex
in mediating interoceptive cues, whether aversive or rewarding, in “homeostatic” deci-
sion making and relates their abnormal processing to particular psychiatric disturbances,
including anxiety and addiction.
PREFACE vii
The concluding Section IV goes on to describe three main neuropsychiatric or develop-
mental applications of research into decision-making cognition. Chapters by Ernst (using
mainly functional resonance imaging paradigms to simulate gambling scenarios) and by
Casey et al (with a focus on neural mechanisms of inhibitory control) elucidate the neural
basis of risky and impulsive decision making in adolescents. The chapters by Goldstein
and by Ersche provide contrasting neuropsychological analyses of decision-making cog-
nition in substance abusers. Sahakian and Morein-Zamir review evidence of affective bias
in decision making cognition in patients with depression, using an approach that com-
bines neuropsychological and functional neuroimaging techniques. These chapters well
exemplify the scope of the field and its potential for clinical application.
W e hope our readers enjoy reading these contributions to the prestigious ‘A&P’ series
as much as we have; it was a truly memorable meeting and we hope to have captured
some of its most exciting elements. We are most grateful to Martin Baum and his col-
leagues at Oxford University Press for their understanding and support in bringing this
project to fruition. Thanks also to Sandra Yoshida for excellent assistance. Finally, we
acknowledge support for the A&P conference that led to this Volume from the National
Institute on Drug Abuse (NIDA), the National Institute on Aging (NIA), Wyeth Inc., and
the James S. McDonnell Foundation, who supported both the conference and a collabo-
rative network between New York University and the University of Cambridge that
inspired this venture.
M.R. Delgado,
E.A. Phelps, and T.W. Robbins,
New York and Cambridge U.K.
October 2010
This page intentionally left blank
Contents
The Attention and Performance Symposia xi
Contributors xiii
Abbreviations xix
Section I P sychological processes underlying decision making
1. Trial-by-trial data analysis using computational models
(Tutorial Review) 3
Nathaniel D . Daw
2. Psychological influences on economic choice: Pavlovian cuing and
emotional regulation 39
Colin F . Camerer
3. The psychology of common value auctions 63
Samuel M. McClure and Wouter van den Bos
4. The instability of value 81
Nick Chater and Ivo Vlaev
5. Do intransitive choices reflect genuinely context-dependent preferences? 101
Tobias Kalenscher and Cyriel M.A. Pennartz
Section II N eural systems of decision making
6. On the difficulties of integrating evidence from fMRI and electrophysiology
in cognitive neuroscience (Tutorial Review) 125
Benjamin Y. Hayden and Michael L. Platt
7. Neuroeconomics of risky decisions: from variables to strategies 153
Vinod Venkatraman , John W. Payne , and Scott A. Huettel
8. Multiple neural circuits in value-based decision making 173
Manami Yamamoto , Xiaochuan Pan , Kensaku Nomoto , and Masamichi Sakagami
9. Model-based analysis of decision variables 189
Kenji Doya , Makoto Ito , and Kazuyuki Samejima
10. Reversal learning in fronto-striatal circuits: a functional, autonomic, and
neurochemical analysis 205
H.F. Clarke and A.C. Roberts
11. Cingulate and orbitofrontal contributions to valuing knowns and unknowns
in a changeable world 235
Mark E . Walton , Peter H . Rudebeck , Timothy E. J . Behrens , and
Matthew F. S . Rushworth