Computation and representation in emotion and decision-making

229 Pages·2017·13.21 MB·English

Computation and representation in emotion
and decision-making
Archy Otto de Berker

A dissertation submitted in partial fulfilment of the requirements for the degree of Doctor of
Philosophy
Institute of Neurology
UCL
November 2016

1
I, Archy Otto de Berker, confirm that the work presented in this thesis is my
own. Where information has been derived from other sources, I confirm that
this has been indicated in the thesis.

_______________________________________
Archy Otto de Berker, November 2016

2
Abstract
This thesis deals with three components of an organism’s interactions with its environment:
learning, decision making, and emotions. In a series of 5 studies, I detail relationships between
these processes, and investigate the representation and computations whereby they are
achieved. In the first experiment I show how subjective wellbeing is influenced by one’s own
rewards and expectations, but also those of other people. Furthermore, I find that parameter
estimates of empathy predict decision-making in a distinct test of economic generosity. In my
second study, I ask how stressful experiences modulate subsequent learning, detailing a specific
impairment in action-learning under stress which also manifests itself in altered pupillary
responses. In the third, I use a hierarchical model of learning to show that subjective uncertainty
in aversive contexts predicts several dimensions of acute stress responses. Furthermore, I find
that individuals who show greater uncertainty-tuning in their stress responses are better at
predicting the presence of threat. In the final pair of studies I ask how decision variables for
value-based choice are represented in the brain. I describe the combination of quality and
quantity into value estimates in humans, revealing a central role for the Anterior Cingulate
Cortex in value integration using functional magnetic resonance imaging. I next characterize the
neural code for value in non-human primate frontal cortex, using single-neuron data from
collaborators. These two studies provide convergent evidence that the value code may be more
diverse and non-linear than previously reported, potentially conferring the ability to incorporate
uncertainty signals directly in the activity of value coding neurons.

3
Table of contents
Abstract ............................................................................................................................................ 3
Table of contents ............................................................................................................................ 4
Acknowledgements ........................................................................................................................ 6
Figures and tables ........................................................................................................................... 8
Chapter 1: General introduction ................................................................................................. 11
1.1 Preliminaries ............................................................................................................................. 11
1.2 Value and decisions .................................................................................................................. 13
1.3 Learning from experience ....................................................................................................... 17
1.4 Uncertainty in the environment, decision-making, and the brain ...................................... 29
1.5 Emotions: causes and consequences .................................................................................... 43
1.6 References ................................................................................................................................. 50
Chapter 2: The social contingency of momentary subjective well-being .............................. 59
2.1 Abstract ...................................................................................................................................... 59
2.2 Introduction ............................................................................................................................... 60
2.3 Methods ..................................................................................................................................... 62
2.4 Results ........................................................................................................................................ 65
2.5 Discussion .................................................................................................................................. 73
2.6 References ................................................................................................................................. 76
Chapter 3: Acute stress selectively impairs learning to act .................................................... 79
3.1 Abstract ...................................................................................................................................... 79
3.2 Introduction ............................................................................................................................... 80
3.3 Methods ..................................................................................................................................... 82
3.4 Results ........................................................................................................................................ 89
3.5 Discussion .................................................................................................................................. 98
3.6 References ............................................................................................................................... 102
Chapter 4: Linking computations of uncertainty to acute stress responses in humans ... 105
4.1 Abstract .................................................................................................................................... 105

4
4.2 Introduction ............................................................................................................................. 106
4.3 Methods ................................................................................................................................... 109
4.4 Results ...................................................................................................................................... 121
4.5 Discussion ................................................................................................................................ 131
4.6 Supplementary figures ........................................................................................................... 134
4.7 References ............................................................................................................................... 143
Chapter 5: Formation of value from quality and quantity in human decision making ..... 146
5.1 Abstract .................................................................................................................................... 146
5.2 Introduction ............................................................................................................................. 147
5.3 Methods ................................................................................................................................... 151
5.4 Results ...................................................................................................................................... 159
5.5 Discussion ................................................................................................................................ 176
5.6 References ............................................................................................................................... 181
Chapter 6: Diversity of value-tuning in primate prefrontal cortex ...................................... 188
6.1 Abstract .................................................................................................................................... 188
6.2 Introduction ............................................................................................................................. 189
6.3 Methods ................................................................................................................................... 192
6.4 Results ...................................................................................................................................... 200
6.5 Discussion ................................................................................................................................ 213
6.6 References ............................................................................................................................... 218
Chapter 7: General discussion ................................................................................................... 223
7.1 Optimality’s ever widening net .............................................................................................. 223
7.2 Value: economics, nature, and neural networks ................................................................ 224
7.3 Models as bridges and sirens ................................................................................................ 226
7.4 Concluding remarks ............................................................................................................... 227
7.5 References ............................................................................................................................... 228

5
Acknowledgements
I have had the pleasure of belonging to two laboratories throughout my time at UCL. Both the
Bestmann and Dolan labs have been fantastic places to work, and I’m thankful to all of the
people who made each of them so stimulating and convivial. I’d also like to thank the technical
staff who made my experiments possible and even pleasurable.
My primary supervisor Sven has been extremely accommodating, encouraging me to pursue my
own ideas and supporting me with remarkably consistent good humour. His optimism and
courage have provided a steady source of encouragement throughout. Sven taught me to stay
close to the data and has emphasized how best to spend my energy and time, neither of which
are as infinite as I initially believed. I have felt similarly empowered by Ray, who has advised and
supported me both directly and through his unusual ability to assemble an unbelievably
talented group of scientists. I feel fortunate to have enjoyed such excellent relationships with
not one but two remarkable supervisors.
I have plundered the time of many kind colleagues over the last four years, but none quite so
rapaciously as that of Robb Rutledge. With characteristic imperturbability, gentleness, and
diligence, he has instructed and inspired at every step of the journey. He has been the first port
of call whenever something has gone awry and a source of authority on everything from
participant instruction sheets to how to kill a sheep. I am also very grateful indeed for having
been involved in his tireless public engagement efforts. Our work at the Roundhouse was one of
the highlights of my PhD. Thanks for being such an exemplary mentor and role model.
If Ray and Sven are my scientific fathers and Robb an adored uncle, Zeb Kurth-Nelson has been
an older brother. The many hours we have spent together in the lab and outside it have been
full of grazed knees, lewd jokes, and finding interesting things under rocks. I have learned a
tremendous amount about thinking, science, and thoughtfulness from Zeb, and I look forward
to learning more. Any ideas in this thesis that might be considered interesting are probably
Zeb’s.
I am much indebted to Laurence, Nish, and Steve, who were generous with their data and their
time in helping me to understand it. I am in awe of the craft and patience with which they do

6
science, and the quality of the data that results. It has been a privilege to think about their
neurons with them, and to occasionally make cups of coffee for Laurence.
I have made too many friends at UCL to thank them all. However, Federico deserves a special
mention as a parmesan importer par excellence and a superb housemate. He and Fran have
provided the camaraderie and the Michelin-starred lunches necessary to navigate the
occasionally rough waters of a doctorate, with Peter battering me on the tennis court whenever I
started to get too cocky. When the time came to write the thesis, they were ably succeeded by
my parents, who provided the craft beer, homemade bread, and trips to the lido sufficient to get
me over the line.
It seems traditional to end by acknowledging the contribution made by one’s partner. Steph will
take some pride in confessing that she has chiefly strived to thwart my academic ambitions by
coaxing me up mountains, into vans, and down ski slopes. I might have got to the end of this
degree quicker were it not for Steph; or, more likely, I’d never have got to the end of it at all.

7
Figures and tables
Figure 1.1 | Framework for this thesis .................................................................................................. 12
Figure 1.2 | Rescorla-Wagner captures classical conditioning ........................................................... 19
Figure 1.3 | Dopamine neurons convey an RPE signal ....................................................................... 23
Figure 1.4 | Integrating multiple sources of evidence ........................................................................ 30
Figure 1.5 | Combining priors and likelihoods in space ..................................................................... 32
Figure 1.6 | The Hierarchical Gaussian Filter ....................................................................................... 36
Figure 1.7 | Theoretical and experimental work on uncertainty coding .......................................... 39
Figure 2.1 | Experimental design ........................................................................................................... 61
Figure 2.2 | Descriptive analysis ............................................................................................................ 68
Figure 2.3 | Model-based analysis of happiness ................................................................................. 69
Figure 3.1 | Experimental design ........................................................................................................... 81
Figure 3.2 | Confirmation of stress induction. ..................................................................................... 91
Figure 3.3 | Stress impairs learning to act ............................................................................................ 93
Figure 3.4 | Stress does not affect the parameters of a Pavlovian learning model ........................ 95
Figure 3.5 | Stress alters pupillary responses to action ..................................................................... 97
Figure 4.1 | Task structure and stress measures .............................................................................. 108
Figure 4.2 | Modelling of learning and stress .................................................................................... 123
Figure 4.3 | Assessing models of learning .......................................................................................... 124
Figure 4.4 | Irreducible uncertainty predicts subjective stress ........................................................ 126
Figure 4.5 | Physiological responses reflect uncertainty and surprise. .......................................... 129
Figure 4.6 | Relationship between uncertainty sensitivity and task performance ........................ 130

8
Supplementary Figure 4.1 | Assessing alternative models of subjective stress ............................ 134
Supplementary Figure 4.2 | Additional physiological stress data ................................................... 135
Supplementary Figure 4.3 | Luminance fitting procedure used for model of pupil diameter .... 136
Supplementary Figure 4.4 | Pupillary and skin conductance sensitivity to uncertainty are
uncorrelated ................................................................................................................................... 137
Supplementary Figure 4.5 | Mean and variance of subjective stress ratings are unrelated to
performance ................................................................................................................................... 137
Supplementary Figure 4.6 | Uncertainty-tuning in the pupil is inversely correlated with
Intolerance of Uncertainty ............................................................................................................ 138
Figure 5.1 | Experimental procedure .................................................................................................. 160
Figure 5.2 | Example participants from behavioural experiment ................................................... 161
Figure 5.3 | Selected subjects for scanning experiment .................................................................. 162
Figure 5.4 | Behavioural results for subjects in scanning experiment. .......................................... 163
Figure 5.5 | Representation of quality, quantity, and their interaction .......................................... 165
Figure 5.6 | Computation of utility from component parts in the anterior cingulate cortex ...... 168
Figure 5.7 | Neural quantity sensitivity relate to choice predictability ........................................... 169
Figure 5.8 | Repetition suppression for value in the anterior cingulate cortex ............................. 171
Figure 5.9 | Modelling of repetition suppression: dependence on tuning and adaptation type 173
Figure 5.10 | Dissecting divisive vs. subtractive repetition suppression effects ........................... 175
Figure 6.1 | Gaussian tuning for orientation and linear tuning for value ...................................... 191
Figure 6.2 | Experimental design ......................................................................................................... 193
Figure 6.3 | Recording locations .......................................................................................................... 195

9
Figure 6.4 | Cue 1 responses ............................................................................................................... 201
Figure 6.5 | Removing linear and monotonic value representations ............................................. 204
Figure 6.6 | Visualizing tuning curves ................................................................................................. 206
Figure 6.7 | Cross-attribute distance plots by area ........................................................................... 207
Figure 6.8 | ACC and OFC contain Gaussian-tuned neurons ........................................................... 208
Figure 6.9 | Control analysis: correlating probability and magnitude tuning ................................ 210
Figure 6.10 | Population decoding of value after linear information removal .............................. 211
Figure 6.11 | Neural coding of pitch and volume .............................................................................. 217

Table 2.1| Bayesian model comparison analysis ................................................................................ 73
Table 3.1 | Cardiovascular stress measures ........................................................................................ 92
Supplementary Table 4.1 | Parameters used in pupil model .......................................................... 139
Supplementary Table 4.2 | Parameters used in skin conductance model .................................... 140
Supplementary Table 4.3 | Hierarchical Gaussian Filter details ...................................................... 141
Supplementary Table 4.4 | Details of each learning model ............................................................. 142

10