Appetitive Pavlovian-instrumental Transfer

20 Pages·2016·2.94 MB·English

by  Emilio Cartoni

NeuroscienceandBiobehavioralReviews71(2016)829–848

ContentslistsavailableatScienceDirect

Neuroscience and Biobehavioral Reviews

journal homepage: www.elsevier.com/locate/neubiorev

Reviewarticle

Appetitive Pavlovian-instrumental Transfer: A review

EmilioCartonia,b,∗,BernardBalleinec,GianlucaBaldassarreb

aDipartimentodiPsicologia,UniversitàdiRoma“LaSapienza”,Rome,Italy
bLaboratoryof Co mputation alEmbodie dN euros cien ce,Institut eofCo gnitiveSciencesandTechnologies,NationalResearchCouncilofItaly,Rome,Italy
cBehavioura lN euroscienceLab oratory,B rain&MindCe ntre,Un ive rsityofSy dney,NS W,A ustralia

a r t i c l e i n f o a b s t r a c t

Articlehistory: Reward-relatedcuesareanimportantpartofourdailylifeastheyofteninfluenceandguideouractions.
Receiv ed7February2016 Thispaperrevie wso neo ft heexperim enta lp ara digm sus ed tost udyth eeffects ofcu es,th ePa vlovian
R23ecSeeivpetedm inb ererv2i0se1d6 f orm to In strum ental Tra nsfe r p arad igm. In this p aradigm, c ues as so ciated wit h rewa rds thro ugh Pavlovian
conditioningaltermotivationandchoiceofinstrumentalactions.Thefirsttransferexperimentsdate
Accepted23September2016
backtothe1940s,butonlyinthelastdecadehasitbeenfullyrecognisedthattherearetwotypesof
Availableonline28September2016
transfer,specificandgeneral.Thispaperpresentsasystematicreviewofboththeneuralsubstratesand

thebehavioralfactorsaffectingbothtypesoftransfer.Italsoexaminestherecentapplicationofthe
Keywords:
paradigmtostudytheeffectofcuesonhumanparticipants,bothinnormalandpathologicalconditions,
Pavlovianconditioning
TInrsatnrsufmeren tal conditioning aonfdtr tahnes fienrtetroacbtuioilnds aonf torvaenrsaflelrp wicittuhr edroufgsth oef pabhuesneo.m Fiennaollny,, ftrhoem paepaerlry atnhaelyosreiess thtoe rtehceeonrtethiciearl aarscpheicctasl

PIT accounts.
Review ©2016TheAuthors.PublishedbyElsevierLtd.ThisisanopenaccessarticleundertheCCBY-NC-ND

license(http://creativecommons.org/licenses/by-nc-nd/4.0/).

Contents
1. Introduction...........................................................................................................................................830

1.1. Scopeandpurposeofthereview..............................................................................................................831

2. Thetransferparadigm.................................................................................................................................831

3. Behavioralresults–variablesinfluencingtransfer...................................................................................................833

3.1. Pavlovianfactors...............................................................................................................................833

3.2. Instrumentalfactors...........................................................................................................................834

3.3. Otherfactorsandresults.......................................................................................................................835

4. Neuralsubstrates......................................................................................................................................835

4.1. Amygdalaandnucleusaccumbens............................................................................................................835

4.2. Molecularprocessesinnucleusaccumbens...................................................................................................836

4.3. Dorsalstriatum................................................................................................................................837

4.4. Midbrainstructures............................................................................................................................837

4.4.1. Dopamine............................................................................................................................837

4.5. Prefrontalcortex...............................................................................................................................838

4.6. Neuralcorrelates...............................................................................................................................838

5. Interactionwithdrugsofabuse.......................................................................................................................838

6. Humantransfer........................................................................................................................................840

7. Theoreticalaspectsoftransfer........................................................................................................................841

8. Conclusionsandfuturedirections.....................................................................................................................844

References.............................................................................................................................................845

∗
Correspondingauthorat:LaboratoryofComputationalEmbodiedNeuroscience,InstituteofCognitiveSciencesandTechnologies,NationalResearchCouncilofItaly,

ViaS.MartinodellaBattaglia44,00185,Rome,Italy.

E-mailaddress:[email protected](E.Cartoni).

http://dx.doi.org/10.1016/j.neubiorev.2016.09.020
0149-7634/©2016TheAuthors.PublishedbyElsevierLtd.ThisisanopenaccessarticleundertheCCBY-NC-NDlicense(http://creativecommons.org/licenses/by-nc-nd/4.

0/).
830 E.Cartonietal./NeuroscienceandBiobehavioralReviews71(2016)829–848

toshockavoidancebutdecreaseactionsleadingtofood(Rescorla

Glossary andSolomon,1967).

Ourunderstandingoftransferhasnaturallydevelopedwithour
Pavlovian conditioning: During Pavlovian conditioning a
understandingofPavlovianandinstrumentalconditioning.Atthe
neutralstimulus,suchasasound,becomesacondi-
sametime,studyingtheinteractionofPavlovianandinstrumental
tionedstimulus(CS)bypairingitsoccurrencewith
conditioninghasoftenyieldednewinsightsintotheseindividual
anunconditionedstimulus(US)thatnaturallyelic-
processes.Atthetimeoftheearlieststudies,forexample,itwas
itssomeresponse.Forexample,asound(CS)might
not clear if Pavlovian and instrumental conditioning constituted
be paired with food (US) by delivering food only
differentformsoflearning.Gradually,however,two-processtheo-
whenthesoundispresent.Attheendoftraining,
riesemergedthatseparatedPavlovianandinstrumentalprocesses
theanimal/participantwillhavelearnedthattheCS
(see Rescorla and Solomon (1967) for a review). Transfer effects
predictstheUSandsoitwillapproachthesiteof
were,atthatstage,understoodastheresultofPavloviancuesgen-
fooddeliverywhenithearsthesounds.
eratinggeneralappetitiveoraversiveemotionalstatesand,indeed,
Instrumental conditioning: During instrumental condition-
thetransferparadigmwastypicallyusedtostudytheinfluenceof
ing an animal/participant is trained to make a
conditioned emotional responses (Rescorla and Solomon, 1967).
response by delivering an attractive outcome. For
Subsequent studies refined this general emotional state finding
example,ahungryratmightbetrainedtopressa
that,inmanyconditions,transferwasbettercharacterisedascon-
lever that delivers food. This training can lead to
trolledbyprimarymotivationalprocessesthanemotionalstates.
two kinds of instrumental behavior: habits, con-
So,forexample,Dickinsonandcolleaguesdemonstratedinstud-
trolledbyantecedentstimulithroughtheformation
iesofthesocalledirrelevantincentiveeffectthatacuepredicting
of stimulus-response (S-R) associations or goal-
sugarsolutionwouldenhanceinstrumentalactionsbothwhenrats
directedactions,controlledbytheconsequencesof
were hungry and when they were thirsty whereas a cue associ-
theactionthroughtheformationofaction-outcome
atedwithdryfoodpelletswouldonlyelevateperformancewhen
(A-O)associations.
hungry. These effects were generally interpreted as suggesting
Devaluation: Outcomedevaluationisaprocedurewherethe
thatprimarymotivationalprocessescouldmodulatetheproduc-
USortheoutcome(O)valueisaltered.Forexample,
tionofconditionedemotionalstates,muchassuggestedbyBindra
thevalueofacertainfoodmightbealteredbyfeed-
(DickinsonandDawson,1987;Balleine,1994;Bindra,1974).How-
ingittosatiationorbypairingitwithillness(the
ever,noneoftheseaccountsexplainedtheinfluencesofPavlovian
latterinducedbylithium-clorideinjections).
cuesonchoice:i.e.,howPavloviancuescouldsometimesbefound
Extinction: A training session where the US predicted by a
toenhanceactionstiedtoaspecificoutcome,e.g.howaCSassoci-
CSortheoutcome(O)predictedbyanactionisno
atedwithgrainpelletsenhancedleverpressingforgrainpelletsbut
longerdelivered,thuspromotingtheextinctionof
notforotherfoodoutcomes(suchassugar).Onepossibilityisthat
the Pavlovian conditioned response or the instru-
bothPavlovianandinstrumentalconditioningleadtotheforma-
mentalaction.
tion of associations: stimulus-outcome associations (S-O) in one

case and response-outcome associations in the other case (R-O),

andthatthecommonoutcomemediatestheinteraction(Trapold

1. Introduction and Overmier, 1972). In the 1980s and 90s much experimental

workwasdevotedtoestablishingthatinstrumentalconditioning

Predictivecuesareanimportantpartofourlifethatcontinu- could be subdivided into two types: habitual actions, controlled

ouslyinfluenceandguideouractions.Hearingthesoundofahorn bystimulus-response(S-R)associations,andgoal-directedactions,

makes us stop before we attempt to cross the street. Seeing an controlledbyresponse-outcome(R-O)associations(Balleineand

advertisementforfastfoodmightmakeushungryandleadusto Dickinson,1998).Inparallel,aseriesofarticlesexaminedtheabil-

seekoutaspecifictypeandsourceoffood.Ingeneral,cuescanboth ity of Pavlovian cues tied to a specific outcome to bias choice

promptustowardsorstopusfromengaginginacertaincourseof betweenspecificactions(ColwillandRescorla,1988;Colwilland

action.Theycanbeadaptive(savingourlifeincrossingthestreet) Motzkin,1994;Rescorla,1991,1994a,b;Delamater,1995,1996).

ormaladaptive,leadingtosuboptimalchoices,e.g.makinguseat At the beginning of the century, investigationof the neural sub-

whenwearenotreallyhungry(ColagiuriandLovibond,2015).In stratesoftransferbegan:someinitiallycontrastingresultsledat

extremecasestheycanevenplayapartinpathologiessuchasin thatstagetotherealisationthattransfereffectscomeintwodif-

addiction,wheredrugassociatedcuesproducecravingandprovoke ferent forms and so had to be subdivided as well. These studies

relapse(Belinetal.,2009). dividedthephenomenonintospecificandgeneraltransfer,each

Oneparticularparadigmusedtostudytheeffectofsuchcuesis characterizedbyadifferentneuralsubstrate(CorbitandBalleine,

thePavloviantoInstrumentalTransferparadigm.Inthisparadigm, 2005,2011,seeSection4).Specifictransferreferstotheabilityof

Pavlovianpredictionsandinstrumentalactionsarefirsttrainedin cuestoenhancespecificactionsassociatedwiththesameoutcome

separateexperimentalphases.Theinstrumentalactionsarethen asthecue,whereasgeneraltransferreferstotheabilityofcuesto

testedinboththepresenceandtheabsenceofthePavloviancues enhancealsoactionspairedwithdifferentoutcomes.
toasse ss thee ffec tofthelat tero nth eforme r. Most data on tra nsfer c ome from ani mal studies,1 however

The first Pavlovian to instrumental transfer studies date back in recent years the transfer paradigm has also been adapted for

tothe1940s,reportingthatstimulipairedwithfoodwereableto humanparticipants.Ingeneral,humanstudieshaveproducedsim-

augment instrumental actions directed towards food (Estes and ilarresultstoanimalstudiesbothintermsofthebehavioralfactors

Skinner, 1941; Walker, 1942; Estes, 1943). Transfer effects can

either promote or discourage actions, with the presence of cues

increasing/decreasingthefrequencyofanactionorbiasingchoice

in favour of certain actions. Amongst other factors, the type of
1 Studiesmostlyinvolvedrats,howeverotherspecieshavebeenusedaswell,
effect obtained depends on the valence of the Pavlovian US, i.e.,
suchasmice(seeLederleetal.,2011indifferentmicestrains),monkeys(Stebbins
whetheritisappetitiveoraversive.Forexample,aPavloviancue
andSmith,1964),dogs(RescorlaandLoLordo,1965),pigeons(MorseandSkinner,
associate d wi thanaver siv eshockm igh tpromote a ctionslead ing 195 8)ande venho rses (Lansadee tal. ,2013).
E.Cartonietal./NeuroscienceandBiobehavioralReviews71(2016)829–848 831

Fig.1. Articlesreferencedinthisreview,groupedin5yearsbins.Lettersindicatelandmarkeventsintransferresearch.Thelastdecadehasseenamarkedincreaseinthe

investigationoftransfer.(a)Firstexperimentsinthe1940s.(b)RescorlaandSolomon(1967)reviewtheoriesofPavlovianandinstrumentalconditioningandtheirinteraction,

advocatingatwo-processtheory.(c)Duringthe1980sand90sinstrumentalconditioningwasclearlysubdividedintohabitual(S-R)andgoal-directed(A-O)actions.Around

1990,RescorlaandcolleaguesproduceaseriesofstudiesexamininghowPavloviancuesaffectinstrumentalrespondinginanoutcome-specificmanner.(d)Attheturnof

thecenturyinvestigationoftheneuralsubstratesoftransferbegins,leadingtothesubdivisionofgeneralandspecifictransfereachwithitsseparateneuralsubstrate.(e)The

transferparadigmisadaptedtohumanparticipantsandneuralsubstratesareinvestigatedusingfMRI.

andtheneuralsubstrates(e.g.,Brayetal.,2008;Talmietal.,2008; conditioninginsteadwillalwaysbereferredtoas‘Pavloviancondi-

Watsonetal.,2014,seeSection6)controllingtransfereffects. tionedresponses’,‘conditionedresponses’orsimplyas‘responses’.

1.1. Scopeandpurposeofthereview 2. Thetransferparadigm

In the last decade there has been an increasing number and There are many variations of the transfer paradigm but it is

rangeofstudiesontransfer,bothinanimalsandhumans,examin- alwayscomposedofthreephases:Pavloviantraining,instrumental

ingtransferunderbothnormalandpathologicalconditions(Corbit trainingandthetransfertest.Thetwotrainingphasescanbecon-

etal.,2007;CorbitandBalleine,2011;Laurentetal.,2015;Ostlund ductedinanyorder(eitherPavlovianorinstrumentalfirst)with

etal.,2014b;Nadleretal.,2011;Morrisetal.,2015;Colagiuriand nochangeintheeffect(butseeHolmesetal.,2010,wherelength-

Lovibond,2015;Garbusowetal.,2016).Althoughtherehasbeena eningthefirstorsecondphasevariedtheamountoftransfer).In

relativelyrecentreviewonthetopic(Holmesetal.,2010)thelit- thePavlovianphaseoneormorestimuli(usuallyauditorycues)are

eratureontransferhasessentiallydoubledinsizeoverthelast5 pairedwiththedeliveryofrewardssuchasfoodpelletsorsucrose

yearsprovidingconsiderablenewinformationonmodelsandthe (seeFig.2).Pairingstimuliwithanaversiveeventtodevelopan

neuralbasesofthetransfereffectacrossspecies.Inparticular,we aversive transfer paradigm has also been conducted (e.g., Lewis

willfocusonappetitivetransferwhichisthesubjectofthelarge etal.,2013;Rigolietal.,2012;Campeseetal.,2013).

majorityoftheserecentstudies.Fig.1summarisesourcoverage Duringtheinstrumentaltraining,acontingencyisestablished

ofresearchandshowssomelandmarksintheinvestigationofthe betweenoneormoreactionsandthedeliveryofoneormoreout-

transfereffect.Inourreviewweincludedonlyarticleswhichfollow comes–usuallyinvolvingpairingleverpressingwithfooddelivery.

thestandardtransferparadigms:i.e.wherethePavloviancondi- Usingoneormoreactions(e.g.oneortwoleverspairedwithdif-

tioningandinstrumentalconditioningareconductedinseparate ferentfoods)leadstocriticaldifferencesinwhatismeasuredinthe

sessions. finaltest(seebelow).

The review is structured as follows: Section 2 describes the Inthelastphase,theanimal/participantcanagainperformthe

transferparadigmanditsvariants;Section3thenreviewsbehav- instrumental actions, but this time the conditioned stimuli (CS)

ioral factors influencing the transfer effect; Section 4 describes trainedinthePavlovianphasearepresentedduringthesession.The

the neural mechanisms underlying transfer; Section 5 reviews effectofpresentingtheCSontheinstrumentalresponse(thetrans-

studies of the interaction between transfer and drugs of abuse; fereffect)isthenassessedbycomparinginstrumentalresponding

Section6reviewstransferstudieswithhumanparticipants;Sec- during periods when no CS is presented (baseline) with periods

tion7reviewstheoreticalaspectsoftransfer;Section8thendraws whenaCSispresented,or,iftwoCS’sarepresented,bycomparing

overallconclusions.Wherepossiblewewillseparateexperimental respondingduringthepresentationofthedifferentCSs(e.g.,one

findings from their theoretical interpretation using procedural pairedwiththesamefoodastheleverandanotheronepairedwith

ratherthantheoreticaldescriptionsordefinitions.Inthisregard, adifferentfood).TheCSsareneverpresentedinthepresenceofthe

weclarifyhereouruseofwhatmaybeseenasatheoreticallyladen instrumentalmanipulandabeforethetest,sonoexplicittraining

term‘action’.Weuse‘instrumentalaction’orjust‘action’torefer ofarelationshipbetweentheCSandtheinstrumentalactiontakes

to measures in the instrumental conditioning phase of transfer place.Whenmultipleactionsareused,thetestmayinvolveachoice

studies, whether the experiments were designed to encourage betweentwoleverspresentedatthesametime(e.g.,Ostlundand

the development of goal-directed instrumental actions or of Balleine, 2008) or separate tests of each instrumental action

habitual instrumental actions. Measures taken from Pavlovian (e.g., Corbit et al., 2001). Test sessions are usually conducted in
832 E.Cartonietal./NeuroscienceandBiobehavioralReviews71(2016)829–848

Fig.2. Transferparadigms.Weillustratethreepossibleexperimentalsetups(“Transferparadigms”)thathavebeenusedtotestthetransfereffect.Ontheleft(“Single-lever

paradigm”),onlyoneCSandoneleveraretrainedwitharewardandanunpairedCSisusedascontrol.Thisfirstparadigmusuallyleadstotheexpressionofgeneraltransfer.

Inthecentre(“Specifictransferparadigm”),twoCS+andtwoleversaretrained,usingtworewards.EachCS+isassociatedwitharewardusedforonlyonelever,thusenabling

theexpressionofspecifictransfer.Ontheright(“Fulltransferparadigm”),thespecificparadigmisenhancedbyaddinganadditionalCS+.ThislastCS+isassociatedwitha

thirdoutcomethatisnotusedforinstrumentaltraining.UsingthisthirdCS+duringthetestphaseprovidesatestofgeneraltransfer.Theparadigmontherightisthusa“full

transferparadigm”inthesensethatitcantestbothspecificandgeneraltransfer.Thebottomrowprovidesschematicgraphsthatexemplifytypicalresultsobtainedinthe

threeparadigms.

extinction,i.e.,nooutcomesaredeliveredeitherafterthestimuli the Pavlovian phase and in the test phase cannot behaviorally

oraftertheactions,toavoidchangesinperformanceduetonew distinguish between the specific and general transfer effects.

learning. Thisisbecausebothgeneralandspecifictransfereffectsenhance

In all cases, what is generally found is that a CS paired with instrumental performance, something that could be because the

an appetitive outcome (CS+) enhances instrumental responding action shares the same outcome as the CS or through a general
com paredtoa nunpaire dCS( CS−).Usua llytheCS+al soincreases effect of the cue (e.g. m otivatio nal ). Fo r s imp licity, th ro ughout

instrumentalresponsescomparedtothebaseline(CS-freeperiod), this review we will call these studies “single lever studies” (see
however,ins omecases, difference sb etw eentheC S+andC S−have Fig. 2).Itmu stb enot edt hatwh ileitisn otposs ibleto behavio rally

emerged but with no difference between the CS+ and baseline, distinguishbetweenspecificandgeneraltransferina“singlelever

producing instead a reduction of CS-induced response suppres- study”,lesionsexperimentssuggestthatusingasingleleverusu-

sion. Furthermore, in the two action case, although typically the allyelicitsonlygeneraltransfer.AswewillseeinSection4,studies

‘same’ CS elevates performance of the action delivering the out- investigatingtheneuralbasisoftransferhavefounddistinctneural

come predicted by the CS relative to both the other action and substrates for specific and general transfer (Corbit and Balleine,

to baseline performance,it has sometimes been found that both 2005,2011),andtransferin“singlelever”studiesisimpairedby

CSsreduceleverpressingcomparedtothebaselinebutthattheCS lesionstargetinggeneraltransfersubstratesandsparedbythose

sharingthesameoutcomeastheinstrumentalactionreducesper- targetingspecifictransfersubstrates(Halletal.,2001;Hollandand

formancelessthanaCSassociatedwithadifferentoutcome.This Gallagher, 2003). It is not known why the “single lever” studies

reduction with respect to the baseline could be due to response elicitgeneraltransferandnotspecifictransfer(asmultiplelever

competition between instrumental and Pavlovian responses. For studiesdo).Ithasbeensuggestedthatthedifferenttypeoftransfer

example, if the CS prompts considerable magazine approach, it elicitedbysingleanddoubleleverproceduresmightbecausedby

will reduce the time spent pressing the lever (see also next themoreorlessdetailedrepresentationoftheoutcome(Holland,

section). 2004). Procedures with multiple levers (and reinforcers) favour

As we mentioned in the introduction, there are two kinds of thecreationofamoredetailedandsensory-specificrepresentation

transfer:specifictransferandgeneraltransfer.Inspecifictransfer, of the reinforcers used, which in turn may lead to the transfer

theCSenhancesactionsassociatedwiththesameoutcomeasthat effect being specific. A single-lever lever procedure does not

pairedwiththeCSwhereas,ingeneraltransfer,aCScanenhance needadetailedrepresentationoftheoutcome,sointhiscasethe

actionsdirectedtootheroutcomesaswell.Studiesusingasingle transfereffectmightbeconveyedbythemore“general”appetitive

lever paired with food usually also use one single paired CS in characteristicsoftheoutcome.
E.Cartonietal./NeuroscienceandBiobehavioralReviews71(2016)829–848 833

Anothercategoryofstudiesusestwolevers,eachpairedwith OntherelationshipbetweentransferandPavlovianextinction,we

different outcomes (e.g. food pellet vs. sucrose) and usually two alsonotethatinasingle-leverhumantransferparadigmLovibond

CSseachpairedwithoneoftheseoutcomes.Inthiscase,duringthe etal.(2015)found,incontrasttotheanimalstudies,thatPavlo-

testphase,eachCSusuallyonlyenhancesleverpressingassociated vianextinctionoftheCSaffectedtransfer.AlthoughthePavlovian

withthesameoutcomeastheCS.Wewillcallthisvariantthe“spe- extinctiondidnotcompletelyeliminatethetransfereffect,itseffi-

cifictransferparadigm”(seeFig.2).Inthisparadigm,whentesting cacycontrastedwiththeresultsobtainedinanimalsbyDelamater

transferoneleveratatime,theCSpresentationscanbedivided (1996).Lovibondetal.(2015)suggestedthatoneofthecriticaldif-

into two conditions: in the same condition the CS and the lever ferencesmightlieinthefactthattheyfocusedonabsoluteresponse

sharethesameoutcomewhereasinthedifferentordiffcondition rate(usingasingle-leverparadigm)whereasinDelamater’s(1996)

theCSpredictsadifferentfoodcomparedtotheoneassociatedwith studythetransfertestinvolvedaresponsechoice(specific-transfer

thelever.UsuallythedifferentCSdoesnotenhanceleverpressing paradigm). This suggestion predicts that Delamater’s extinction

relative to the baseline or it does so to a lesser extent than the procedurewouldhavebeeneffectiveifhehadtestedhisactions

specific transfer effect in the same condition (see Fig. 2, bottom individuallyratherthaninachoicetest.However,giventhatsingle-

graphs). It is still unclear why the different lever is not equally leverparadigmsseemtoevokegeneraltransfer,itremainspossible

enhanced through a general transfer effect, especially when, in thatCSextinctiondifferentiallyaffectsspecificandgeneraltransfer.

experimentalsituationsabletoexpressboththespecificandgen- And,ofcourse,thesameislikelytobetrueofconflictbetweenthe

eraltransfereffects,thetwoeffectstendtohaveacomparablesize CRandtheinstrumentalaction;suchconflictcoulddirectlyalter

(seeFig.2). responsevigorandsothesizeofanygeneraltransfereffectbuthas

This last category of experiments is usually conducted using difficultyexplainingvariationsinspecifictransferbecauseanygen-

two levers, each delivering a different outcome, and with three eraleffectonperformanceshouldhavesimilareffectsacrossboth

CSs,twopairedwiththeoutcomesdeliveredbytheleversandone actions.

paired with a third outcome. During the test, this paradigm has The duration and timing of the CS can also affect transfer;

beenreportedtoshowboththespecificandthegeneraltransfer Crombag et al. (2008a) found in mice that a CS+ lasting 10s

effects.ThetwoCSspairedwiththesameoutcomesasthelevers (with reward delivered during the last 5s of the stimulus) pro-

enhancerespondingontheleversharingthesameoutcome(spe- ducedstrongconditionedreinforcementbutnotransfer,whereas

cific transfer). Again we note that the CS+ paired with the food a CS+ lasting 2min (with rewards delivered randomly during

associated with one lever does not enhance responding on the the interval) produced robust transfer but no conditioned rein-

other lever although the third CS, paired with an outcome that forcement. Delamater and Holland (2008) also examined the

wasnotusedintheinstrumentaltraining,enhancespressingon effects of varying the CS-US interval: results confirmed that

both levers (general transfer). We will call this the “full transfer sensory-specificstimulus-outcomeassociations(i.e.,thoseunder-

paradigm”. lyingspecifictransfer)wereestablishedoverawiderangeoflong

butnotshortintervals.Theseresultsarealsoconsistentwitholder

studies, e.g. Meltzer and Brahlek (1970) where a long CS (120s)
3. Behavioralresults–variablesinfluencingtransfer
ledtopositivetransferwheareasashortCSledtosuppressionof

responding, althought in that case the CS’s were trained during
There are many behavioral factors affecting transfer effects.
instrumentalconditioningandsoitwasnotacanonicaltransfer
These are analysed in detail in this section. designasdefi nedhere.2De lam ate ra ndH olla nd (2008)al sofound

that conditioned responses (magazine approach) had an inverse

3.1. Pavlovianfactors relationshiptotheCS–USinterval,withlongerintervalsleadingto

alowermagazineapproachperformance.Assuch,measuresofcon-

ResponsecompetitionbetweenPavlovianresponsesandinstru- ditionedrespondingsuchasmagazineapproachdonotnecessarily

mental responding can make transfer effects harder to detect correlatewithmeasuresofspecifictransfer.Thishasalsobeencon-

(Lovibond,1983).Aswenotedintheprevioussection,aCS+usu- firmedbythefindingthatitispossibletohaveaCSwhichreduces

allyenhancesinstrumentalrespondingcomparedtothebaseline. magazineapproachbutstillincreasesinstrumentalactions,aswe

However,thePavlovianresponseselicitedbytheCS,suchasmag- will see below (Shiflett, 2012). Delamater and Oakeshott (2007)

azine approach, can compete with lever pressing and lead to a furnishesadditionalsupportforadissociationbetweenmagazine

reduction of lever pressing compared to the baseline. Using dis- approach and transfer. In a specific transfer paradigm they gave

crete(e.g.alightcue)vs.diffuse(asound)cuesastheCScanfavour ratsdifferentamountsofPavloviantraining,rangingfrom4to112

thedevelopmentofcompetingsign-trackingresponses(e.g.appro- presentationsofa60sCS.Duringthetestsessions,theamountof

achingtheCS)whichcancompetewithinstrumentalresponses,or magazine approach varied greatly between the different groups,

favour them if the cue is located near the manipulanda (Tomie, with more training leading to more magazine approach. Also, a

1996).ThedegreeofsimilaritybetweenPavlovianandinstrumen- tendency to concentrate approaching during the last part of the

talresponsesmightalsohelporhinderthetransfereffect(Baxter CS(whentheUSwaspreviouslydelivered)onlydevelopedforthe

andZamble,1982).Holmesetal.(2010)showedthatextinguish- group with most Pavlovian training. In contrast, the amount of

ingPavlovianresponsescanenhancetransferinasubsequenttest. transfer displayed was less influenced by length of training and

This result might be seen to conflict with previous experiments theincreaseinleverpressingwasmorepronouncedtowardsthe

byDelamater(1996)inwhichthetransfereffectwasshowntobe endoftheCSforallbuttheshortesttraininggroup.Soboththe

unalteredaftervarioustypesofCSextinctionsuchasnonreinforce- amountandtimingofmagazineapproachandtransferappearto

ment,pairingwithanalternativeoutcomeandexposuretorandom developduringtrainingatdifferentrates.

orexplicitlyunpairedS-Ocontingencies.Thedifferencemightlie Backward conditioning, in which the US precedes the CS,

inthelengthofpreviousCStraining,whichinthecaseofHolmes can also support transfer (Delamater et al., 2003; Shiflett, 2012;

etal.(2010)wasdeliberatelyextendedtocreatestrongPavlovian Cohen-Hatton et al., 2013); however depending on the US-CS

approachresponses.Inotherwords,Pavlovianextinctionmightbe

beneficialtothetransfereffectonlywhenthePavloviantraining

issufficientlystrongthatitinterfereswithinstrumentalrespon-
2 ButseealsoVanDyne(1971)orLovibond(1981)forshortCSleadingtosup-
ding (in the ways described above) but has no effect otherwise.
pression.
834 E.Cartonietal./NeuroscienceandBiobehavioralReviews71(2016)829–848

interval, backward conditioning can produce either Pavlovian Lastly,wenotethatvandenBosetal.(2004)triedtovarythe

excitorsorPavlovianinhibitors,i.e.CSsthat,respectively,enhance amountofrewardassociatedwiththeCS(1or3pellets)butfound

orsuppressPavlovianresponsessuchasconditionedapproach.As noeffectofrewardmagnitudeontransfer.

aconsequence,theinfluenceofbackwardconditioningontransfer

is complex and its application in a transfer paradigm can result 3.2. Instrumentalfactors

in either positive (Shiflett, 2012; Cohen-Hatton et al., 2013) or

‘negative’transfereffects(Delamateretal.,2003).Thelattereffects, The amount of training and type of reinforcement schedule

observedinoutcomespecifictransfer,oftenreflecttheoppositeof canaltertheamountoftransferattest.Testingdifferentvariable-

thestandardexcitatoryspecifictransfer;i.e.,ratherthanelevating interval(VI)schedulesMeltzerandHamm(1974)foundthatVI’s

theactionthatdeliveredthesameoutcomeasthatpredictedby withlongerintervals,whichleadtolowerresponserates,produced

thestimulus,abackwardlypairedCScanbiaschoicetowardsother strongertransfereffects.Thismightbeduetotransferbeingeasier

actionsthatanimalshavelearneddonotdeliverthebackwardly todetectonalowerbaselineandindeedmanytransferexperiments

pairedUS.ThiswasexplicitlyinvestigatedbyLaurentandBalleine includeaperiodofinstrumentalextinctionpriortothetestonthe

(2015)whocomparedtheeffectofazerodelayandofa10-sdelay viewthatitmakespositivetransfereasertodetect(Dickinsonetal.,

between US and CS finding standard specific transfer with the 2000). Lovibond (1981) used discrimination training on a single

formerandthe‘negative’orreversedtransfereffectwiththelatter. leverinwhichanimalshadtodiscriminatebetweentwoalternating

Sucheffectssuggest,therefore,thatitistheinformationconveyed periodsofreinforcement(onaVR15scheduleS+,chamberfanson)
by th e back wardly p aired CS t hat is im por tant for the direction andnon -r einforcement( S−,c h ambe rfansoff ).D uringthe tran sfer

ofchoice;i.e.,thattheCScanprovidebothinformationaboutthe test,thepresenceoftheCSsuppressedrespondingintheS+period
lik elihood ofa forth com in gou tcomefo ruse inactionsele ction, but whe area sduring the S− pe riodtheCS producedp os itiv etr ansfer

canalsoprovokedirectchangesinconditionedreflexesandthat whichlastedbeyondthedurationoftheCSitself(10s).Thisiscon-
thes ene ednotbe thesa me;inde ed itisimporta nttono tein this sistent witht heabov erep ortedfin di ngs bec ause the S− peri od was

context that the effect of backward conditioning on transfer can indeedaperiodwithlowerbaseline.Lovibond(1981)alsoverified

be dissociated from its effects on conditioned approach. Shiflett thattheenhancementofleverpressingoccurredwhenthebaseline

(2012) managed to train a backward CS that exhibited positive wasloweredbyothermeans.Inasecondexperiment,hetestedthe

transfer but suppressed conditioned approach responses (i.e., CS after rats had achieved a low baseline due to satiety: in this

CS presentation increased lever pressing but reduced magazine case, the CS tended to suppress responding rather than enhance

approach compared to baseline). Interestingly, in Laurent et al. it. This result tells us that it was probably not a lower baseline

(2015) and in Laurent and Balleine (2015) a backward condi- per se that produced the stronger transfer effects but rather the

tioningprocedurewasusedtoobtainoutcome-specificPavlovian interactionbetweentheexpectancyoffoodgeneratedbytheCS

inhibitorsfortwodifferentrewards.Whentestingspecifictransfer, andtheexpectancyoffoodcontrolledbytheinstrumentalsched-

thesebackward-CSsdidnotreducepressingontheleversharing ule(Lovibond,1981).Inotherwords,theCSismoreeffectiveon

the same reward as the CS, instead, they increased pressing on anextinguishedbaselinebecauseitbringsanexpectationoffood

theleverpairedwiththeotherreward.ThiswasusedinLaurent when the current expectation is low. The reduction in transfer

and Balleine (2015) to demonstrate that rats can engage in a observedwhentestingundersatietycanalsobeseenasconsistent

formof“counterfactualreasoning”,meaningthattheycanusethe with subsequent results by Corbit et al. (2007) and Aitken et al.

informationaboutanabsentreward(furnishedbythebackwardly (2016)whichshowedthatsatietycanabolishgeneraltransfer(see

paired CS) to promote the selection of actions associated with nextsection).

other outcomes (in this case increasing responding on the other Beyond this, Holland (2004) showed that longer training on

lever).Oneimportantaspectofthesestudieswasthefindingthat a VI schedule leads to an increase in the transfer effect when

specific inhibitory predictions mirrored the effects of excitatory assessedinasingle-leverparadigm.Subsequently,Wiltgenetal.

predictions and altered action selection quite specifically away (2012) showed that mice trained under a VI schedule exhibited

fromtheactiondeliveringtheoutcomethattheinhibitorystim- moretransferthanthosetrainedunderarandom-ratio(RR)sched-

ulus predicted would be withheld. To confirm that this effect of uleusinganoutcomespecifictransferdesign.However,although

backwardly pairing of CS and US was due to inhibition, Laurent thegrouptrainedontheRRscheduleswassensitivetodevalua-

and Balleine (2015) went on to assess the effects of conditioned tion–whereasagrouptrainedontheVIscheduleswasnot–both

inhibitorsestablishedusingothermethods;i.e.,afeature-negative groupsshowedsimilarratesofperformanceduringtraining.This

procedureandanoverexpectationprocedure.Bothofthesepro- isunusual;itiscommonlyfoundthatratioschedulespromotefar

ceduresproducedidenticaleffectstothoseinducedbybackward higher rates of responding than interval schedules (see Dawson

conditioning; i.e., whereas the conditioned excitors elevated andDickinson,1990)suggestingthat,forsomeunspecifiedreason,

performance of the action with which they shared an outcome, themiceintheRRgroupmayhavefailedtodetectthefullimpact

theconditionedinhibitorsproducedashiftawayfromthespecific oftheratiocontingency.Furthermore,whattransferWiltgenetal.

action delivering the outcome the inhibitor predicted would be (2012)reportedwasclearlynotspecifictransfer;boththesameand

withheld. In summary, excitatory and inhibitory conditioning differentactionswereelevatedrelativetobaselinebutdidnotdif-

exertsymmetricallyopposingeffectsonspecifictransfer;biasing ferfromeachother.Theauthorsconcludedthatrespondingthatis

choicetowardsorawayfromanactionbasedontheinformation insensitivetodevaluation,andsohabitual,mightbemoresensitive

providedbythecuesregardingtherelativelikelihoodofearning totransfereffectsthangoal-directedactionsand,whilethiscould

somespecificoutcomeorother.Inhumans,AlarcónandBonardi bethecase,itisimportanttoqualifythisstatement;theevidence

(2016)usedaspecifictransferparadigminwhichtheyalsotrained suggeststhathabitualactionsaremoresensitivetogeneraltrans-
aPavlo vian in hibitor usingaf eature-neg at ivedes ign( i.e.A →O1, ferthang oal- directed actions. Ift hisist rue,this eff ectwou ldalso
A X→∅,wit hXbeing thes ti mulustrainedas aninhi bitor ). W hen exp lain Holland’s(200 4)result s, inw hi chtra nsfe rcorr elatedw ith

theinhibitorwaspairedwithanotherCSassociatedwiththeinhib- theamountofVItraining,withtheeffectgrowinglargerasinstru-

itedoutcomeduringthetransfertest,itabolishedspecifictransfer mentalperformanceshiftedfromgoal-directedtohabitualcontrol.

andtheparticipantsinsteadshowedatendencytorespondonthe Furthermore,inanumberofstudies,Balleineandcolleagueshave

otheravailableresponse,parallelingLaurentandBalleine’s(2015) trained rats using ratio schedules and found very clear evidence

results. ofspecifictransfer(CorbitandBalleine,2005,2011;Laurentetal.,
E.Cartonietal./NeuroscienceandBiobehavioralReviews71(2016)829–848 835

2012,2014;LaurentandBalleine,2015).Assuch,Wiltgenetal.’s speculated that the differential training reduced the effect of CS

(2012)resultsmaybeanomalous,resultingfromaneffectspecific presentation by making the contexts more informative and thus

toinstrumentaltrainingratherthantothetransfertestperse,or renderingthecuesredundant.Wewilldiscussthisresultfurtherin

mayreflectaneffectongeneralasopposedtospecifictransfer.On Section7.

balanceitseemsmorelikelythatthelatteristhecorrectconclusion. Somestudieshaveexploredtherelationshipbetweentransfer

andstressbothacuteandchronic,andbetweentransferandstress-

3.3. Otherfactorsandresults relatedmoleculessuchascorticotropinereleasingfactor(CRF)or

glucocorticoids. Zorawski and Killcross (2003) tested the effects

Inameta-analysisof30transferstudies,Holmesetal.(2010) ofdexamethasone,aglucocorticoidreceptoragonist,inaspecific

foundacorrelationbetweenthedegreeoftransferandtheamount transferparadigm.Theyfoundthatadministeringdexamethasone

ofphase1andphase2training,regardlessofwhetherthePavlo- at the end of Pavlovian sessions impaired the ability of the CS

vianorInstrumentaltrainingwasconductedfirst.Increasingphase trainedduringthosesessionstoevokespecifictransfer.Asimilar

1 training seems to increase transfer, whereas increasing phase resultwasalsofoundbyPielocketal.,2013b.In2006,Pecin˜aetal.

2 training seems to decrease it. It is not clear why this training examined the effects of CRF microinjections in medial shell and

effectemergedbut,presumably,ithassomethingtodowiththe foundthatthehighestdoseof500ngCRF(butnotthe250ngdose)

stabilisation of Pavlovian and instrumental learning and, there- enhancedtransferinasingle-leverparadigm.Later,Morgadoetal.
fore,therela tiv elyconsiste ntim pactoftheo utcomep redic tionon (2012)has foundt ha tc hronicstres s3canred ucesp ecifictran sf er.

instrumentalperformance.Perhapswithlesstrainingtheassocia- Astress-freeperiodreversedtheeffect.Inahumantransferstudy

tivestrengthoftheCSisstillbelowasymptoteanditsassociative (Pooletal.,2014)acutestresswasalsofoundtoenhancetransfer

statusrelativelyambiguouscomparedtobettertrainedCS’s.Spe- althoughPielocketal.(2013a)failedtofindaneffectofacutestress

cifictransferhasbeenshowntobeimmunetodevaluation:thatis, ontransferinrodents,withboththesestudiesusing“singlelever”

pairingafoodoutcomewithillness(LiCl-induced)priortothetest paradigms.Soares-Cunhaetal.(2014)alsofoundthatinuteroexpo-

doesnotreducethesizeoftheeffectinasubsequenttest(Holland, suretoelevatedlevelsofglucocorticoidsimpairedbothspecificand

2004;seealsoRescorla,1994busingdiscriminativestimuli).How- generaltransferinrats.Reducedlevelsofdopaminewereobserved

ever,Corbitetal.(2007)foundthatdevaluationbysatiationwas inprefontralandorbitofrontalcorticesandnormalizationofthese

able to eliminate general transfer while sparing specific trans- levels(usingeitherL-DOPAoraD2/3agonist,butnotaD1ago-

fer. This is in accord with earlier results that found no effect of nist)restoredtransfer.Inhumans,Quailetal.(2016)examinedthe

devaluationonspecifictransfer(Holland,2004)butapparentlyin relationshipbetweenscoresontheDepressionAnxietyandStress

contrast with Holland’s (2004) results in which devaluation did Scale(DASS)andtransfer,usingafulltransferparadigmwhichalso

not appear to affect transfer in a single-lever paradigm. Never- includedafourthcueassociatedwithnoreward.Participantswith

theless, as suggested above, it is difficult to distinguish specific higher scores on the Anxiety and Stress subscale showed higher

andgeneraltransferusingthiskindofdesignandsothisfailure cue-drivenresponsevigour,meaningthattheyshowedincreased

tofindaneffectcouldbeduetoanyspecifictransfercomponent instrumental responding even in the presence of the fourth cue

engagedbyHolland’stask.Itisalsopossiblethatthisdiscrepancy associatedwithnoreward.Highanxietyparticipantsalsoseemed

isduetothedevaluationmethodused(satiationvsillness)orper- toshowasomewhatbluntedspecifictransfer,withthecuespaired

haps other variables. More recently Dailey et al. (2016) failed to withthetwoinstrumentalrewardsenhancingboththesameand

detecttransferinasingle-leverparadigmwiththetestconducted differentinstrumentalactionstoasimilardegree,albeitthistrend

undersatiety.Conversely,transferwasobservedundersatietyafter comparedtolowanxietyparticipantswasnotstastisticallysignif-

administation of an antagonist of ghrelin, a peptide related to icant.

appetitesignalling.Aitkenetal.(2016)usingasingle-leverdesign

withdifferentfoodsforthePavlovianandinstrumentaltraining, 4. Neuralsubstrates

alsoconfirmedthatsatietycanabolishgeneraltransfer.

CorbitandBalleine(2003a)showedthattransfercandifferen- 4.1. Amygdalaandnucleusaccumbens

tially affect components of an instrumental chain. In particular,

theyemployedaparadigmwherepressingaleverledtotheappear- Starting from the beginning of this century, lesion studies on

anceofasecondleverandpressingthislatterleverdeliveredthe ratshavebeguntouncovertheneuralbasisofthevarioustrans-

foodreward.Devaluationandtransferdifferentiallyaffectedres- fer effects, reporting that both nucleus accumbens (Hall et al.,

pondingonthesetwolevers.Transferonlyenhancedresponding 2001;Corbitetal.,2001;deBorchgraveetal.,2002)andamygdala

to the proximal lever (i.e., the lever closest to reward delivery) (Blundelletal.,2001;Halletal.,2001;HollandandGallagher,2003)

whereas devaluation depressed responding on the more distal arenecessaryforittotakeplace.Duringthefirststudies:aninitial

lever. disagreementaroseaboutwhichpartsofamygdala(BLAorCeA)

Inarecentexperiment,Gilroyetal.(2014)demonstratedthat andwhichpartsofnucleusaccumbenswereinvolved(NaccCore

specifictransfercanbeaffectedbytestcontext.Theytrainedrats orShell).ItwasreportedthatCeAandCore,butnotBLAorShell

topresstwoleversfortwodifferentfoodsusingtwodifferentcon- wereinvolvedintransfer(Halletal.,2001;HollandandGallagher,

texts. In one group of rats (Group Differential), each lever-food 2003)andalsotheoppositepatternofresults(Blundelletal.,2001;

pairingwastrainedinaspecificcontext,whereasintheothergroup Corbitetal.,2001).ThesedatawerelaterreconciledwhenCorbit

(GroupNon-Differential)bothlever-foodpairingsweretrainedin and Balleine (2005) introduced the full transfer paradigm which

bothcontexts(alternatingcontextsindifferenttrainingsessions). wasabletodistinguishspecifictransferfromgeneraltransfer.Spe-

Pavloviantrainingwasconductedinathirdcontextforbothgroups. cifictransferisabolishedbyBLAandShelllesionswhereasgeneral

Whentestedfortransferineachofthethreecontexts,groupNon- transferisabolishedbyCeAandCorelesions(CorbitandBalleine,

differential always showed specific transfer. In contrast, Group 2005,2011).Muscimol-inducedinactivationofcoreandshellhave

DifferentialfailedtoshowspecifictransferinthePavloviantrain-

ingcontextbut,whentestedintheinstrumentalcontexts,showed

specific transfer when the CS and the context where not associ-
3 Usingachronicunpredictablestressparadigm,composedofdailyexposuresof
atedwiththesamefoodreward.Thus,GroupDifferentialexhibited
1htoonethefollowingstressors:coldwater,vibration,restraint,overcrowding,
lessspecifictransferoverallthanGroupNon-differential.Itcanbe an d a hota irst ream.
836 E.Cartonietal./NeuroscienceandBiobehavioralReviews71(2016)829–848

similareffectsaslesionsofthesestructures(CorbitandBalleine, withtheabsolutesizeofthetransfereffect.Interestingly,byusing

2011).Asafurtherconfirmation,ShiflettandBalleine(2010)used adisconnectionprocedureLeungandBalleine(2015)thendemon-

adisconnectionprocedurebetweenBLAandeithermedialshellor strated that disrupting the VP-m to VTA pathway reduced the

coreandfoundthatonlythedisconnectioninvolvingshellabol- overall rate of responding (similar to a finding by Corbit et al.

ishedspecifictransfer.LingawiandBalleine(2012)foundthatboth (2007) following VTA inactivation) whereas disconnecting the

anteriorandposteriorCeAlesionsabolishedtransferinasingle- VP-mtoMDpathwayremovedthebiasofthespecificpredictive

lever paradigm (therefore presumably general transfer). Human cuesonchoice(i.e.,thespecifictransfereffect).InthislatterVP-m

fMRIstudieshavealsoconfirmedtheinvolvementofamygdalaand to MD pathway disconnection, the CS elevated the performance

ventralstriatumintransfer(Talmietal.,2008;Brayetal.,2008; of both actions; i.e. both the lever delivering the same outcome

Prévostetal.,2012;Mendelsohnetal.,2014). asthestimulusandthedifferentaction,suggestingthattheVTA

Studiesinvestigatingtheroleofglutamatereceptorsarealsoin mediatesthemotivationalcomponentofspecifictransfer(overall

line with the dissociation of neural substrates involved in single rate of lever press performance) whereas the MD mediates the

and two-lever studies and furnish some details on the mecha- cognitivecomponent(theeffectofpredictivelearningonchoice).

nismsworkinginsideamygdalathatmediatetransfer.Meadand

Stephens(2003a,b)usedasingle-leverparadigmtoinvestigatethe 4.2. Molecularprocessesinnucleusaccumbens

effects of AMPA receptor subunits GluR1 and GluR2 deletion in

mice.NeitherdeletionimpairedPavlovianorinstrumentaltrain- A series of studies has attempted to uncover the molecular

ing,howeverGluR1deletionimpairedconditionedreinforcement underpinnings of transfer. Remus and Thiels (2012) investigated

(usageofCS+asreinforcers)leavingtransferintact,wheareasGluR2 ERKkinaseactivationduringtransferincoreandshell.Theyfound

deletion impaired transfer while leaving conditioned reinforce- thatCSpresentationscausedasignificantincreaseinERKactiva-

mentintact.Theauthorssuggestedthatthisdissociationmightbe tioninbothsubregions,withtheeffectbeingslightlymorerobust

explainedbyGluR1andGluR2deletionimpactinglearninginBLA inthecorethantheshell.Anin-depthreviewofmolecularmecha-

andCeArespectively,asthebehavioralconsequencesofthesedele- nismsinvolvingERKintransferandinstrumentalconditioningcan

tionsmimickedtheeffectsoflesionsonthesestructures(Meadand be found in Shiflett and Balleine (2011). Lex and Hauber (2008)

Stephens,2003b).Thishypothesiswassupportedbyalaterfinding examined instead the effects of D1 and D2 receptor antagonism

byJohnsonetal.(2007)whofoundthatGluR1deletionimpaired usingSCH-23390andracloprideinjections,respectively,intoboth

specifictransfer.InparticulartheyfoundthatmicewithoutGluR1 coreandshell.BothcoreandshellD1antagonismabolishedtrans-

expressednonselectivetransferwhentrainedinaspecifictransfer ferinasingle-leverparadigm,withD2antagonismalsoreducing

paradigm: i.e. they increased pressing on both levers to simi- transferbuttoalesserextent.SimilarlyPecin˜aandBerridge(2013)

lar degree when presented with a CS paired with the outcome testedtheabilityofdopaminestimulation(amphetaminemicroin-
del iveredb yone oftheleve rs,wh e are aswild -type mic eshowed jection ) ve rsus (cid:2) -o pioid stim ulation (DA MGO microinje ction) in

specifictransfer.Alltheseresultsarethusconsistentwiththeview either core or shell to amplify transfer. Both amphetamine and

thatsingle-leverstudiesevokegeneraltransfer(mediatedbyCeA) DAMGOaugmentedthetransfereffectinasingle-leverdesignand

wheareas the specific transfer in two-lever studies is mediated didsowheninfusedinbothcoreandmedialshell,excludingonly

byBLA,withoutthenecessarypresenceofCeA.However,subse- a small far-rostral strip of shell. Consistently with this result, in
qu ently Cromba g et al. (2008b ) found t ha t mu tations on GluR1 W ebere tal.(2016 )adm ini sterin ga(cid:2)-opioid antag onist (naltre x-

phosphorylationsitescanabolishsingle-levertransferinmice.This one)reducedtransferinhumans.IncontrasttotheeffectsofLex

isatoddswithpreviousresultsanditmaybeduetoGluR1dele- and Hauber (2008), Laurent et al. (2014) found that SCH-23390,

tiontriggeringdifferentcompensatoryeffectstothoseinducedby but not raclopride, injections specifically in the Nacc shell abol-

alteratingphosphorylationsitesinCrombagetal.(2008b). ished transfer in a specific transfer paradigm. Furthermore, in a

Malvaez et al. (2015) further characterized specific transfer complexseriesofexperiments,Laurentetal.(2014)showedthat

processes in the BLA. They monitored glutamate concentrations delta-opiod receptors (DORs) on cholinergic interneurons (CINs)

duringthetransfertestandfoundthatglutamatetransientswere mediate specific transfer in Nacc shell by altering CIN firing and

time-lockedtoandcorrelatedwithonlytheinstrumentalpressing theireffectonD1-expressingmedium-spinyneurons(MSNs).First,

directedtotheleversharingthesameoutcomeastheCS(i.e.,in theyconfirmedtheinvolvementofshellD1-expressingMSNsby

aspecifictransfertest).Inaddition,localblockadeofAMPArecep- measuringERKphosphorylation(pERK)afterthetransfertestand

tors,butnotNMDAreceptors,abolishedspecifictransfer.Related byinfusionofSCH-23390andraclopride.Resultsshowedenhanced

results were also obtained by George et al. (2009) who showed pERKinD1butnotinD2-expressingMSNsconfirmingtheeffect

thattheselectivemGluR5antagonist2-methyl-6-(phenylethynyl)- of SCH-23390 relative to raclopride. They then provided data to

pyridine (MPEP) reduced transfer in a single-lever study and by support the interaction between processes involving DORs and

Murschall and Hauber (2005) who did not detect any impact on D1Rsusingasymmetricalinfusionsintheshell.Theresultsshowed

single-levertransferusingsystemicAMPA/KAandNMDAblockade. thatratswithinfusionsofSCH-23390inonehemisphereandnal-

LeungandBalleine(2013)characterizedthecircuitdownstream trindole (a DOR antagonist) in the other, exhibited no specific
fromtheN acc shellby investi gatingoneofi tsm ainpro jections,the transfer. 4 In add ition, electr oph ysio logical recording s of CINs in

medialventralpallidum(VP-m).Ratsexposedtoaspecifictransfer Naccshellslicestakenafterthetransfertestconfirmedalterations

testshowedhigherexpressionofthecellularactivitymarkerc-fos intheirfiringpatternswhenexposedtodeltorphin(DORendoge-

in both shell and VP-m compared to controls. Also, both VP-m nousligand)comparedtoCINsinslicestakenfromratsexposedto

inactivation and shell-VP-m disconnection procedures abolished non-contingentCStraining.Itwashypothesizedthattheeffectof

specifictransfer.Inapreviousstudy,lesionsofmediodorsalthala- DORonCINfiringalteredacetylcholinereleaseandthatchanges

mus(MD),whichisfurtherdownstreamasitreceivessVPoutputs, inacetylcholingreleaseaffectedtheactivityofD1neurons.With

werefoundtoimpairspecifictransfer(OstlundandBalleine,2008). regardtothislattereffect,itwasfurtherhypothesizedthatacetyl-

Asaconsequence,LeungandBalleine(2015)examinedthefunc- cholinealtersD1MSNactivitythroughtheactivityofGi-coupled

tionalcontributionsofbothMDandVTA,whichisanothertargetof

VP-m.ResultsshowedthatVP-mneuronsprojectingtoMDwere

moreactive(c-fos)duringthetransfertestthanthoseprojectingto
4 InLaurentetal.(2015)naltrindoleinfusionsintoshellwerealsoshownto
VTA,butitwastheactivationoftheselatterneuronsthatcorrelated
abolish the reversed specific transfer induced by backward conditioning
E.Cartonietal./NeuroscienceandBiobehavioralReviews71(2016)829–848 837

dopamine release in Nacc core. In particular, they showed that

muscarinicreceptorantagonismsuppressesbothtransferandcue-

evoked dopamine release, while nicotinic receptor antagonism

augmentstransferandcue-evokeddopaminerelease.Thediscrep-

ancybetweentheselatterresultsandOstlundetal.’s(2014a)results

inwhichbothscopolamineandmecamylamineimpairedtransfer

maybeeitherduetothetypeofinfusions(systemicvs.local)or

duetotheparadigmused(specifictransfervs.single-lever).

4.3. Dorsalstriatum

CorbitandJanak(2007b)foundthatinactivationofeitherdor-

somedial(DMS)ordorsolateral(DLS)striatuminaspecifictransfer

paradigmhaddifferenteffectsontransferexpression:DLSinacti-

vationabolishedspecifictransferwhereasDMSrenderedtransfer

non-specific,withtheCSincreasingperformanceonboththesame

and different outcome lever. In a later study (Corbit and Janak,

2010),DLS,anteriorDMS(aDMS)orposteriorDMS(pDMS)were

inactivated during Pavlovian and instrumental training: inacti-

vation of DLS and pDMS appeared to impair the acquisition of

stimulus-outcome (S-O) associations whereas aDMS and pDMS

inactivation impaired response-outcome (R-O) associations, as

revealedbyPavlovianandinstrumentaldevaluationtestsrespec-

tively.Inalloftheinactivationgroups–aDMS,pDMS,andDLS–

specifictransferwasimpairedinasubsequenttransfertest.This

wasexpectedbecausespecifictransferrequirestheintegrationof
Fig.3. Modelofthecomplexinteractionsofopioid,cholinergicanddopaminesys-
tem si ntheN ac csh elldurin gspecifictr an sfer,fro mLaurent eta l.(2014). DOR bothS-O(Pavlovian)andR-O(instrumental)associationsandany
recep tor son CINc ellbo diesar eactivat edbyEN K.ENK alterst he firi ngpatte rnof loss of th ese should, quit e na turally, be pre dicted to im pair this
CINs,lead ing toa low erAch rele ase.Lowe rA chre lease leads tol essac tivation of inte gra tivepr ocess.
M4re ceptors on D 1R-ex press ingneu rons.In turn ,thelow erac tiv ityo fM4perm its
Incontrast,Pielocketal.(2011)examinedtheeffectsof6-OHDA
cAMPpathwaysignalling,increasedD1neuronactivityandsoincreasedspecific
inducedDAdepletioninaDMSandpDMSontransferusingasin-
tRreapnrsifnetre edxwpritehsspioernm. Sisesei otnex.t for details of the experiments leading to this model. glelever tra nsferparad ig m.Nei ther deplet ion hadany effect in the

single-lever transfer paradigm. Furthermore, in a second experi-

mentthatusedthespecifictransferparadigm,aDMS6-OHDAagain

M4 receptors recently found to be uniquely expressed on post- hadnoeffectwhereasinthepDMSithadonlyaminor,ifany,effect,

synapticD1MSNs.IncreasedactivityattheM4receptorhasbeen suggestingthatthedopamineinnervationofdorsalstriatumisnot

found to inhibit D1 activity and reduced M4 binding to increase necessaryfortransfer.

D1activity.IfDORactivityinhibitsacetylcholinereleaseandnal-

trindoleblocksthatreductionthenareadyexplanationforLaurent 4.4. Midbrainstructures

etal’sresultscouldbeprovided.Totestthis,Laurentetal.(2014)

soughttoblockspecifictransferusingperipherallyadministered In a single-lever paradigm, Murschall and Hauber (2006)

naltrindoleandthentoreleasethatblockadebyinfusingtheM4 found that ventral tegmental area (VTA) inactivation by

antagonist MT3 into the Nacc shell. Although MT3 had no effect baclofen/muscimol can abolish transfer (supposedly general

onitsown,itsinfusionrescuedspecifictransferafteritwasabol- transfer). However, Corbit et al. (2007) using the same dose

ishedbynatrindole,Tosummarize,Laurentetal.(2014)suggest effectiveinMurschallandHauber(2006)foundinthefulltransfer

that,intheNaccshell,specifictransferismediatedbyacomplex paradigm that VTA inactivation only attenuated specific and

interactioninvolvingopioid,cholinergicanddopaminergicsystems general transfer; essentially reducing but not abolishing these

(seeFig.3).ThebasisforthisinteractionisformedduringPavlo- effects.

vian training, with the increased expression of DORs on CINs at El-AmamyandHolland(2007)usedinsteadadisconnectionpro-

thecellmembrane(Bertran-Gonzalezetal.,2013).DORsarethen ceduretoseparateCeAfromeithersubstantianigraparscompacta

activated during the transfer test by enkephaline (ENK), the lat- (SNpc)orVTA.Inasingleleverparadigm,bothCeA-SNpcandCeA-

terpossiblyreleasedbyD2MSNs,alteringCINactivityandcausing VTA unilateral lesion reduced transfer; a CeA-SNpc controlateral

lower acetycholine release overall and thus a lower level of M4 lesion (disconnection) also reduced transfer, with no additional

binding, resulting in increased D1 neuron activity and increased effecttotheunilaterallesion.Puzzlingly,CeA-VTAdisconnection

specifictransferasaconsequence. was found to restore transfer to control levels. The result was

A confirmation of the involvement of acetylcholine in spe- explained by reference to possible cross-hemispheric inhibitory

cific transfer can also be found in Ostlund et al. (2014a), were connections between the two VTAs and CeA output to the SNpc

the effects of scopolamine (muscarinic receptor antagonist) and (seealsoLeeetal.,2011).

mecamylamine(nicotinicreceptorantagonist)weretested.Inthis

case,bothmuscarinicandnicotinicacetylcholinesystemicantago- 4.4.1. Dopamine

nismimpairedspecifictransfer.Collinsetal.(2016)alsoexamined Other results have, however, pointed to the involvement

theroleofacetylcholineanditsinteractionwithdopamine,how- of dopaminergic areas such as VTA and SNpc in transfer.

ever they focused on Nacc core, using a single-lever paradigm. Dickinsonetal.(2000)showedthatdopamineantagonists(such
Usin gloca linfusion sof scopo lamin eand m ecamylamine combined ˛-flupenth ix ol) reduced transfe rusin gasingle- leverparad igm,a

with fast-scan cyclic voltammetry they showed that acetyl- resultalsoreplicatedinWassumetal.(2011).Later,LexandHauber

choline antagonism can modulate both transfer and cue-evoked (2008)usedD1andD2receptorantagonismincoreandshelland
838 E.Cartonietal./NeuroscienceandBiobehavioralReviews71(2016)829–848

confirmed the ability of dopamine antagonism to reduce trans- moving rats during transfer test sessions. They found that the

ferinasingle-leverparadigm.Conversely,transferispotentiated CS+amplifiedneuronalresponsestobothinstrumentalnosepokes

byindirectdopamineagonistssuchasampethamine(i.e.Wyvell andPavlovianapproachesinallofthesestructures.However,the

and Berridge (2000), see also Section 5). Using fast-scan cyclic amplification of the instrumental responses correlated with the

voltammetry Wassum et al. (2013) monitored dopamine release strengthoftransferonlyinthemPFCandOFCbutnotDS.Morever,

inrealtimeinnucleusaccumbenscoreandshowedthatduringa DSneuronsrepresentedtransferandapproachbehaviorthrough

single-levertransfertestphasicdopaminereleaseevokedbytheCS mostly-segregated populations, whereas in mPFC and OFC they

correlatedwiththeincreaseinleverpressing.Thisresultwasalso wererepresentedinoverlappingpopulationsofneurons.

replicated in Aitken et al. (2016). Ostlund and Maidment (2012) InSaddorisetal.(2011)electrophysiologicalrecordingsofthe

instead examined the effects of flupentixol in a specific transfer singlelevertransfertestsessionwereconductedintheNacccore

paradigm, presenting both levers during the test session. They and shell. Multiple groups were used with one exposed to sep-

foundthatalthoughflupentixolreducedtheresponseinvigoration arate cocaine self-administration training. In all groups, neurons

generatedbytheCS,itdidnotinfluencetheabilityoftheCStobias in both core and shell encoded information about cues, rewards

action selection towards the lever sharing the same outcome as andresponses.Incontrolanimals,coreneuronsweremorelikely

theCS.Inhumans,Weberetal.(2016)administeredamisulpride(a toencodethisinformation,whichcorrelatedwithbehavioralper-

selectiveD2/D3antagonist)andfoundreducedtransferinasingle- formance in the transfer test. However, neurons that expressed

lever paradigm. All these results (and also Soares-Cunha et al. transfer-specific encoding (their lever-press related activity was

(2014),Laurentetal.(2014)describedelsewhereinthisreview) increasedduringCSperiods)correlatedwithtransferperformance

indicatethattheincreaseinleverpressingduringthetransfereffect intheshell,butnotinthecore.Thegroupwithahistoryofcocaine

is mediated by dopamine (and likely by VTA); interestingly, the self-administrationshowedincreasedtransferandincreasedneu-

biastowardsoneleverinachoicesituationmightbedopamine- ral encoding of task events in the shell. Generally, these studies

independentinstead(OstlundandMaidment,2012). areconsistentwithwhathasbeenfoundinlesionandinactivation

studies;however,thepossibilityofmixedtransfereffectsemerg-

4.5. Prefrontalcortex inginsingleactionstudieslikelyaccountsforthebreadthofthe

observedeffects.

BasalgangliastructuressuchastheNaccformaclosedloopwith

prefrontalcortexandso,giventhepreviousresults,theinvolve-

ment of PFC structures in transfer would be expected. Indeed, 5. Interactionwithdrugsofabuse

Ostlund and Balleine (2007) found that post-training lesions of
OFCabo lishe dspecific transfe r5,whi lepr e-traininglesio nshad no The importance of conditioned stimuli in promoting drug-

effect.SubsequentlyBalleineetal.(2011)reportedthatthispost- taking and relapse is widely recognised in addiction research

trainingeffectwaslikelyaproductofsparingaspectsofventralOFC. (Belin et al., 2009). Conditionedstimuli associatedwith drugs of

Complete ventral and lateral OFC lesions produced pre-training abuse can promote drug use through various mechanisms, such

deficitsinspecifictransfer(Balleineetal.,2011).Similarly,Scarlet as conditioned approach, conditioned reinforcement as well as

etal.(2012)foundareductioninspecifictransferusingpre-training Pavlovian-instrumental transfer (Belin et al., 2009). Despite this,

OFC lesions, albeit using a very different paradigm. In Bradfield relatively few studies have been conducted using the transfer

et al. (2015) lesions targeting the medial OFC did not eliminate paradigmwithdrugsofabuse.Asanexample,searchingPubmed

transfer but instead made it non-specific: i.e. the CS no longer with terms referring to transfer (e.g. PIT, transfer) and addiction

enhancedleverpressinginthesamecondition,butalsointhediff (e.g.addiction,cocaine,ethanol,opiate)wefoundonly27articles

condition.DespitebothACCandPLhavingconnectionwithNacc, with experiments investigating the relationship between trans-

Cardinaletal.(2003)foundnoeffectsofACClesionsontransfer feranddrugsofabuse.Thisisasmallnumbercomparedtoother

usingasingle-leverparadigm,whereasCorbitandBalleine(2003b) paradigmsusedinaddictionresearch,e.g.conditionedplacepref-

foundnoeffectsofPLlesionsinaspecifictransferparadigm.More erence, for which it is possible to find hundreds of results. As

recently,Keistleretal.(2015)foundthatbilateralILlesionsabol- expressedbyLeBlancetal.(2012),thismightbedueto“experi-

ishspecifictransfer.TheyalsoemployedanIL-Shelldisconnection encedorperceiveddifficultiesingeneratingthePIT[transfer]effect

procedureandconfirmedthatILmediatesthiseffectviafunctional usingconventionaldrugself-administrationprocedures”.

connectivtywithNaccshell,alsopartofthespecifictransfercir- Studies investigating transfer with drugs of abuse have

cuitry. involveddifferentsubstances,includingamphetamine(Wyvelland

Berridge,2000,2001;Pecin˜aetal.,2006;Pecin˜aandBerridge,2013;

4.6. Neuralcorrelates HallandGulley,2011;Shiflett,2012;Shiflettetal.,2013),cocaine

(Kruzich et al., 2001; Saddoris et al., 2011; LeBlanc et al., 2012,

Finally, we briefly mention two studies that have investi- 2013,2014;Ostlundetal.,2014b),andethanol(Krank,2003;Ripley

gatedtheneuralcorrelatesoftransferusingelectrophysiological etal.,2004;Glasneretal.,2005;CorbitandJanak,2007a;Krank

recordings: Homayoun and Moghaddam (2009) and Saddoris et al., 2008; Milton et al., 2012; Depoy et al., 2014; Corbit et al.,

et al. (2011). Both studies used a single-lever transfer paradigm. 2016)inrodentsandtobacco(Hogarthetal.,2007,2013b,2014,

HomayounandMoghaddam(2009)recordedfrommedial(mPFC), 2015; Hogarth and Chase, 2011, 2012; Hogarth, 2012) and beer

orbitalprefrontalcortex(OFC)anddorsalstriatum(DS)offreely (Martinovicetal.,2014;Garbusowetal.,2014,2016)inhumans.

Thesestudiesshowthattransfercanbeobservedwithdrugsof

abusejustasitisobservedwithnaturalrewards:aCSassociated

withdrugsofabusecanenhanceinstrumentalrespondingdirected
5 WenotethatinOstlundandBalleine(2007)andinOstlundandBalleine(2008)
tothedrugitself,justasaCSassociatedwithfoodenhancesfood
aleltshioonusg ohf rOatFsCa acntuda mllyedkieopdtoprrseasl stihnaglammourse winetrhee resapmoreteadn dtod aifbfocolinshd istpioencsificco tmrapnasrfeedr res pon ding (e.g.K rank ,2 0 03; LeBlancet al.,2 012; Hogarthe tal.,

tothebaseline.Althoughtheeffectwasnotstatisticallysignificantinthoseexperi- 2007).However,therearealsosomepeculiaritiesoftransferwhen
me nts ,itisposs iblethat thes elesio nsd ono tabolishsp ecifictrans fer butre nderit thesub jectsaree xpose dto drug sofab use.Infact,so m estudie shave

“non-specific”.Indeed,inLeungandBalleine(2015),disconnectionofVP-mfromthe
foundastrongertransfereffectinsubjectsthathavebeenexposed
MDproducedasimilar“non-specific”transfereffectsuchthattheCSsignificantly
elev atedrespo n dingon boththesame anddiff lever. todrugsofabusecomparedtocontrols(WyvellandBerridge,2000,