Visual perception. Part 2, Fundamentals of awareness: multi-sensory integration and high-order perception

319 Pages·2006·12.78 MB·English

by  Susana Martinez-Conde

List of Contributors
D. Alais, Department of Physiology and Institute for Biomedical Research, School of Medical Science,
Auditory Research Laboratory, University of Sydney, Sydney, NSW 2006, Australia
E. Aminoff, MGH Martinos Center for Biomedical Imaging, Harvard Medical School, 149 Thirteenth
Street, Charlestown, MA 02129, USA
S. Anstis, Department of Psychology, UCSD, 9500 Gilman Drive, La Jolla, CA 92093-0109, USA
M. Bar, MGH Martinos Center for Biomedical Imaging, Harvard Medical School, 149 Thirteenth Street,
Charlestown, MA 02129, USA
P. Berbel, Instituto de Neurociencias de Alicante UMH-CSIC, Campus de San Juan, Apartado 18, 03550
San Juan de Alicante, Spain
N.P. Bichot, McGovern Institute for Brain Research, Massachusetts Institute of Technology, Bldg. 46-
6121, Cambridge, MA 02139, USA
J.W. Bisley, Center for Neurobiology and Behavior, Columbia University, 1051 Riverside Drive, Unit 87,
New York, NY 10032, USA
S. Blau, Department of Neuroscience, Brown University, Providence, RI 02912, USA
D. Burr, Dipartimento di Psicologia, Universita` degli Studi di Firenze, Via S. Nicolo`, Florence, Italy and
Istituto di Neuroscience del CNR, Via Moruzzi 1, Pisa 56100, Italy
I.C. Cuthill, School of Biological Sciences, University of Bristol, Woodland Road, Bristol BS8 1UG, UK
B. de Gelder, Cognitive and Affective Neurosciences Laboratory, Department of Psychology, Tilburg
University, PO Box 90153, Tilburg, 5000 LE, The Netherlands
V.F. Descalzo, Instituto de Neurosciencias de Alicante, Universidad Miguel Hernandez-CSIC, Apartado
18, 03550 San Juan de Alicante, Spain
R. Desimone, McGovern Institute for Brain Research, Massachusetts Institute of Technology, 77
Massachusetts Avenue, Bldg. 46-3160, Cambridge, MA 02139, USA
M.J. Fenske, MGH Martinos Center for Biomedical Imaging, Harvard Medical School, 149 Thirteenth
Street, Charlestown, MA 02129, USA
E. Gallego, Department of Psychology, University of La Corun˜a, Campus de Elvin˜a, La Corun˜a, 15071,
Spain
R. Gallego, Instituto de Neurosciencias de Alicante UMH-CSIC, Campus de San Juan, Apartado 18,
03550 San Juan de Alicante, Spain
J.V.Garcia-Velasco,InstitutodeNeurocienciasdeAlicanteUMH-CSIC,CampusdeSanJuan,Apartado
18, 03550 San Juan de Alicante, Spain
M.E.Goldberg,Center forNeurobiologyandBehavior,ColumbiaUniversity,1051RiversideDrive,Unit
87, New York, NY 10032, USA
J. Gottlieb, Center for Neurobiology and Behavior, Columbia University, 1051 Riverside Drive, Unit 87,
New York, NY 10032, USA
J.M. Groh, Center for Cognitive Neuroscience, Department of Pschycology and Neuroscience, and
Department of Neurobiology, Duke University, LSRC Rm B203, Durham, NC 27708, USA
N. Gronau, MGH Martinos Center for Biomedical Imaging, Harvard Medical School, 149 Thirteenth
Street, Charlestown, MA 02129, USA
ix
x
X. Huang, Keck Center, Department of Physiology, University of California, San Francisco, CA 94143-
0444, USA
S. Kastner, Department of Psychology, Center for the Study of Brain, Mind and Behavior, Princeton
University, Green Hall, Princeton, NJ 08544, USA
A.Kingstone,DepartmentofPsychology,UniversityofBritishColumbia,2136WestMall,Vancouver,BC
V6T 1Z4, Canada
S.P.MacEvoy,DepartmentofNeurobiology,DukeUniversityMedicalCenter,Durham,NC27710,USA
S.L. Macknik, Departments of Neurosurgery and Neurobiology, Barrow Neurological Institute, 350 W.
Thomas Road, Phoenix, AZ 85013, USA
H.K.M. Meeren, Cognitive and Affective Neurosciences Laboratory, Department of Psychology, Tilburg
University, PO Box 90153, Tilburg, 5000 LE, The Netherlands
L.G.Nowak,CentredeRecherche‘‘CerveauetCognition’’,CNRS-Universite´ PaulSabatier,133Routede
Narbonne, 31062 Toulouse Cedex, France
A. Oliva, Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, 77
Massachusetts Avenue, Building 46-4065, Cambridge, MA 02139, USA
M.A. Paradiso, Department of Neuroscience, Brown University, Providence, RI 02912, USA
C.A. Pa´rraga, Department of Experimental Psychology, University of Bristol, 8 Woodland Road, Bristol
BS8 1TN, UK
K.K. Porter, University of Alabama, School of Medicine, Birmingham, AL 35294, USA
K.D.Powell,LaboratoryofSensorimotorResearch,NationalEyeInstitute,NationalInstitutesofHealth,
Bethesda, MD 20892, USA
R. Righart, Cognitive and Affective Neurosciences Laboratory, Department of Psychology, Tilburg
University, PO Box 90153, Tilburg, 5000 LE, The Netherlands
A.F. Rossi, Department of Psychology, Vanderbilt University, Nashville, TN 37203, USA
N.Sagiv,CentreforCognitionandNeuroimaging,BrunelUniversity,Uxbridge,MiddlesexUB83PH,UK
M.V.Sanchez-Vives,InstitutodeNeurocienciasdeAlicanteUMH-CSIC,CampusdeSanJuan,Apartado
18, 03550 San Juan de Alicante, Spain
K.A.Schneider,DepartmentofPsychology,CenterfortheStudyofBrain,MindandBehavior,Princeton
University, Green Hall, Princeton, NJ 08544, USA
G. Shalev, Department of Neuroscience, Brown University, Providence, RI 02912, USA
S. Soto-Faraco, Hospital Saint Joan de De´u (Edifici Docent), C/Santa Rosa 39-57, Planta 4a, 08950
Esplugues de Llobregat, Barcelona, Spain
C. Spence, Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford
OX1 3UD, UK
L. Spillmann, Neurozentrum, University Hospital, Breisacher Street 64, 79106 Freiburg, Germany
M. Stevens, School of Biological Sciences, University of Bristol, Woodland Road, Bristol BS8 1UG, UK
P. Stoerig, Institute of Experimental Psychology II, Heinrich-Heine-University, Building 23.03,
Universita¨tsstr 1, D-40225 Dusseldorf, Germany
M. Tamietto, Department of Psychology, University of Turin, Turin, Italy
A. Torralba, Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of
Technology, 77 Massachusetts Avenue, 32-D462, Cambridge, MA 02139, USA
T.Troscianko,DepartmentofExperimentalPsychology,UniversityofBristol,8WoodlandRoad,Bristol
BS8 1TN, UK
F. Valle-Incla´n, Department of Psychology, University of La Corun˜a, Campus de Elvin˜a, La Corun˜a
15071, Spain
W.A.C. Van De Riet, Cognitive and Affective Neurosciences Laboratory, Department of Psychology,
Tilburg University, P.O. Box 90153, Tilburg, 5000 LE, The Netherlands
xi
J.vanDenStock,CognitiveandAffectiveNeurosciencesLaboratory,DepartmentofPsychology,Tilburg
University, PO Box 90153, Tilburg, 5000 LE, The Netherlands
J. Ward, Department of Psychology, University College London, 26 Bedford Way, London WC1H 0AP,
UK
K. Wunderlich, Department of Psychology, Center for the Study of Brain, Mind and Behavior, Princeton
University, Green Hall, Princeton, NJ 08544, USA
General Introduction
‘‘VisualPerception’’isatwo-volumeseriesofProgressinBrainResearch,basedonthesymposiapresented
during the 28th Annual Meeting of the European Conference on Visual Perception (ECVP), the premier
transnational conference on visual perception. The conference took place in A Corun˜a, Spain, in August
2005. The Executive Committee members of ECVP 2005 edited this volume, and the symposia speakers
provided the chapters herein.
The general goal of these two volumes is to present the reader with the state-of-the-art in visual per-
ceptionresearch,withaspecialemphasisintheneuralsubstratesofperception.‘‘VisualPerception(Part1)’’
generallyaddressestheinitialstagesofthevisualpathway,andtheperceptualaspectsthancanbeexplained
at early and intermediate levels of visual processing. ‘‘Visual Perception (Part 2)’’ is generally concerned
with higher levels of processing along the visual hierarchy, and the resulting percepts. However, this
separation is not very strict, and several of the chapters encompass both early and high-level processes.
The current volume ‘‘Visual Perception (Part 2) — Fundamentals of Awareness, Multi-Sensory Inte-
gration and High-Order Perception’’ contains 18 chapters, organized into 4 general sections, each ad-
dressing one of the main topics in vision research today: ‘‘The role of context in recognition’’; ‘‘From
perceptive fields to Gestalt. A tribute to Lothar Spillmann’’; ‘‘The neural bases of visual awareness and
attention, and ‘‘Crossmodal interactions in visual perception’’. Each section includes a short introduction
andfourtofiverelatedchapters.Thetopicsaretackledfromavarietyofmethodologicalapproaches,such
assingle-neuronrecordings,fMRIandopticalimaging,psychophysics,eyemovementcharacterizationand
computational modeling. We hope that the contributions enclosed will provide the reader with a valuable
perspective on the current status of vision research, and more importantly, with some insight into future
research directions and the discoveries yet to come.
Manypeoplehelpedtocompilethisvolume.Firstofall,wethankalltheauthorsfortheircontributions
and enthusiasm. We also thank Shannon Bentz, Xoana Troncoso and Jaime Hoffman, at the Barrow
Neurological Institute, for their assistance in obtaining copyright permissions for several of the figures
reprinted here. Moreover, Shannon Bentz transcribed Lothar Spillmann’s lecture, and provided general
administrativehelp.XoanaTroncosowasheroicinherefforttohelpustomeetthesubmissiondeadlineby
collating and packing all the chapters, and preparing the table of contents. We are indebted to Johannes
MenzelandMaureenTwaig,atElsevier,foralltheirencouragementandassistance;ithasbeenwonderful
working with them.
xiii
xiv
Finally, we thank all the supporting organizations that made the ECVP 2005 conference possible:
Ministerio de Educacio´n y Ciencia, International Brain Research Organization, European Office of Aer-
ospaceReseachandDevelopmentoftheUSAF,Consellerı´adeEducacio´n,IndustriaeComercio-Xuntade
Galicia, Elsevier, Pion Ltd., Universidade da Corun˜a, Sociedad Espan˜ola de Neurociencia, SR Research
Ltd.,Consellerı´adeSanidade-XuntadeGalicia,MindScienceFoundation,MuseosCientı´ficosCorun˜eses,
Barrow Neurological Institute, Images from Science Exhibition, Concello de A Corun˜a, Museo
Arqueolo´xico e Histo´rico-Castillo de San Anto´n, Caixanova, Vision Science, Fundacio´n Pedro Barrie´
de la Maza, and Neurobehavioral Systems.
Susana Martinez-Conde
Executive Chair, European Conference on Visual Perception 2005
On behalf of ECVP 2005’s Executive Committee: Stephen Macknik, Luis Martinez,
Jose-Manuel Alonso and Peter Tse
SECTION I
The Role of Context in Recognition
Introduction
Predator or prey? Big or little? City skyline or the of object-based and context-based object recogni-
latest line of kitchen cabinetry? These questions tion.OlivaandTorralbadiscusstheimportanceof
may seem random, but it all depends on the con- context to our ability to recognize the gist of a
text in which they are posed. In fact, they are all scene at a single glance: without considering con-
questions that become coherent and can be an- text, we might not be able to tell apart the city
swered only if the context is known. Well, the skylinefrom the kitchen cabinets. De Gelder, Me-
skyline/cabinetry question may still seem out of eren, Righart, Van den Stock, van de Riet, and
theblue,butitwillallsnapintofocusasyouread Tamietto show how context also plays a critical
thefollowingfourchaptersthatexplorehowvisual role in face recognition. Stevens, Cuthill, Parraga,
objectrecognitioncriticallydependsonthecontext and Troscianko discuss how disruptive coloration
that the objects lie in, and their relevance to the in camouflage serves to conceal objects within the
observer. context of their surroundings.
Fenske,Aminoff,Gronau,andBarstartthesec-
tion with a chapter that discusses how top-down
facilitation modifies the differential contributions Stephen L. Macknik
Martinez-Conde,Macknik,Martinez,Alonso&Tse(Eds.)
ProgressinBrainResearch,Vol.155
ISSN0079-6123
Copyrightr2006ElsevierB.V.Allrightsreserved
CHAPTER 1
Top-down facilitation of visual object recognition:
object-based and context-based contributions
(cid:1)
Mark J. Fenske, Elissa Aminoff, Nurit Gronau and Moshe Bar
MGHMartinosCenterforBiomedicalImaging,HarvardMedicalSchool,149ThirteenthStreet,Charlestown,MA02129,
USA
Abstract: The neural mechanisms subserving visual recognition are traditionally described in terms of
bottom-up analysis, whereby increasingly complex aspects of the visual input are processed along a hi-
erarchical progression of cortical regions. However, the importance of top-down facilitation in successful
recognitionhasbeenemphasizedinrecentmodelsandresearchfindings.Hereweconsiderevidencefortop-
down facilitation of recognition that is triggered by early information about an object, as well as by
contextual associations between an object and other objects with which it typically appears. The object-
basedmechanismisproposedtotriggertop-downfacilitationofvisualrecognitionrapidly,usingapartially
analyzedversionoftheinputimage(i.e.,ablurredimage)thatisprojected fromearlyvisualareasdirectly
to the prefrontal cortex (PFC). This coarse representation activates in the PFC information that is back-
projectedas‘‘initialguesses’’tothetemporalcortexwhereitpresensitizesthemostlikelyinterpretationsof
the input object. In addition to this object-based facilitation, a context-based mechanism is proposed to
trigger top-down facilitation through contextual associations between objects in scenes. These contextual
associations activate predictive information about which objects are likely to appear together, and can
influence the ‘‘initial guesses’’ about an object’s identity. We have shown that contextual associations are
analyzed by a network that includes the parahippocampal cortex and the retrosplenial complex. The
integratedproposaldescribedhereisthatobject-andcontext-basedtop-downinfluences operatetogether,
promoting efficient recognition by framing early information about an object within the constraints pro-
vided by a lifetime of experience with contextual associations.
Keywords: object recognition; top-down; feedback; orbitofrontal cortex; low spatial frequencies; visual
context; parahippocampal cortex; retrosplenial cortex; visual associations; priming
Successful interaction with the visual world de- thattheyrelyontopromotehighlyefficientvisual
pends on the ability of our brains to recognize recognition through top-down processes. The ev-
visual objects quickly and accurately, despite infi- idence we review, from studies by our lab and
nitevariationsintheappearanceofobjectsandthe others, suggests that top-down facilitation of rec-
settings in which they are encountered. How does ognition can be achieved through an object-based
the visual system deal with all of this information mechanism that generates predictions about an
in such a fluent manner? Here we consider the object’s identity through rapidly analyzed, coarse
cortical mechanisms and the type of information information. We also review evidence that top-
down facilitation of recognition can be achieved
(cid:1) through the predictive information provided by
Correspondingauthor.Tel.:[phone];Fax:+1617-
contextualassociationsbetweenanobjectorscene
726-7422;E-mail:[email protected]
DOI:10.1016/S0079-6123(06)55001-0 3
4
and the other objects that are likely to appear
together in a particular setting. In the following
sections, we first consider each of these forms of
top-down enhancement of object recognition sep-
arately, and then consider how these object- and
context-based mechanisms might operate together
to promote highly efficient visual recognition.
An object-based cortical mechanism for triggering
top-down facilitation
Fig.1. Schematicillustrationofacorticalmechanismfortrig-
geringtop-downfacilitationinobjectrecognition(Bar,2003).A
The traditionalview regardingvisual processingis lowspatialfrequency(LSF)representationoftheinputimageis
that an input image is processed in a bottom-up projected rapidly, possibly via the magnocellular dorsal path-
cascade of cortical regions that analyze increas- way,fromearlyvisualcortextotheprefrontalcortex(PFC),in
additiontothesystematicandrelativelyslowerpropagationof
ingly complex information. This view stems from
informationalongtheventralvisualpathway.Thiscoarserep-
the well-defined functional architecture of the vis-
resentationissufficientforactivatingaminimalsetofthemost
ualcortex,whichhasaclearhierarchicalstructure. probableinterpretationsoftheinput,whicharethenintegrated
However, several models propose that both bot- with the relatively slower and detailed bottom-up stream of
tom-up and top-down analyses can occur in the analysis in object-processing regions of the occipito-temporal
cortex(e.g.,fusiformgyrus).
cortex simultaneously (Grossberg, 1980; Kosslyn,
1994; Ullman, 1995; Desimone, 1998; Engel et al.,
2001;Friston,2003;LeeandMumford,2003),and image in the temporal cortex. For example, if an
recent evidence suggests that top-down mecha- imageofanovalblobontopofanarrowlongblob
nisms play a significant role in visual processing were extracted from the image initially (Fig. 1),
(e.g.,Kosslynetal.,1993;Humphreysetal.,1997; thenobjectrepresentationssharingthisglobalpro-
Barcelo´ et al., 2000; Hopfinger et al., 2000; file would be activated (e.g., an umbrella, a tree, a
Miyashita and Hayashi, 2000; Gilbert et al., mushroom, a lamp). Combining these top-down
2001; Pascual-Leone and Walsh, 2001; Mechelli ‘‘initial guesses’’ with the systematic bottom-up
et al., 2004; Ranganath et al., 2004a). Neverthe- analysis could thereby facilitate visual recognition
less,thewayinwhichsuchtop-downprocessingis by substantially limiting the number of object rep-
initiated remains an important outstanding issue. resentations that need to be tested.
The crux of the issue concerns the fact that top-
down facilitation of perception requires high-level
information to be activated before some low-level Orbitofrontal involvement in top-down facilitation
information. of recognition
Recently,Bar(2003)proposedamodelthatspe-
cifically addresses the question of how top-down Theproposalthatobjectrecognitionmightbenefit
facilitation of visual object recognition might be from a cortical ‘‘short-cut’’ of early, cursory in-
triggered.Thegistofthis proposal concernsa cor- formationabouttheinputimageimpliesthatthere
tical (or subcortical) ‘‘short-cut’’ of early informa- should be an additional cortical region that shows
tion through which a partially analyzed version of recognition-related activity before other object-
the input image, comprising the low spatial fre- processingregions.Thecorticalregionsmostoften
quency(LSF)components(i.e.,ablurredimage),is associated with visual object recognition are situ-
projectedrapidlyfromearlyvisualareasdirectlyto ated in the occipito-temporal cortex (Logothetis
the prefrontal cortex (PFC). This coarse represen- etal.,1996;Tanaka,1996).Withinthisregion,the
tation is subsequently used to activate predictions fusiform gyrus and the lateral occipital cortex are
about the most likely interpretations of the input especially crucial for visual recognition in humans
5
(Kosslynetal.,1995;Martinetal.,1996;Baretal., related activity should develop in this site before
2001; Grill-Spector et al., 2001; Malach et al., other object-processing regions ofoccipito-tempo-
2002). The PFC has also been shown to be in- ralcortex.Totestthisprediction,Baretal.(2006)
volved in visual recognition (e.g., Bachevalier and used the same object recognition task as Bar etal.
Mishkin, 1986; Wilson et al., 1993; Parker et al., (2001) while obtaining magnetoencephalography
1998; Freedman et al., 2001), and recent evidence (MEG) recordings, which provide millisecond-
suggeststhattheorbitofrontalcortex(OFC)might resolution measurements of cortical activity. As
be specifically related to top-down visual process- predicted, the contrast between recognized and
ing (Bar et al., 2001, 2006; Bar, 2003). Bar et al. not-recognized trials in this study revealed differ-
(2001), for example, used functional magnetic res- ential activation in the OFC 50ms before it devel-
onance imaging (fMRI) to compare the cortical oped in the object-processing regions of the
activity elicited by trials in which objects were occipito-temporal cortex (Fig. 3). Moreover, a
successfully recognized with that elicited by the time-frequency, trial-by-trial covariance analysis
samepicturesunderidenticalconditionswhenthey of the MEG data demonstrated strong synchrony
were not recognized. As expected, this contrast between occipital visual regions and the OFC at a
showed differential activity in the occipito-tempo- relatively early stage (beginning at approximately
ral regions previously associated with object rec- 80ms after stimulus onset), and a strong sync-
ognition.However,successfulrecognitionwasalso hrony between the OFC and the fusiform gyrus
associated with increased activity in a site within activity at a relatively later stage (130ms after
theOFC(Fig.2).Theconnectionbetweenactivity stimulus onset). Taken together, these results pro-
in the OFC and successful object recognition vide strong converging support for the proposal
makes it a prime candidate for being involved in that this frontal region plays a critical role in top-
top-down facilitation of visual recognition. down facilitation of object recognition.
Rapid recognition-related activity in orbitofrontal Early orbitofrontal activity is triggered by low
cortex spatial frequencies
If the recognition-related region of the OFC iden- The finding that early activity in the OFC is re-
tified in the Bar et al. (2001) study is critical for lated to successful object recognition is consistent
early top-down facilitation, then recognition- with the model of top-down facilitation proposed
byBar(2003).Thismodelpositsthatearlyactivity
intheOFCisrelatedtothedirectprojectionfrom
early visual areas of an LSF representation of the
input image (Bar, 2003). The projection of this
early and rudimentary information to the OFC
would thereby allow it subsequently to sensitize
the representation of the most likely candidate
objects in the occipito-temporal cortex as ‘‘initial
guesses.’’ThepossibilitythatearlyOFCactivityis
driven by LSF information is supported by phys-
iological findings that the magnocellular pathway
Fig. 2. A group averaged statistical activation map showing
recognition-related cortical activity (adapted from Bar et al., conveys LSF information (i.e., a blurred image)
2001)onthelefthemisphere(ventralview)ofabrainthathas early and rapidly (Shapley, 1990; Merigan and
been‘‘inflated’’toexposebothsulci(darkgray)andgyri(light Maunsell, 1993; Bullier and Nowak, 1995; Grab-
gray). Successful object recognition produced greater activity
owska and Nowicka, 1996; Leonova et al., 2003),
than unsuccessful recognition attempts in inferior regions of
and by evidence from anatomical studies showing
boththeoccipito-temporal(e.g.,fusiformgyrus)andthefrontal
(e.g.,orbitofrontalcortex)lobes. bidirectional connections between early visual