Neuroimaging contributions to the understanding of discourse processes

24 Pages·2007·0.21 MB·English

In M. Traxler & M. A. Gernsbacher, (Eds.), BRAIN IMAGING OF DISCOURSE PROCESSING
Handbook of Psycholinguistics (pp. 765-799). Amsterdam: Elsevier.

Neuroimaging contributions to the understanding of discourse processes

Robert A. Mason and Marcel Adam Just
Center for Cognitive Brain Imaging
Department of Psychology
Carnegie Mellon University
Pittsburgh, PA

Neuroimaging research is providing new types of information and insight about the cortical activity
underlying discourse processing. Knowing the intensity and location of the brain activity during
discourse comprehension adds significantly to the information provided by behavioral measures
alone. The combination of neuroimaging data and behaviorally based discourse theories indicate
that discourse processing is underpinned by a system of several distinguishable cortical networks
that are activated for discourse processing, above and beyond the activation evoked by
comprehension at the word and sentence level. Whereas the multiplicities of the processes in
discourse comprehension are sometimes seen as a drawback to behavioral experiments, it is
something of a benefit in neuroimaging research. Controlled neuroimaging experiments, with their
multidimensional measures, can help determine when each of these components contributes to
discourse processing. By making some assumptions about the cortical regions/network that
underlie this processing, we can begin to determine when an area becomes activated and to what
degree it is activated as a function of the discourse properties.

In this chapter, we describe some key action), are relatively new to the discourse processing
neuroimaging studies of discourse processing, and theory (although comprehending the motivations of
observe some systematic patterns of results that apply characters in a story is probably an ancient skill). At the
across the described studies. To foreshadow these same time, there is uncertainty about the reality of these
patterns, we list here five specialized networks we networks and about their anatomical location. Moreover,
believe to be involved in discourse processing. these networks must function in interaction with
somewhat lower level comprehension processes that
Parallel Networks of Discourse operate at the lexical and sentence level (for a review
see: Bookheimer 2002; Gernsbacher & Kaschak, 2003).
1. A coarse semantic processing network Despite these cautions, it seems useful to consider the
(right middle and superior temporal) recent research with some framework in mind, and we
2. A coherence monitoring network suggest that these five networks/discourse functions
(bilateral dorsolateral prefrontal) provide an initial attempt at such a framework. As we
3. A text integration network (left inferior proceed through the chapter we will elaborate on the
frontal-left anterior temporal) characteristics of these networks as they are illuminated
4. A network for interpreting a protagonist’s by the various studies. At the conclusion of the chapter
or agent’s perspective (bilateral medial we will expand on a theoretical framework based on
frontal/posterior right temporal/parietal) these networks.
5. A spatial imagery network (left dominant, Although the focus of this paper is decidedly not on
bilateral intraparietal sulcus) “where” in the brain discourse processing occurs, a brief
overview of which areas of the brain play a role in
This list should be treated with caution and with discourse comprehension is useful. In almost every
excitement. It is exciting that some of the discourse processing task, the traditional left hemisphere
components of discourse processing revealed by language network activates in a contrast with a fixation
neuroimaging research, like protagonist perspective baseline condition. This traditional left hemisphere
monitoring (here we use protagonist as a shorthand language network includes the left hemisphere inferior
for any agent in the story capable of intentional frontal gyrus, the superior and middle temporal gyrus,
In M. Traxler & M. A. Gernsbacher, (Eds.), BRAIN IMAGING OF DISCOURSE PROCESSING
Handbook of Psycholinguistics (pp. 765-799). Amsterdam: Elsevier.

potions of the inferior temporal gyrus and the angular lower levels of language processing. Both
gyrus. In addition to this basic language processing circumstances would make the deficit less apparent to
network, we expected additional discourse networks clinicians. In recent years, however, more sensitive
to activate during discourse processing. An overview neuropsychological investigations have detected
of the Parallel Networks of Discourse and a rough impaired discourse functioning at several levels and have
sketch of the cortical regions in which they are found such impairments to be correlated with right
localized are shown in Figure 1. This schematic hemisphere damage. Unlike the situation with Broca’s
representation is shown via surface projection on a and Wernicke’s patients, consistent focal lesions have
rendered brain. It is not intended to depict an not been found in these discourse aphasics. Various
exhaustive account of discourse processing networks patients have lesions in right hemisphere homologues of
but instead highlight some key areas. It is likely that Broca’s area (e.g., the inferior frontal gyrus), Wernicke’s
the networks are differentially engaged in the area (e.g., posterior-superior and middle temporal gyrus)
research presented here and the localization of the as well as the right hemisphere angular gyrus,
peak activation for a specific task could be expected dorsolateral prefrontal cortex and the medial frontal
to vary somewhat within an anatomical region. gyrus. Additionally, several other populations, such as
Discourse theories become critical in developing individuals with autism (Dennis, 2001) and Alzheimer’s
this understanding of the cortical discourse patients (Papagno, 2001), with non-focal lesions or no
processing network. In addition, neuroimaging lesions at all have difficulty with discourse processing.
research has led to the development of several new Thus, unlike the case for lower-level language
discourse theories such as the coarse coding theory of processing research, neuropsychological studies of
right hemisphere processing (Beeman, 1998), the discourse processing in brain-damaged and other special
dynamic recruitment of networks in response to text populations did not provide much information about
constraints (Ferstl, Rinck & von Cramon, 2005; brain function other than a generalized notion that the
Mason and Just, 2004; Xu et al., 2005), a Theory of right hemisphere was somehow involved in discourse
Mind system responsible for awareness of different processing.
perspectives (Gallagher and Frith, 2003), and the It was not until the late 1980’s that researchers
spillover of processing to other differential began using brain imaging as a technique to investigate
specialized networks in response to capacity cognitive process. At the time neuroimaging was
utilization (Just et al., 1996). The cortical activation primarily based on Positron Emission Tomography
and these new theories are based on, or at least (PET). One of the earliest neuroimaging investigations
consistent with, traditional discourse theories such as of a “discourse variable” was a PET study of metaphor
Kintsch’s Construction-Integration framework comprehension conducted by Bottini et al., (1994).
(1988), Gernsbacher’s Structure Building Framework Subjects listened to either literal or metaphorical
(1990), Myers and O’Brien’s Resonance model sentences, including such examples as:
(1998), Giora’s Graded Salience Hypothesis (1997) plausible: “The investors were squirrels collecting
and van den Broek’s Landscape architecture (1996) nuts”
as well as others. implausible: “The investors were trams.”
The Beginnings of Brain Imaging in Discourse The sentences containing metaphors elicited more
Comprehension activation in the right hemisphere, particularly the right
Dating back to Broca's and Wernicke’s findings inferior frontal gyrus and right posterior temporal cortex.
on brain-damaged patients with specific language The increased involvement of the right hemisphere may
deficits in the late 1800’s, psychologists have had have been the result of an inference process that
some idea of the brain’s functioning as a language combined world knowledge with the contents of the
processing mechanism. There is however, no sentence to resolve the ambiguity. This early finding of
similarly well-known case of a patient with a deficit right hemisphere involvement in metaphor
in discourse processing abilities. One reason for this comprehension, combined with neuropsychological and
lack may be the difficulty in defining what is meant visual field presentation data, contributed to the
by a discourse processing deficit. Another reason for hypothesis that the right hemisphere is critically
the lack of reports on patients with a discourse- involved in metaphor comprehension and in other facets
processing deficit is that many patients with such of discourse processing. This view is still widely held
deficits are either still able to function well in today although much refined. The right hemisphere
everyday life or they also have severe deficits at coarse-coding hypothesis for example (described more
2
In M. Traxler & M. A. Gernsbacher, (Eds.), BRAIN IMAGING OF DISCOURSE PROCESSING
Handbook of Psycholinguistics (pp. 765-799). Amsterdam: Elsevier.

Figure 1. A Schematic representation of the Parallel Networks of Discourse. Shaded regions represent surface
rendered anatomical regions as described in the text. A rough localization within anatomical regions are represented
by colored ellipses.

3
In M. Traxler & M. A. Gernsbacher, (Eds.), BRAIN IMAGING OF DISCOURSE PROCESSING
Handbook of Psycholinguistics (pp. 765-799). Amsterdam: Elsevier.

fully in the lateralization section) developed by appropriate experimental paradigms within the
Beeman and colleagues (Beeman, 1993; Beeman, constraints of brain scanning methodology. A brief
1998; Beeman et al., 1994; Brownell et al., 1986) is a consideration of imaging methodology will make this
broader theory which encompasses the right point more clear. Imaging of brain activity using PET
hemisphere discourse view. (Positron Emission Tomography) requires uptake of a
Nichelli, et al. (1995) was among the first to tracer substance into the blood stream and into the brain
visually present passages consisting of multiple during task performance. The relative insensitivity to
sentences (Aesop’s Fables) passages in a the tracer requires long sampling intervals, such that
neuroimaging study. (Another early auditory study brain activity can be measured only over many tens of
was reported by Mazoyer et al., 1993.) Participants in seconds. As a result, PET studies have to utilize a
Nichelli’s PET study were asked to monitor either “blocked” or “epoch” design in which many trials or
semantic details, syntactic details or the moral of the items of the same type (i.e. constituting the same
fable. Nichelli et al. concluded that the extra experimental condition) are presented in a sequence or
activation observed in the right hemisphere during “block,” and brain activity is measured during the
moral monitoring and not in the other conditions was processing of the entire block of stimuli. The activity is
due to the drawing of an inference about the passage. then contrasted with other blocks (experimental
Although there may have been some alternative conditions) in which a different set of processes occur.
accounts offered for this finding, this early text The activity in the experimental conditions is typically
comprehension study advanced the view that there estimated with respect to a baseline task, usually a
was something special about the role of the right simpler task that is believed to be common to the two or
hemisphere in discourse processing. In the decade more experimental conditions. The resulting image is
since these early discourse studies, considerable then “subtracted” from the experimental blocks so that
advancement has been made in both methods and the non-baseline processing can be isolated. Even with
theory. some shortcomings of the subtraction paradigm
There were several possible reasons for the (Newman et al., 2001) this methodology was very
scarcity of neuroimaging research at the discourse fruitful for the early days of imaging. At the very least it
processing level, some of which remain as problems: served to illuminate the extent of various networks
(1) neuroimaging research into language processing engaged in many cognitive processes. But the nature of
at any level is fairly new, (2) discourse processing is PET imaging led to some concessions in experimental
a broad field without a tradition for concern about design. Because PET is less sensitive and requires a
neural mechanisms, (3) many of the key research radioactive tracer, most mainstream functional
questions that have been asked in the discourse neuroimaging has turned to functional Magnetic
processing area are not easily addressed with Resonance Imaging, or fMRI.
neuroimaging (due to the sluggish nature of the The underlying assumption of fMRI is that it is a
hemodynamic response and the low temporal measure of neuronal activity, which should increase in
sampling rate), (4) several of the frequently-used some area when a cognitive process makes use of
paradigms used to investigate discourse processing substrate in that area. This increase in neuronal activity
are not easily adaptable to the scanner environment results in an increase in local blood flow and volume.
(e.g., talk aloud protocols, naming), and (5) The oxygen content is then elevated in cortical areas
neuropsychological research on discourse processing which are being used. This will result in an increase in
deficits is limited. Recent advancements in the MR signal, which is affected by the ratio of
technology and knowledge about cortical function deoxygenated hemoglobin to oxygenated hemoglobin
have alleviated many of these constraining factors. (Cohen & Bookheimer 1994).
The release of some of these constraints has opened A key advantage of the fMRI methodology is that a
many exciting new possibilities for the investigation significantly smaller temporal window can be used in the
of the neural substrate underlying discourse measurement of brain activity than had been possible
processing. with PET. In some cases, the cognitive processing that
we would like to measure is very short in duration and,
Specialized imaging paradigms for discourse particularly with language processing, it occurs on the
processing research order of tens of milliseconds. There is obvious difficulty
Perhaps the largest factor constraining discourse when trying to measure a rapid cognitive process with a
imaging research is the challenge of developing very slow measurement such as PET. In fact, PET
4
In M. Traxler & M. A. Gernsbacher, (Eds.), BRAIN IMAGING OF DISCOURSE PROCESSING
Handbook of Psycholinguistics (pp. 765-799). Amsterdam: Elsevier.

doesn’t really allow a consideration of the moment blank interval appears (about 7 to 14 seconds, hence the
by moment changes in activation. It results in a term slow-paced) between the items, sufficient to allow
single average image from a large temporal window. the hemodynamic response to return to a baseline level
fMRI can be used to acquire an image in many (Mason, et al., 2003, Mason & Just, 2004) so that the
consecutive temporal windows. The exact temporal activations associated with individual sentences are
window (termed TR, for “time for repetition”) is separable. The development of the slow-paced and fast-
related to the strength of the magnet, the amount of paced event-related experimental designs for fMRI
cortex being imaged, the criteria for signal to noise allowed for imaging researchers to both randomize
and, critically, the amount of time in which it takes presentation of items as well as isolate specific cognitive
the protons in the imaged substance to return to processing to single sentences as opposed to examining
baseline after the introduction of a radio frequency gross levels of processing differences.
pulse that causes them to tilt. Even though the The Mason and Just study (2004) provides an
temporal window in imaging varies across example of how the event-related approach can be used
experiments, the range is roughly between 1 second to both randomize items as well as isolate specific types
(e.g., Just et al., 2004; Mason, et al., 2003) and 3 of cognitive processing. The study examined causal
seconds (e.g., Martin and Weisberg, 2003; Robertson, inferencing in the comprehension of two-sentence
et al. 2000; St. George et al., 1999). passages, drawn from previous stimulus materials
Although early fMRI studies of discourse (Keenan, Baillet, & Brown, 1984; Myers, Shinjo, &
comprehension used a block design, they took Duffy, 1987). The critical sentence in each passage was
advantage of the more rapid temporal window and the second one, and the experimental manipulation was
the ability to compare conditions without subtracting the degree of causal relatedness to the preceding
out lower levels of cognitive processing. For sentence. The second sentence was followed by a
example, Robertson et al. (2000) presented readers fixation point so that the activation could return to
with blocks of sentences which either contained an baseline before the next passage. Using this technique, it
indefinite article or a definite article. They found was possible to identify the time interval of cortical
greater right hemisphere activation for the lists of activation that corresponded to the processing of the
sentences that used definite articles rather than critical sentence. The processing was expected to differ
indefinite articles. The definite article sentences among the experimental conditions, and this expectation
were assumed to lead to more coherent discourse was confirmed, as described later in the chapter.
than the indefinite article sentences. They concluded Even the relatively rapid 1 second temporal
that the additional right hemisphere activation sampling rate used by Mason et al. (2003) seems slow
reflected processes used to anaphorically relate the relative to the time measures and effect sizes that are
nouns in a text. Ferstl and von Cramon (2001) typical of behavioral studies of discourse
compared pairs of sentences that were coherent or comprehension. For example, the reading time
incoherent as well as cohesive or incohesive. The difference between reading a sentence with an indefinite
coherence manipulation resulted in activation in the article and the definite article may be only a matter of
left frontal gyrus. The cohesion manipulation tens of milliseconds. Nevertheless, even small
involved adding lexical connectives to the pairs of differences in processing time can produce measurable
sentences to make them easier to understand as a differences in the brain activity that has been aggregated
single unit. Unlike Robertson et al. they did not find over one or more seconds, as several studies (e.g.,
any additional right hemisphere activation. In both Robertson et al., 2000) have shown. Although averaging
the Robertson et al. and Ferstl and von Cramon over 1 or more sec can be viewed as a weakness of
studies, the researcher utilized sentences or pairs of fMRI, there is a sense in which it conveys an advantage,
sentences in which a similar type of discourse level because it is often difficult to isolate the cognitive
processing could be assumed to occur in all items of process of interest to a specific temporal window during
one type and not the items of another type. the reading of a passage. For example, McKoon and
Recently the development of slow-paced event- Ratcliff (1986, 1989, & 1992) have shown that
related fMRI has enabled the randomization of items predictive inferences could be drawn either on the first
within experiments (Buckner, 1996), and more sentence at which it was possible to draw the inference
importantly, has enabled the measurement of brain or in the subsequent sentence. Consider how Robertson,
activity during the comprehension of individual et al., (2000), Ferstl and von Cramon (2001), and Mason
sentences. In a slow-paced event-related fMRI, a and Just (2004) dealt with this issue. In all three cases,
5
In M. Traxler & M. A. Gernsbacher, (Eds.), BRAIN IMAGING OF DISCOURSE PROCESSING
Handbook of Psycholinguistics (pp. 765-799). Amsterdam: Elsevier.

they used well constrained materials of either a single passages for the majority of subjects. For example, if the
sentence or sentence pairs to maximize the generation of a bridging inference is being examined, the
probability that the effect would occur in a specific inference must be generated at a similar point within the
temporal window. text across stimuli and across readers to be measured.
In modern discourse research, the trend has been The limitation on number of passages and subjects
towards much longer and more naturalistic passages. constrains the ability to average over a large set of
This creates an added methodological burden. Every passages.
second of an imaging session is precious, because a Even with the constraints on experimental design,
participant will lie still for only so long. Although the imaging still has the benefit of examining how the
background and introductory sections of these longer, network as a whole functions. This is true even in those
naturalistic passages are critical, the expectation is cases in which a process, such as inferencing, might not
that cognitive processing during this context-setting occur at a specific temporal point. Thus, it is possible to
period is similar across conditions. The data acquired see the cognitive workload required for generating an
during this temporal window would either have to be inference with imaging even if it cannot be determined
discarded or treated as an additional factor in the behaviorally whether or when an inference was drawn.
design. Similarly, if the critical sentence does not fMRI can thus be sensitive to processes that have only
include the expected cortical processing (perhaps the small effects on behavioral measures, while at the same
processing is delayed to a post-target sentence) then time capturing the qualitative variation and parallel
the likelihood of finding cortical evidence of a nature of the processes underlying discourse
cognitive process is greatly reduced given that often comprehension to which response times are insensitive.
there are as few as ten to twenty items per condition
in neuroimaging experiments. New Perspectives on Text Integration
Perhaps the largest advantage that imaging Text attributes at the discourse level enter into
research has over behavioral research is that it combinations with other information to allow a reader to
provides a fairly direct measure of the processing weave individual sentences into an integrated narrative
activity in each of the neural networks underpinning structure. The resulting conceptual structure incorporates
discourse comprehension. Measuring cognitive pragmatic information and connects the text with the
workload in most purely behavioral studies is reader’s world knowledge. This discourse process
difficult, frequently necessitating the use of extends beyond strictly linguistic information. For
secondary tasks as means of measuring processing example, discourse comprehension requires that the
load. fMRI studies allow the collection of button reader generate inferences and extract meaning that is
pressing data and response times, but many other not explicitly encoded in the text. Readers must make
data collection methods are problematic. inferences in order to integrate sentences in a coherent
Vocalization responses such as naming times and fashion, filling in what is absent from or ambiguous in
talk-aloud protocols are more difficult to acquire. the text. Several researchers have tried to describe the
Although, noise-canceling technologies have properties of the internal representation of discourse.
facilitated the extraction of voice responses in the One of the most influential was the situation model as
noisy scanning environment, the head motion developed by van Dijk and Kintsch (1983). The situation
induced by jaw movements can render data unusable. model arises from linguistic processing of the text itself
The normal scanning procedure also makes it (lexical access, syntactic processing, and construction of
difficult to collect discourse recall data from a a propositional based micro-structure) and an interaction
scanning session. As with voice responses, these with non-linguistic cognitive processes. The situation
problems can be overcome with some effort, but the model is a result of this interaction created by
problems have not been systematically solved so far. connecting the text with knowledge derived from the
In summary, brain imaging methodology reader’s long-term memory, and involves additional
imposes some constraints on discourse processing demands upon attention (e.g., the ability to shift points
experimental design. Passages must be designed of view and parse sequences of events), working
carefully to maximize the chance of finding how a memory (the ability to retain longer term, anaphoric
complex cortical network functions in discourse references), and the contribution of visual imagery,
comprehension. Specific process must be temporally empathy, and emotional knowledge.
localized to a specific point in the text. Moreover, the Some early attempts at imaging investigation of
same process must occur across the majority of discourse processing were designed to determine the
6
In M. Traxler & M. A. Gernsbacher, (Eds.), BRAIN IMAGING OF DISCOURSE PROCESSING
Handbook of Psycholinguistics (pp. 765-799). Amsterdam: Elsevier.

neural underpinnings of the construction of a First, late in the evening, chain your dog to his
situation model. An fMRI study conducted by St. doghouse, build a small bonfire and let it burn
George and colleagues (1999) presented syntactically overnight. (Support 2nd - #1)
well-structured paragraphs which were
They are insatiably attracted to heat, become
uninterpretable by virtue of never specifying the
enamored of the fire, leave your dog, jump into
referent of the text. The paragraphs were similar to
the flames, and die. (Support 2nd- #2)
those previously used by Bransford and Johnson
(1972.) These paragraphs were either preceded or Fleas will be eliminated from your dog or his
not preceded by a title that disclosed the referent. For doghouse with the use of a bonfire. True or
example, these paragraphs contained sentences such false?
as “Typically, success requires that you start with
your left leg, and make sure that it is securely in 'Topic last' condition
place. Then swing your body high into the air.” First, late in the evening, chain your dog to his
Without knowledge of the referent, it is difficult if doghouse, build a small bonfire and let it burn
not impossible to understand the passage. But with overnight. (Support 1st - #1)
foreknowledge provided by a title (“Riding a horse”)
They are insatiably attracted to heat, become
all of the sentences become interpretable.
enamored of the fire, leave your dog, jump into
The fMRI results revealed that the left
the flames, and die. (Support 1st - #2)
hemisphere, as a whole, exhibited no effect of
whether the paragraph was presented along with the This is a totally guaranteed method to
title, while the right hemisphere revealed completely eliminate a flea infestation on your
significantly greater involvement during the dog or around his doghouse. (Topic last)
presentation of the untitled paragraphs. More
Fleas will be eliminated from your dog or his
specifically, the left middle and superior temporal
doghouse with the use of a bonfire. True or
sulci became more active during the processing of the
false?
titled paragraphs, and conversely, the right middle

and superior temporal sulci became more active
This paradigm made it possible to measure the brain
during the processing of the untitled paragraphs.
activation associated with the comprehension of each of
These results support the idea that the right
the three sentences in each paragraph separately. The
hemisphere is concerned with the mapping of
results revealed differential effects in the two
information into a text representation, as discussed
hemispheres. The right temporal cortex revealed greater
below. In addition, these results suggest that the
involvement during the processing of topic sentences,
processing roles of the two hemispheres are, in fact,
regardless of their location within the paragraph. In
distinguishable.
contrast, the left temporal cortex was sensitive to the
Tomitch et al. (2004) attempted to investigate
location of the topic sentence.
the differential processing of the left and right
Tomitch et al. cited Gernsbacher’s (1990) Structure
hemisphere during text integration using fMRI. They
Building Framework (SBF) to account for the processing
manipulated the serial position of the topic sentence
underlying the cortical activation. According to SBF,
in short, three-sentence paragraphs. The topic
discourse comprehension builds cohesive mental
sentence contained a unifying super-ordinate theme,
representations using three general processes: laying the
while the supporting sentences in the paragraph
foundation, mapping incoming information to previous
instantiated that thematic concept. The serial position
information, and initiating a new substructure if the
manipulation consisted of varying the position of the
incoming information is not adequately coherent with
topic sentence in the paragraph, placing it either in
previous information. SBF states that the first step in
the first position - topic 1st or in the third position-
building a mental representation of the text is to lay a
topic last, as shown below.
foundation to which subsequent information presented in

the text can be attached. Presumably this first stage must
'Topic first' condition
occur across all passages regardless of the order of the
This is a totally guaranteed method to
topic sentence; the consistent left temporal activation on
completely eliminate a flea infestation on
the first sentence suggests that the left temporal region is
your dog or around his doghouse. (Topic
involved in laying the foundation of the text
first)
representation.
7
In M. Traxler & M. A. Gernsbacher, (Eds.), BRAIN IMAGING OF DISCOURSE PROCESSING
Handbook of Psycholinguistics (pp. 765-799). Amsterdam: Elsevier.

A second prediction SBF makes is based on encounter inconsistent emotional or temporal
more involvement of the “shifting” process in information in a passage. They suggested the contrast of
paragraphs whose topic sentence is in the final consistent and inconsistent information that should be
position. When the topic sentence in the final coded at the situation model level would allow them to
position, increased shifting is, therefore, expected to examine cortical networks specialized for situational
result in higher activation levels. This response was level text representation. They concluded that activation
also observed in the left temporal region. in the frontal cortex indicated a specialization for
The right temporal region was sensitive to the building and maintaining a situation model
presence of a topic sentence but not to its location. representation. The specific region within the
This suggests that the right temporal region 1) is frontomedial cortex varied as a function of the type of
sensitive to whether a sentence is a potential information and whether it was involved in noticing or
statement of the topic and 2) performs additional resolving the inconsistency. First, the ventral portion of
processing on the potential topic sentence. The right the medial frontal cortex was involved in detecting
hemisphere may then be responsible for using the inconsistent emotional information. In contrast, the
information forwarded by the left to “fill in” or map dorsomedial frontal cortex was active in the processing
information onto the text representation built by the of emotional consistencies suggesting that a protagonist
left and to connect it with the participant’s world interpreter network was engaged during the attempted
knowledge. This right temporal activation “mapping” resolution of the inconsistent information. The
activation was also found by Robertson et al. (2000) chronological inconsistency activation was located much
for both indefinite and definite articles; in contrast, more anteriorly in the orbital portion of IFG and the
the less coherent, indefinite article texts also resulted frontopolar region. Interestingly, the right anterior
in additional right frontal activation than the definite temporal lobe was more active during the processing of
article texts. inconsistent texts regardless of the text type, suggesting
Several other cortical regions have been found to that as text processing became more difficult, the
play a role in tasks that require structure building. specialized text integration network spilled over into the
Partiot et al. (1996) investigated script processing and right hemisphere.
found bilateral precuneus/posterior cingulate regions To summarize, while there is still a significant
to be activated, along with bilateral medial parietal amount of research to be done to clarify the
cortex, during processing of event sequences. contributions of the left and right hemisphere in text
Maguire et al. (1999), using Bransford and Johnson integration, neuroimaging studies such as those outlined
(1972) passages with or without titles, also found here are making good progress. Although it must be true
these same areas to be active in linking textual that the two hemispheres work together to accomplish
information with subjects’ prior knowledge. These such a complex function as comprehension, it does
operations must be central to construction of a appear as though they are involved in different aspects of
situation model, connecting the narrative text with discourse processing. It seems as though the
knowledge about the real world. identification of the main idea and the building of the
Narrative-specific activations have also been text representation is separable from the mapping of that
found in the temporo-parieto-occipital junction, information onto the discourse structure with both
angular gyrus, and superior temporal sulcus. A hemispheres working in an interactive manner in order
prevailing view has been that the angular gyrus plays to construct a coherent representation of the text.
a key role in grapheme to phoneme translation. But Furthermore, the type of the information within the text
this region is in fact multifunctional, and has been plays a role in which areas are responsible for building
implicated in a variety of cognitive processes— and maintaining a representation of the text.
attention, semantic association, problem solving, and Propositional level information may be processed by left
mental imagery (Cabeza and Nyberg, 2000) that are frontal and left temporal areas as long as resources are
likely to be engaged in the narrative context. For available. Situational model information results in an
example, the angular gyrus is activated when subjects engagement of the medial frontal cortex. Specific areas
visualize a scene derived from a written text (Mellet within the cortex may be dependent upon the nature of
et al., 2002)—precisely the sort of mental model the input, for example, emotional information related to
representation required during narrative processing. protagonist interpreter should activate more dorsal
Recently, Ferstl, Rinck and von Cramon (2005) frontal regions.
examined passages in which the reader might
8
In M. Traxler & M. A. Gernsbacher, (Eds.), BRAIN IMAGING OF DISCOURSE PROCESSING
Handbook of Psycholinguistics (pp. 765-799). Amsterdam: Elsevier.

New Perspectives on Inference Processing primed in both left visual field-Right Hemisphere and
Often, the links between events in a story are not right visual field-Left Hemisphere at a later time. This
explicitly expressed and the reader must connect finding suggests that the right hemisphere is particularly
them by generating linking inferences and integrating involved in inference processing.
them with the presented information. Almost every There have been few neuroimaging studies of
text requires a reader to draw on a rich store of shared inference processing. One question that has been
knowledge about the world. There is a strong debated is whether logic-based inferencing relies on the
relation between inference generation and text same processes as text-based inferencing. In an fMRI
integration. Inference generation is often necessary study, Caplan and Dapretto (2001) directly addressed
in building accurate text representations and this issue by comparing the generation of logic-related
conversely, an accurate text representation is often inferences versus text-based inferences. There were two
necessary for inference generation. types of sentence pairs as shown below
Inferences may be drawn to fill in missing
information, resolve discrepancies or to predict yet Text based: “Do you believe in angels?”
unmentioned events or facts. There are several types “Yes, I have my own special angel”
of inferences, including coherence, predictive,
elaborative, and causal. The classification of various Logic-related: “Do you like having fun?”
types of inferences has been a continuing topic in “Yes, because it makes me happy”
discourse research and there have been several
excellent attempts to resolve this classification While the logic condition produced greater
problem (e.g. Singer, 1994; van den Broek, 1994). activation within the left language areas, the text-based
A successful inference generally occurs as a condition revealed more activation in the right
result of generating a possible inference and then hemisphere. This study is significant in that it suggests
integrating that inference into the internal that at the neural level, and consequently at the process
representation of the text. The Construction- level, there are significant differences between logic-
Integration (CI) model of text comprehension based and text-based inferencing. Also, the results
(Kintsch, 1988) is consistent with this general converge with the neuropsychological findings showing
description of inferencing. According to the CI that the right hemisphere is intimately involved in text-
model, there is a first process in which the many related inference processing.
possible inferences are liberally generated (inference Mason and Just (2004) reported an fMRI
construction), followed by a second process of experiment that was designed to examine the cortical
integrating only those inferences that have a high areas that are involved in making causal inferences
degree of connection with the reader's representation during reading. Participants read sentence pairs that
of the preceding text (integration). A successful varied in terms of their causal relatedness. For example,
integration of an inference results in a representation an “outcome” sentence (e.g., The next day his body was
of the text that involves both the specific propositions covered with bruises.) was preceded by one of three
contained in the text and those propositions that were sentences (equated for overall length and number of
generated by the reader to connect information in the propositions) that described an antecedent condition:
text.
Patients with lesions to the right hemisphere Highly Related: Joey's big brother punched him
generally have trouble drawing inferences in order to again and again.
integrate sentences and maintain coherence (Beeman,
Moderately Related: Joey's crazy mother became
1993; Brownell et al., 1986). These patients are less
furiously angry with him.
likely to mistakenly false alarm to inferences in a text
recognition task, presumably because they never Distantly Related: Joey went to a neighbor's house
generated the inferences in the first place (Grafman et to play.
al., 1987). Such patients also make elaborative
inferences more easily than bridging inferences In previous behavioral studies using similar
(Tompkins and Mateer, 1985). Furthermore, Beeman materials, Keenan, Baillet, and Brown (1984) and
and colleagues (1994) have shown that when probes Myers, Shinjo, and Duffy (1987) found that reading
are inference-related, they are primed in the left times on the sentences increased as the degree of
visual field-Right Hemisphere immediately, and are relatedness between the sentences decreased; that is, the
9
In M. Traxler & M. A. Gernsbacher, (Eds.), BRAIN IMAGING OF DISCOURSE PROCESSING
Handbook of Psycholinguistics (pp. 765-799). Amsterdam: Elsevier.

reading times increased from the highly- to suggest that, as proposed by Just et al. (1996), there is an
moderately- to distantly-related conditions. inherent limit on how much cognitive processing can be
Paradoxically, however, the participants’ memory for done per unit of time in left hemisphere language areas.
the two-sentence passages (as measured using a In the context of text processing, this limit means that to
variety of recall and recognition tests) followed an the degree that working memory resources are being
inverted U-shaped function; that is, the sentences in used to process the text and generate inferences, those
the moderately-related condition were remembered resources will not be available for integrating those
better than those in both the highly- and distantly- inferences into long-term memory. Reichle and Mason
related conditions. Thus, the participants’ memory propose a computational model to demonstrate that these
for the sentences does not seem to be a simple resources are exceeded only in the moderately related
monotonic function of either their reading times or condition, such that the remaining processing load that
the degree of causal relatedness between the two cannot be accommodated in the left hemisphere spills
sentences being read. over into the right hemisphere.
In the fMRI study of causal relatedness, Mason Research is only now beginning to map out the
and Just (2004) found three main foci of fMRI- cortical network associated with drawing inferences in
measured cortical activation among the language reading. Evidence to date suggests that the right
areas. In the left hemisphere language areas, the hemisphere plays a key role in such a process (Beeman,
activation volume did not vary across the three et al., 1994; Mason & Just 2004). At this point, at least
relatedness conditions. In contrast, the activation three plausible theories have been developed to
volume in the bilateral dorsolateral prefrontal cortices illuminate the right hemisphere’s role. Reichle and
showed a marked (albeit not statistically reliable) Mason (in press) building on work by Just et al. propose
increase as the sentences became less causally that a limited capacity is exceeded across inferencing
related. Finally, the most interesting pattern of component processes. This then leads to processing
activation volume was observed in the right being passed to the right hemisphere. This is in contrast
hemisphere homologues of the language areas: the to an account proposed by Mason et al.; they propose
activation volume was consistent with the patterns that the right hemisphere is utilized during the
that had been reported with recognition and recall integration of an inference and that inferences are
measures. generated by utilizing the dorsolateral prefrontal cortex.
To account for their fMRI data, Mason and Just Finally, Beeman’s coarse coding theory leads to the
(2004) proposed that two different cortical networks prediction that inferences are accomplished as the result
support the generation and the integration of of activation of coarsely coded semantic information in
inferences during reading. The first network, the right hemisphere. In this account, it is not the
consisting of the left and right dorsolateral prefrontal inference per se that is processed in the right hemisphere
cortex is more involved in generating the inferences. but rather the information from which the inference is
As the causal distance between two sentences developed. A further account combines aspects of these
increases, the dorsolateral prefrontal regions generate previous explanations. Inference generation is supported
more inferences, leading to an increased volume of by the right hemisphere coarse semantic network.
activation. The second network, consisting of the Additional attempts to utilize this network will be
right inferior frontal gyrus, right superior and middle signaled by the dorsolateral prefrontal coherence
temporal gyri, and right inferior parietal lobe, is more monitor, provided resources are available. As the
involved in integrating the possible inferences that propositionalization work of the left anterior temporal
have been generated. Because of the relative text integration network becomes more demanding (and
differences in integrating the inferences in the three resources are consumed), this processing will spillover
conditions, the volume of cortical activation that was into the right anterior temporal region. It is clear that
observed in this region is described by an inverted U- further research will be needed to specify at which level
shaped function, with more activation occurring with of difficulty each of the networks are engaged for
the moderately related than either the highly-related various readers’ abilities.
or distantly-related sentences.
Reichle & Mason (in press) present a working New Perspectives on Figurative Language Processing
memory account for the additional right hemisphere The study of metaphor comprehension has long
activation that occurs as a result of generating an been a major area of interest in behavioral studies of
inference in the moderately related condition. They figurative language (e.g., Allbritton, McKoon & Gerrig,
10