ERIC EJ1092148: Evaluating the Impact of Two Globalization Projects on College Students' Cultural Competence and Cultural Intelligence (CQ)

2013·0.3 MB·English

by  ERIC

The Journal of Effective Teaching
an online journal devoted to teaching excellence

Evaluating the Impact of Two Globalization Projects on College
Students’ Cultural Competence and Cultural Intelligence (CQ)

Solange A. Lopes-Murphy1
Indiana State University, Terre Haute, 47809

Abstract

Cultural competence and CQ involve awareness of cultural similarities and differences,
knowledge of differences in cultural values, and intercultural encounters. To assess col-
lege students’ cultural competence and cultural intelligence gains, this experimental study
evaluated the impact of two globalization projects on these two constructs. The control
group conducted library research on aspects of cultural integration, whereas the treatment
group participated in a community engagement experience. There was no increase in cul-
tural competence and CQ as measured by the surveys for either of the two activities. Re-
sults also suggest that neither activity was more effective than the other. The study sug-
gests the need for sustained infusion of activities that enhance students’ cultural compe-
tence and CQ.

Keywords: Cultural competence, cultural intelligence, community engagement.

Multicultural and diversity sensitivity is a skill employers point out as being one of the
top ten skills they look for in prospective employees (Kennedy, 2012). The methods and
processes needed to equip future professionals to become globally conscious and globally
competent have attracted the attention of researchers (Ang et al., 2007), who are looking
at ways to incorporate cultural intelligent practices throughout a child’s education and to
create a plan to assess CQ.

According to the Cultural Intelligence Center website (2012), cultural intelligence (CQ)
is defined as “a consistent predictor of performance and adjustment in multicultural set-
tings.” To date, the body of research on CQ is still theory-focused (Sternberg & Grigo-
renko, 2006), and the experiences that lead individuals to adjust in multicultural settings
have not yet been widely explored (Gelfand, Erez, & Aycan, 2007). Few studies have
measured the effectiveness of different methods for increasing cultural competence and
intelligence. The scarcity of research on CQ is due to the fact the construct is still rather
new, especially as it relates to educational settings (Ang et al., 2007).

The objective of the present, empirical study is to compare the impact of two globaliza-
tion projects on college students’ cultural competence and CQ. The central research
question that guided this study was:

1 Corresponding author's email: [email protected]
The Journal of Effective Teaching, Vol. 13, No. 1, 2013, 5-18
©2013 All rights reserved
Lopes-Murphy 6

Do community engagement activities provide college students with higher levels
of cultural competence and CQ compared to projects that only require research
about facts related to other cultures?

Two additional research questions were examined:

Were there any student characteristics that predicted cultural competence and
CQ?

Did students’ perception about the effect of the projects on their cultural compe-
tence and CQ match the effect revealed by the pre- and post-surveys on cultural
competence and CQ?

Theoretical Background

There is an array of definitions of cultural competence and cultural intelligence. Accord-
ing to Ladson-Billings, cultural competence is “about student acquisition of cultural
knowledge regarding their own cultural ways and systems of knowing society and thus
expanding their knowledge to understand broader cultural ways and systems of knowing”
(as cited in Milner, 2011, p. 71). Cultural intelligence, on the other hand, is “the ability
to make oneself understood and the ability to create a fruitful collaboration in situations
where cultural differences play a role” (Plum, 2007, p. 1). A close look at both defini-
tions suggests that cultural intelligence includes a dimension that goes beyond the
“knowledge” level, a fundamental condition for culturally competent individuals. A cul-
turally intelligent person is someone who not only knows about differences in cultural
backgrounds, but who allows him/herself to be changed by intercultural encounters.

It is not uncommon for cultural competence and intelligence to be used interchangeably
in the literature. However, Peterson (2004) believes that competence is “…not something
we should ultimately strive for but rather, should excel beyond.” (p. 87). He then pro-
poses aiming for cultural intelligence as it represents the higher goal. Peterson defines
cultural intelligence as

…the ability to engage in a set of behaviors that uses skills (i.e., language or in-
terpersonal skills) and qualities (e.g., tolerance for ambiguity, flexibility) that are
tuned appropriately to the culture-based values and attitudes of the people with
whom one interacts. (p. 89)

Research by Alred, Byram, and Fleming (2006) and Zapata (2011) suggests that to help
individuals develop the behaviors, skills, and qualities that define them as being cultural-
ly competent and culturally intelligent, they need to engage in face-to-face encounters
with people who represent different cultures, values, beliefs, etc. It is through these in-
tercultural encounters that one can develop the skills needed to engage in healthy inter-
ethnic interactions and meet the needs of a pluralistic society, thus preparing these indi-
The Journal of Effective Teaching, Vol. 13, No. 1, 2013, 5-18
©2013 All rights reserved
Evaluating the Impact of Two Globalization Projects 7
viduals to compete in the global arena (Crider, 2007; Rivera, Johnson, & Ward, 2010).
Researchers (Ang et al., 2007, and Karnyshev & Kostin, 2010) also agree that to develop
effective graduates, university programs must focus on helping students understand “the
norms, practices and conventions in different cultures acquired from education and per-
sonal experiences” (Ang et al., 2007, p. 338). Such understanding includes awareness of
cultural similarities and differences (Brislin, Worthley, & MacNab, 2006) and knowledge
of differences in cultural values (Hofstede, 2001).

To gain a deeper understanding of cultural similarities and differences, college students
must learn more than just facts about other cultures; college curricula must include op-
portunities for personal experiences and intercultural encounters in a college setting (Za-
pata, 2011). Educators need to be intentional in their instructional design and incorporate
opportunities for cross-cultural interactions in their classes (Karnyshev & Kostin, 2010).
These interactions will expose these future professionals to situations that will require
them to intentionally select the right behaviors when interethnic contacts occur (Crider,
2007).

Community engagement activities can provide opportunities for these interactive, first-
hand encounters. Community engagement activities are those embedded in the academic
curriculum with the purpose of involving students in the community (Zapata, 2011).
Such involvement can happen in a variety of ways, as long as service is provided to the
members of the community through direct contact. Direct contact with individuals from
diverse groups and with stories from another culture will allow for cultural exposure and,
consequently, greater cultural understanding. These out-of-the-classroom activities pro-
vide numerous benefits in college education beyond cultural knowledge gain, among
which are awareness of differences in interests, values, and views; verbal and social
gains; prejudice reduction; and personal acceptance (Kuh, 1995; Tutt & McCarthy,
2006). According to Zapata (2011), community engagement activities have become one
of the most effective ways of promoting cultural understanding and competence to col-
lege students.

The Study

This study aimed to evaluate the impact of two globalization projects on college students’
cultural competence and CQ. The two projects were part of the course requirements for
two sections of an introduction to language and culture class at a public midwestern state
university during one academic semester. The course is required for all students majoring
and minoring in the Language Studies program. The course is an experiential introduc-
tion to the sources, representations, changes, varieties, and social, political and cultural
functions of language. Both sections of the course were taught by the same instructor,
and the course requirements were the same for both sections. In both sections of the
course, students were randomly assigned to participate in one of the two globalization
projects – the StoryWalk® project or the Globalized World project.

The Journal of Effective Teaching, Vol. 13, No. 1, 2013, 5-18
©2013 All rights reserved
Lopes-Murphy 8
The StoryWalk® Project

The StoryWalk® project is a community engagement activity created by Anne Ferguson
of Montpelier, Vermont, in collaboration with the Vermont Bicycle & Pedestrian Coali-
tion and the Kellogg-Hubbard Library (Vermont Bicycle & Pedestrian, 2012).

The project consists of a combination of reading storybooks while walking, thus promot-
ing physical fitness. The project involves the selection of reading materials targeted to
younger readers to stimulate their desire for reading in a fun and interactive way. Once
selected, the pages in the reading materials are separated, laminated, and attached to a
pole, a fence, or any other area that allows readers to go through the story while walking
on a path.

A central component of the StoryWalk® activity is the interaction that takes place be-
tween an experienced reader and a novice reader during the reading and walking process.
These interactive exchanges aim to stimulate novice readers’ reading skills through ques-
tioning, imaging, creating, predicting, and discovering. These are recommended strate-
gies to maintain children’s interest in the story (Adamson, 1980).

For the present study, the stories chosen focused on the topic of language and culture to
allow for the analysis of the impact of the activity on participants’ cultural competence
and CQ. Participants who were selected to work on this project were free to choose read-
ing materials on any culture of their interest as long as the materials met the five follow-
ing criteria:

1. The story line had to be appealing to readers in third through fifth grades.
2. The story should have minimal text and a great story line.
3. The story line needed to be such that it could be used in different seasons.
4. The illustrations should not cross the center of the book.
5. The books should be in average 30 pages long.

StoryWalk® participants selected four storybooks targeted to third through fifth grade
readers with stories on four different cultures and languages: Spanish, Korean, Brazilian,
and Japanese.

The StoryWalk® activity was conducted in a local elementary school with the highest
number of culturally and linguistically diverse students. Permission was given by the
school principal to have college students selected to work on this project to read stories
and interact with third through fifth graders on the school grounds during recess time.

The pages of the reading selections were attached to a long fence by the school play-
ground. Each project participant conducted the StoryWalk® activity with an average of
three to four elementary level children. The stories were read to four different groups of
elementary children for a total period of two hours on two different days.

The Journal of Effective Teaching, Vol. 13, No. 1, 2013, 5-18
©2013 All rights reserved
Evaluating the Impact of Two Globalization Projects 9
The Globalized World Project

The Globalized World was also a team-based project but did not provide opportunities
for participants to interact with anyone outside their teams. It required participants to
conduct a piece of library research on any aspect of cultural integration. Cultural integra-
tion could be researched from the perspective of integration of culturally and linguistical-
ly diverse students in PK-12 schools, religious integration in the American society, or any
aspect of cultural integration the team was interested in investigating.

Both projects - StoryWalk® and Globalized World - required participants to develop a
presentation to be delivered to the whole group at the end of the semester in addition to a
reflective journal all individuals had to turn in. The two projects differed only in that the
StoryWalk® contained a community engagement component, whereas the Globalized
World project did not.

Method

This study was approved by the Institutional Review Board where the study was con-
ducted.

Participants

The participants in this study were 67 undergraduate students at a co-educational
midwestern state university. The group combined both majors and minors in Language
Studies and was represented by a variety of academic levels (i.e., freshmen, junior, soph-
omore, and senior). Approximately half of the group (N = 35) was randomly selected to
work in the StoryWalk® project and the other half (N = 32) in the Globalized World pro-
ject. Two teams of six to nine members were formed to work on each project for a total
of two StoryWalk® teams and two Globalized World teams. All participants’ native lan-
guage was English. The majority of the participants (N = 58) spoke a foreign language at
a proficient or somewhat proficient level. Almost half of the group (N = 30) had traveled,
lived, or studied abroad and 63 out of the 67 participants had friends from different cul-
tures.

Instruments

Students in both sections of the course were asked to complete a pre-test and post-test
survey (see Appendix) that aimed to evaluate students’ cultural competence and CQ.
Students responded to the 21 statements in the survey on a 5-point Likert scale (1 = “fully
agree” and 5 = “fully disagree”). The statements in the survey were developed as a result
of a review of the literature on the types of experiences that lead to cultural competence
and CQ and analysis of existing validated surveys on cultural competence (Hammer,
Bennet, & Wiseman, 2003) and CQ (Van Dyne & Ang, 2006). A new survey was devel-
oped, rather than using an existing survey, because existing surveys include statements
that do not directly address the context of college level students. The survey developed
for the present study included statements that are similar to some of the statements in val-
The Journal of Effective Teaching, Vol. 13, No. 1, 2013, 5-18
©2013 All rights reserved
Lopes-Murphy 10
idated surveys but added statements that describe situations and experiences that college
students might encounter in an academic context.

Several questions were asked in such a way that the value of 5 (“Fully disagree”) on the
Likert scale represented the most culturally competent and intelligent response, but for
most of the questions, the response corresponding to most culturally competent and intel-
ligent had a value of 1 (“Fully agree”). To place all questions on the same scale and to
produce a measure of cultural competence and intelligence in which the higher scores
indicate greater cultural competence and intelligence, all responses were converted so
that a 5 represented the most culturally competent and intelligent response.

The survey was piloted in three classes (N = 68) during the semester prior to the present
study to assess the reliability of the instrument; reliability was reasonably high
(Cronbach’s alpha = 0.80).

Students completed the pre-test survey prior to becoming involved in the globalization
projects. In the pre-survey, the participants were asked to evaluate three additional
statements that asked participants whether: 1) they spoke a foreign language or a lan-
guage other than English; 2) they had traveled, lived, or studied abroad; 3) they had
friends from different cultures. The goal of these three items was to assess participants’
previous cultural experiences.

Post-surveys were completed at the end of the academic semester after students had con-
ducted their project presentations. The post-test survey contained the 21 questions from
the pre-test survey but did not include the questions on cultural experiences. Instead, it
added a question “Do you believe you have become more culturally competent as a result
of this activity?”. The purpose of this question was to allow a comparison between stu-
dent perceptions of increases with actual increases measured by pre- and post-test differ-
ences in the survey results.

The reliability of 21 questions for the pre-test survey was reasonable (Cronbach’s alpha =
0.73) but less than the reliability of the pilot administration of the survey. Cronbach’s
alpha was increased to 0.77 by removing seven questions from the analysis. For both the
pre-test and post-test surveys, the sum of the retained 14 questions was used to calculate a
score of cultural competence and intelligence. Because seven students on the pre-test
survey and four students on the post-test survey failed to answer one of the questions, a
percent score was used, which was calculated by dividing the sum of the scores for the
answered questions by the maximum possible sum for the answered questions (i.e., five
times the number of answered questions).

Normality of all variables were verified with Shapiro-Wilks, and if the distribution of the
variables differed significantly from normality, nonparametric tests were used. Equality
of variance between compared groups was verified with Levene’s test.

The Journal of Effective Teaching, Vol. 13, No. 1, 2013, 5-18
©2013 All rights reserved
Evaluating the Impact of Two Globalization Projects 11
Results

Sixty-seven students completed the pre-test survey, and 59 completed both the pre-test
and post-test surveys.

Comparison between Classroom Activities

Students in the StoryWalk® group did not show greater increases in cultural competence
and intelligence between the pre-test and post-test surveys than did students in the Glob-
alized World group (Figure 1). The average change in the cultural competence and intel-
ligence score did not differ between the students who completed the StoryWalk® project
(average ± SD = 0.98 ± 6.11) and the students who completed the Globalized World pro-
ject (1.19 ± 5.81; independent samples t-test, t = -0.136, df = 58, p = 0.89).

Figure 1. Average percent scores on pre- and post-test surveys of cultural compe-
tence for groups of students completing two different projects. Error bars represent
95% confidence intervals.

Because there were no differences between the two groups in the change in cultural com-
petence and cultural intelligence, the two groups were pooled to determine whether stu-
dents’ cultural competence and intelligence increased between the pre-test and post-test
surveys as a result of the projects. There was no significant change in the cultural com-
petence and intelligence scores of students from the pre- to the post-test (mean change
= -1.08; 95% confidence interval = -2.61 – 0.45; paired t-test, t = -1.41, df = 59, p =
0.16).

The Journal of Effective Teaching, Vol. 13, No. 1, 2013, 5-18
©2013 All rights reserved
Lopes-Murphy 12
Predictors of Cultural Competence & Cultural Intelligence

The pre-test measure of cultural competence and intelligence was not related to any of the
three measures of cultural experiences. Students who spoke a foreign language (checked
“fully agree” to “agree to some extent” on question 1, N = 58) did not have significantly
higher pre-test scores (average = 78.74) than students who did not speak a foreign lan-
guage (checked “disagree” or “fully disagree”, N = 9, average = 74.68, independent sam-
ples t-test, t = 1.39, df = 65, p = 0.17). Students who had traveled, lived, or studied
abroad (checked “fully agree” to “agree to some extent” on question 2, N = 33) did not
have significantly higher pre-test scores (average = 78.99) than students who did not
(checked “disagree” or “fully disagree”, N = 34, average = 77.42, independent samples t-
test, t = 0.77, df =65, p = 0.44). Students who had friends from other cultures (checked
“fully agree” to “agree to some extent” on question 3, N = 63) did not have significantly
higher pre-test scores (average = 78.20) than students who did not (checked “disagree” or
“fully disagree”, N = 4, average = 78.26, independent samples t-test, t = –0.016,
df = 65, p = 0.99). In contrast to the individual measures of cultural experience, there
was a statistically significant negative correlation between student’s pre-test scores and
the sum of their responses to the questions on foreign language proficiency, experience
abroad, and friends from other cultures (Spearman rank correlation r = –0.357,
s
p = 0.003).

Student Perceptions

The proportion of students indicating that their cultural competence and intelligence had
increased as a result of their participation in the activities did not differ between those
completing the StoryWalk® group (23 of 31 = 74.2%) and the Globalized World group
(21 of 28 = 75.0%; p = 0.643 Fisher exact test). Of the 59 students that completed the
pre- and post-test surveys, 44 (74.58%; 95% confidence interval 63.4 – 86.4%) reported
that they felt that their cultural competence and cultural intelligence had increased as a
result of their participation in the activities. This proportion was significantly greater than
50% (p < 0.001, binomial test).

The students’ perception of whether their cultural competence and intelligence changed
as a result of the activities did not match the change measured by the surveys. There was
no relationship between the change between pre-test and post-test scores and whether
students felt that the activities increased their cultural competence and cultural intelli-
gence (point biserial correlation, r = 0.018, p = 0.892).

Discussion

The present study focused on one central research question.

Do community engagement activities provide college students with higher levels
of cultural competence and CQ compared to projects that only require research
about facts related to other cultures?

The Journal of Effective Teaching, Vol. 13, No. 1, 2013, 5-18
©2013 All rights reserved
Evaluating the Impact of Two Globalization Projects 13
Participants’ responses in the pre- and post-surveys show that there was no increase in
cultural competence and CQ as measured by the surveys for either of the two activities.
Results also suggest that neither activity is more effective than the other.

A likely reason why the participants in the StoryWalk® project did not show a greater
increase in their cultural competence and CQ than participants in the Globalized World
project may have been related to the length of time students were involved in this com-
munity engagement activity. That is, participants were engaged in this interactive activity
for a total of two hours throughout the semester. Although they interacted with children,
a group with whom they may not interact with frequently and who represented different
cultures, languages, religious backgrounds, and ability levels, the amount of time that al-
lowed for those interactions may have been too limited to generate significant changes in
their cultural competence and cultural intelligence. The short time of engagement was
also true of the Globalized World project and may, therefore, also explain why that group
did not show an increase in their cultural competence and CQ. A complete and thorough
test of the hypothesis that community engagement activities are more effective than li-
brary research projects in increasing students’ cultural competence and CQ would require
examination of activities that are sustained over longer periods of time.

Two additional questions were examined in the study. They were:

(1) Were there any student characteristics that predicted cultural competence and
CQ?
(2) Did students’ perception about the effect of the projects on their cultural com-
petence and CQ match the effect revealed by the pre- and post-surveys on cul-
tural competence and CQ?

Results show that there is either no relationship or perhaps a negative relationship be-
tween students’ previous cultural experiences and cultural competence and intelligence as
measured by the survey. Interestingly, there was a negative relationship between cultural
competence and intelligence and the sum of the scores on the cultural experience ques-
tions.

It is possible that those individuals with a greater combined proficiency in a foreign lan-
guage, extensive experience abroad, and larger number of friends from different cultures
may be more sensitive as to where they stand in the cultural competence and cultural in-
telligence continuum. These participants may have a deeper understanding of these two
constructs and, therefore, understand they are not as culturally competent and intelligent
as they would like to become. In contrast, those with fewer or less extensive cultural ex-
perience may have a more superficial understanding of the meaning of the two constructs
and the behaviors and skills of a truly culturally competent and intelligent person.

The results also show that there is no alignment between the students’ perceptions of their
own increases in cultural competence and intelligence and increases measured by the sur-
vey. The majority of the participants thought their cultural competence and cultural intel-
ligence increased as a result of the projects; however, their responses on the pre- and
The Journal of Effective Teaching, Vol. 13, No. 1, 2013, 5-18
©2013 All rights reserved
Lopes-Murphy 14
post-surveys show they have not. One possible explanation would be that the validity of
the instrument is not sufficiently high. Although the instrument was based on what the
literature identifies as culturally competent and culturally intelligent skills and behaviors
and the types of statements included in other validated surveys, there remains the possi-
bility that the survey used in this study may not have measured cultural competence and
cultural intelligence accurately.

Another, more likely, explanation is that the participants’ own perceptions regarding the
two constructs are not accurate. Considering that this was their first semester in a lan-
guage studies class, they may not yet clearly understand the meaning of cultural compe-
tence and cultural intelligence and what is entailed by culturally competent and culturally
intelligent behaviors. Even though the majority of the participants have had some level
of exposure and interaction with culturally and linguistically diverse individuals and may
have immersed themselves in a foreign context, the participants may not have had suffi-
cient experiences or training to truly understand the meaning of cultural competence and
cultural intelligence. They may have considered themselves competent and intelligent
from the cultural perspective of their previous exposure to diversity, yet that exposure
may not have been sufficient to develop a deep understanding of cultural competence and
intelligence.

A final possible explanation about the lack of alignment between students’ perceptions of
their own increases in cultural competence and cultural intelligence and increases meas-
ured by the surveys may be responder bias. Students may have responded in the way
they believed the instructor expected them to answer.

A review of 22 studies on intercultural research and outcomes assessment from 1992
through 2012 reveals the uniqueness of the present study. To date, all studies have fo-
cused on intercultural competence and intelligence as a result of immersion experiences,
in particular study abroad experiences (Bender, Wright, & Lopatto, 2009; Fry & Paige,
2007). None of the studies reviewed used a randomized design in an educational setting
to evaluate the impact of course projects on cultural competence and cultural intelligence.
There was only one study (Westrick, 2004) that examined the impact of service-learning
activities on students’ intercultural sensitivity. The results of the study show that alt-
hough service-learning or community engagement activities can influence the develop-
ment of multicultural and diversity sensitivity in students, they do not necessarily do so.
These findings suggest the need for empirical research, similar to the one done in the pre-
sent study, to examine the types of classroom activities that may impact learners’ cultural
competence and intelligence and to develop a plan to assess college students’ cultural
competence and CQ over the course of their undergraduate studies.

Implications

The results of the present study have important implications for how universities attempt
to enhance the cultural competence and CQ of their students. Future empirical research
is needed to determine whether other short-term activities might increase cultural compe-
tence and CQ in college students. Data are needed on the specific experiences that lead
The Journal of Effective Teaching, Vol. 13, No. 1, 2013, 5-18
©2013 All rights reserved