ERIC ED573087: Programs for Engagement and Enhancement. Professional File. Article 131, Spring 2013

2013·6.7 MB·English

by  ERIC

PROGRAMS FOR ENGAGEMENt ANd ENhANcEMENt
Gloria Crisp, Lisa Palacios, and John Kaulfus
PROFESSIONAL FILES | SPRING 2013 VOLUME
Supporting quality data and decisions for higher education.
© Copyright 2013, Association for Institutional Research
Letter from the Editor
AIR is pleased to reintroduce Professional Files. This volume contains three articles recom-
mended for inclusion by AIR members through a blind peer-review process. Each article
puts forth ideas and analysis for our consideration as institutional research professionals.
Most salient for me in my role as volunteer coordinating editor is that the articles frame
the work of IR as more than pure technical analysis through acknowledgment of the im-
portance of context.

Zan, Yoon, Khasawneh, and Srihari unpack differences in enrollment projections, with a particular focus on the influ-
ence of the unit of analysis on the appropriateness of the model. Saupe and Eimers examine the under-appreciated
area of grade point average reliability; something that will be of greater importance as policy dialogues move to multi-
ple measures of college readiness and adjusted metrics of student outcomes. Crisp, Palacios, and Kaulfus empower IR
professions in efforts to better understand student engagement efforts on campus.

It has been an honor to be a part of this volume of Professional Files and to strive to elevate the work of peers for the
benefit of the field. We look forward to receiving your submission next.

Sincerely,
Christopher M. Mullin
IN thIS ISSUE...
EdItORS
Article 129 Page 1
Author: Zan et al. Christopher M. Mullin
A Comparison Study of Return Ratio-Based Volunteer Coordinating Editor
Academic Enrollment Forecasting Models
American Association of Community
Colleges
Article 130 Page 13
Leah Ewing Ross
Author: Saupe & Eimers
Managing Editor
Alternative Estimates of the Reliability
Association for Institutional Research
of College Grade Point Averages
Lisa Gwaltney
Editorial Assistant
Article 131 Page 23
Author: Crisp et al. Association for Institutional Research
Programs for Engagement and Enhancement
ISSN 2155-7535
PROFESSIONAL FILE
ARtIcLE 131
© Copyright 2013, Association for Institutional Research
PROGRAMS FOR ENGAGEMENt ANd
ENhANcEMENt
Gloria Crisp, Lisa Palacios, and students and enhance their lives by connections with peers, and out-of-
John Kaulfus introducing new ideas, challenging class interactions with faculty), that in
past behaviors or events, and creat- turn directly impact traditional mea-
ing intellectual discord and tension sures of student success (i.e., grades,
About the Authors
(Keeling, Wall, underhile, & Dungy, persistence).
Gloria Crisp is associate professor of
2008). Institutional effectiveness is
higher education at The university of
dependent, in part, on institutions According to a survey involving 185
Texas at San Antonio. Lisa Palacios is
providing students with opportuni- colleges and universities across the
director of graduate recruitment at The
ties to purposefully engage (Harper & country, the most prevalent services
university of Texas at San Antonio. John
Quaye, 2009). According to Pascarella, and programs provided to students to
Kaulfus is assistant vice president/dean
“an excellent undergraduate education promote student engagement in the
of students at Texas A&M university-
is most likely to occur at those colleges first year include tutoring, academic
Commerce.
and universities that maximize good coaching and counseling, writing sup-
practices and enhance student engage- port services, academic advising, and
Abstract
ment” (2001, p. 22). As such, institutions testing services (National Resource
The following article describes pro-
that value student success will take Center for the First-Year Experience
grams used by universities and colleges
every opportunity to engage students and Students in Transition, 2008). In
to engage students; these programs
both academically and socially (Culp, large part due to work by Kuh and col-
include mentoring, learning communi-
2007). leagues, engagement programs and
ties, and first-year success courses and
activities have become increasingly
programs. We begin with a brief over-
Simply defined, student engagement is viewed as an important component
view of student development theory,
how universities organize their human of student success (Kuh, Kinzie, Buck-
program descriptions and citations,
capital and resources to encourage ley, Bridges, & Hayek, 2007). As such,
and article summaries for key refer-
students to involve themselves in aca- increasing attention has been given to
ences. Next, we introduce prominent
demic, interpersonal, and cocurricular the implementation, administration,
national surveys available to institu-
activities (Astin, 1993). Student engage- and assessment of educational experi-
tions that are interested in measuring
ment is typically not viewed as a direct ences designed to engage students.
student engagement (inside or outside
measure of student learning, but rather
formal programs). We conclude with
is used as a measurement of participa- Although engagement programs are
additional references and recommen-
tion in meaningful educational experi- typically created and managed by stu-
dations for institutional researchers
ences and activities that facilitate both dent affairs professionals, institutional
involved in program review and/or stu-
social and academic integration (Tinto, researchers should be familiar with
dent outcomes assessment of student
2000) and lead to student development programmatic efforts on their campus,
engagement programs.
(LaNasa, olson, & Alleman, 2007). More and should understand how program
specifically, opportunities for students outcomes can be used to address ac-
INtROdUctION
to engage are provided through for- creditation standards and institutional
Higher education is not a passive expe- malized programs designed to directly planning and assessment goals (as
rience that leaves students untouched. support student integration and/or de- demonstrated in volume 141 of New
Rather, college life involves a variety of velopment outcomes (i.e., study strate- Directions for Institutional Research,
experiences, both inside and outside gies, time/stress management skills, 2009). The present article describes
the classroom, designed to engage motivation, academic self-confidence, several programs currently used by
SPRING 2013 VoLuME | PAGE 23
postsecondary institutions to engage table 1. Student Engagement theory References
students with the intent of providing
institutional researchers with knowl-
References for Student Impact for Institutional
edge to support assessment efforts.
Engagement Theory Researchers
The article begins with an overview of
relevant student development theory
Astin, A. (1993). What matters in col- Presents the results from a study of
that serves as a conceptual ground-
lege: Four critical years revisited. San almost 25,000 students at 200 colleges
ing for engagement programs. Next,
Francisco: Jossey-Bass. and universities. Findings demonstrate
we provide program descriptions for how colleges and universities can
programs that have been linked to enhance student development during
engagement (e.g., social engagement, college through a variety of in-class
academic skills, time management, and out-of-class experiences.
and career selection) and academic
outcomes. Citations and article sum- Hamrick, F. A., Evans, N. J., & Schuh, Provides an overview of theories that
maries for key references are provided J. H. (2002). Foundations of student relate to student growth and develop-
in table form following each section for affairs practice: How philosophy, theory, ment during college and explores ways
and research strengthen educational that institutions can enhance students’
institutional research professionals who
outcomes. San Francisco: Jossey- educational experiences.
are interested in learning more about
Bass.
student engagement and/or enrich-
ment programs. Third, we highlight
Harper, S. R., & Quaye, S. J. (2009). Based on theory, explores ways that
prominent national surveys available to
Student engagement in higher edu- diverse populations of students (e.g.,
institutions that are interested in mea-
cation: Theoretical perspectives and racial and ethnic minorities, LGBT
suring student engagement (inside or
practical approaches for diverse popu- students) as compared to nondiverse
outside of formal programs). The article
lations. New York: Routledge. populations might struggle to engage
concludes with additional references during college.
and recommendations for institutional
researchers involved in program review Kuh, G. D., Kinzie, J., Schuh, J. H., & Provides a theoretical framework, the
and/or student outcomes assessment Whitt, E. J. (2005). Assessing condi- Inventory for Student Engagement and
of student engagement programs. tions to enhance educational effective- Success (ISES), to examine student
ness. The inventory for student en- engagement within a program, division,
gagement and success. San Francisco: college, or entire institution. Explains
OVERVIEw
Jossey-Bass. how information can be used for pro-
OF StUdENt gram reviews, planning, and accredita-
tion.
dEVELOPMENt
Pascarella, E., & Terenzini, P. T. (2005). Presents findings from 15 years of
thEORy
How college affects students: A third research on college’s impact on student
Research and theory by Erikson and decade of research. San Francisco: learning, growth, and development.
Chickering provide a foundation for our Jossey-Bass. Also presents implications for research,
policy, and practice.
current understanding of student de-
velopment. Erikson’s theory of psycho-
Tinto, V. (1993). Leaving college: Synthesizes research on student reten-
social development (1968) explains that
Rethinking the causes and cures of tion demonstrating the importance of
individuals must work through eight
student attrition (2nd ed.). Chicago: institutions providing students with op-
stages in order to successfully form
University of Chicago Press. portunities to engage with the campus
an identity and discover purpose and
community.
meaning in life. According to Erikson,
adolescents move through a develop-
mental stage during college termed the ing a personal identity; it is defined by student development (1969) explain
“identity versus role confusion” stage a “crisis” that must be resolved in order that college students move through
before moving into adulthood. This for students to avoid an “identity crisis” seven vectors or stages as they become
stage involves students successfully, or that leads to stagnation or regression. more self-aware and as they have more
in some cases unsuccessfully, develop- Similarly, Chickering’s seven vectors of complex thoughts, which is spurred
PAGE 24 | SPRING 2013 VoLuME
by interactions with peers and faculty overview of engagement programs, ment measure (Crisp & Cruz, 2009).
and the introduction of new concepts but rather to provide institutional However, according to a comprehen-
and ideas. Chickering’s work has since researchers with examples of programs sive review of the psychological, busi-
been updated to be inclusive of non- currently employed on college and ness, and education literature by Nora
traditional students (i.e., Chickering & university campuses across the country and Crisp (2007), students perceive
Reisser, 1993). that have been empirically shown to a holistic mentoring experience to
enhance students’ experiences and to include four separate yet interrelated
More recently, Erikson and Chickering’s promote students engagement. types of support: (1) psychological
work has been expanded by Astin and and emotional support, (2) support for
Tinto in an effort to understand the Mentoring Programs setting goals and choosing a career
factors related to student success and Mentoring programs that involve a vari- path, (3) academic subject knowledge
persistence. Astin’s theory of involve- ety of engagement activities such as support aimed at advancing a student’s
ment (1984, 1999) postulates that academic advising, academic skills de- knowledge relevant to his or her cho-
student involvement in college has a velopment, personal development, and sen field, and (4) support in the form of
direct impact on psychosocial develop- career selection are becoming increas- a role model.
ment and assists in identity formation ingly prevalent. Mentoring programs
as students work toward graduation. and experiences have been empirically An assortment of mentoring programs
Astin’s work also demonstrates that shown to be associated with numer- designed to serve a variety of student
student learning and development are ous academic and developmental populations including first-generation,
dependent on active involvement in outcomes, including improving critical minority, at-risk, and/or low-income
academic and social aspects of a col- thinking skills, self-confidence, per- students have been described in the
lege experience. Moreover, his theory sistence, and academic performance. literature (e.g., Bordes & Arredondo,
argues that development is influenced Mentoring has also been found to help 2005; Pagan & Edwards-Wilson, 2003;
by both the quality and the quantity of students develop their latent abilities, Wallace, Abel, & Ropers-Huilman,
involvement. and to raise students’ expectations and 2000). For instance, the Puente Project,
future aspirations (e.g., Astin, 1999; evaluated by Laden (1999), is a nation-
Similarly, Tinto’s theory of student de- Bank, Slavings, & Biddle, 1990; Camp- ally recognized program designed to
parture (1993) demonstrates that stu- bell & Campbell, 1997; Freeman, 1999; raise Latino/a students’ educational
dents are more likely to persist toward Girves, Zepeda, & Gwathmey, 2005; and career aspirations. other examples
graduation if they become socially and Johnson, 1989; Mangold, 2003; Pagan of programs that involve a mentoring
academically integrated into the col- & Edwards-Wilson, 2003; Roberts, 2000; component include TRIo Programs
lege environment. He postulates that Ross-Thomas & Bryant, 1994). (Wallace et al., 2000), the Adventor Pro-
integration is achieved when a student gram (Shultz, Colton, & Colton, 2001),
and the institution share similar values Institutional researchers should con- and the Search for Education, Eleva-
and the student is engaged in positive sider and draw from published pro- tion and Knowledge (SEEK) Program
social and academic interactions. Tinto’s gram overviews and evaluations when (Sorrentino, 2007). Table 2 provides a
work demonstrates the importance assisting in the development and/or list of select published work including a
of support from faculty and university assessment of programmatic efforts. mentoring theory and scale (i.e., Crisp,
staff. Table 1 contains key references unfortunately, there is little agreement 2009) to guide assessment efforts.
related to theory underpinning student regarding how college students experi-
engagement programs to guide the ence mentoring, or on the components Learning communities
development of programmatic activi- that should be included in a mentoring Recently, there has been increased
ties and goals. program. Moreover, it has been noted interest from both academic and
that the majority of empirical work on student affairs practitioners to enhance
ENGAGEMENt mentoring has been limited due to and/or expand innovative programs
methodological weaknesses including such as learning communities and
PROGRAMS
limitations in how mentoring is defined first-year experiences (Dale & Drake,
and measured, a lack of sophisticated 2007). Learning communities provide
The following section provides a
data analysis and theoretical ground- college students with the opportunity
program overview for mentoring,
ing, failure to control for selection bias, to get to know other students as well
learning communities, and first-year
and an overreliance of self-reported as faculty; these communities integrate
success courses and programs. This is
benefits of mentoring as the assess- students into the university commu-
not meant to provide a comprehensive
SPRING 2013 VoLuME | PAGE 25
table 2. Mentoring Program References
There is a wealth of literature on learn-
ing communities to suggest that pro-
References for Designing/ Impact for Institutional
grammatic efforts can be used to influ-
Assessing Outcomes for Researchers
ence retention and learning outcomes.
Mentoring Programs
Namely, ongoing evaluations of the
opening Doors Learning Communities
Crisp, G. (2009). Conceptualization and Offers a theoretically grounded survey
(oDLC) program by MDRC are utilizing
initial validation of the College Student to be used by institutions that are
Mentoring Scale (CSMS). Journal of interested in measuring the mentoring experiments that test a cause-and-ef-
College Student Development, 50(2), experiences of undergraduate college fect relationship between participation
177–194. students. Includes the 25-item survey in learning communities and outcomes
as an appendix. for various groups of students through
the use of random assignments (e.g.,
Laden, B. V. (1999). Socializing and Presents highlights from the Puente Bloom & Sommo, 2005; Richburg-
mentoring college students of color: Project, a program designed to support Hayes, Visher, & Bloom, 2008; Scrivener,
The Puente Project as an exem- first-generation Latino/a college stu- Bloom, LeBlanc, Paxson, & Sommo,
plary celebratory socialization model. dents at California community colleges.
2008). Additionally, nonexperimental
Peabody Journal of Education, 74(2),
research by Zhao and Kuh (2004) has
55–74.
revealed that students who partici-
pate in the learning community have
Shultz, E. L., Colton, G. M., & Colton, Describes the Adventor Program,
higher levels of academic effort, active
C. (2001). The Adventor Program: designed by the College of Education
learning, interactions with faculty, and
Advisement and mentoring for students at Kutztown University to serve racial/
participation in diversity activities.
of color in higher education. Journal of minority students. Presents informa-
Participants also reported more posi-
Humanistic Counseling, Education, and tion about the program design and
Development, 40, 208–218. implementation, and follows with pilot tive associations with advisers, campus
findings. support services, and overall experi-
ences, as well as self-reported gains in
Sorrentino, D. M. (2007). The SEEK Presents a description and overview personal and social development and
mentoring program: An application of of the SEEK Program at the College of basic skills advancement. Furthermore,
the goal-setting theory. Journal of Col- Staten Island, City University of New qualitative work by Tinto and Goodsell
lege Student Retention, 8(2), 241–250. York (CUNY), that provides academic (1993) involving a linked writing course
mentoring to students at risk for aca- and seminar found that learning com-
demic dismissal.
munities supported the development
of students’ time management, writing,
Wallace, D., Abel, R., & Ropers- Utilizes qualitative interview data to and study skills. Table 3 provides a list
Huilman, B. R. (2000). Clearing a path explore first-generation, low-income
of key references to studies on learning
for success: Deconstructing borders students’ experiences with and percep-
communities.
through undergraduate mentoring. The tions of formalized mentoring programs.
Review of Higher Education, 24(1),
First-year/Orientation/Success
87–102.
Programs
nity in a meaningful way (Price, 2005; et al., 2003; Price, 2005). Learning com- According to a survey by the National
Taylor, Moore, MacGregor, & Lindblad, munities exist in a variety of formats Resource Center for the First-Year
2003; Tinto, 1998; Weber, 2000). More to facilitate students’ connection to Experience and Students in Transition
specifically, learning communities faculty, other students, and the institu- (2008), nearly 85% of colleges and
provide students with an educational tion (Tinto, 1998); these formats include universities currently offer a first-year
environment that supports student team-taught programs, paired or program. First-year programs, stu-
engagement through an integrated clustered courses, cohorts of students dent success courses, and orientation
and interdisciplinary curriculum that enrolled together in large courses, and courses all focus on assisting college
may cross departments or divisions and residence-based programs (Price, 2005; students’ transition and/or enhancing
focuses on high levels of participation Taylor et al., 2003; Tinto, 1998; Weber, engagement and success in college
and support from faculty (Brower & 2000). (Cook, 1996). These programs are de-
Dettinger, 1998; oertel as cited in Taylor signed to teach students strategies for
PAGE 26 | SPRING 2013 VoLuME
table 3. Learning communities References ic efforts may be offered as both credit
and noncredit courses (Donnangelo &
Santa Rita, 1982), and may be required
References for Designing/ Impact for Institutional
or optional (Zimmerman, 2000). Pos-
Assessing Outcomes for Learn- Researchers
sible program offerings range from
ing Communities
a half-day orientation (Hollins, 2009)
to semester- or year-long programs
Richburg-Hayes, L., Visher, M. G., & Details the design and results of a
(Donnangelo & Santa Rita, 1982; Glass
Bloom, D. (2008). Do learning com- study that utilized an experimental
munities affect academic outcomes? design to explore a cause-and-effect & Garrett, 1995).
Evidence from an experiment in a com- relationship between participation in
munity college. Journal of Research on a learning community and students’ Although the majority of research to
Educational Effectiveness, 1(1), 33–65. engagement and attachment to the date has focused on examining the
institution’s community. impact of programs on retention or
learning outcomes (e.g., Derby & Smith,
Scrivener, S., Bloom, D., LeBlanc, A., Explains how Kingsborough’s Opening 2004; Glass & Garrett, 1995; Pascarella,
Paxson, C. E., & Sommo, C. (2008). Doors Learning Communities (ODLC) Terenzini, & Wolfle, 1986; Raymond &
A good start: Two-year effects of a program utilizes an experimental de-
Napoli, 1998; Stovall, 1999), findings
freshmen learning community program sign with random assignment to study
from the National Resource Center for
at Kingsborough Community College. the effect of learning communities on
the First-Year Experience and Students
MDRC, New York. student success.
in Transition (2008) survey indicate
that success courses may also be as-
Taylor, K., Moore, W. S., MacGregor, Presents findings from a systematic
sociated with engagement outcomes
J., Lindblad, J. (2003). What we know literature review of research and as-
such as increasing peer connections,
now. National Learning Communities sessment specific to learning communi-
use of campus services, participation
Project Monograph Series. The Wash- ties conducted by the National Learning
ington Center for Improving the Quality Communities Project. in campus services, and out-of-class
of Undergraduate Education at The interaction with faculty. Additionally,
Evergreen State College in coopera- research conducted by the Commu-
tion with the American Association for nity College Research Center (CCRC) at
Higher Education, Washington, DC. Teachers College, Columbia university,
found that programmatic efforts may
Tinto, V. (1997). Classrooms as Presents a mixed methods study of the integrate students both socially and
communities: Exploring the educa- use of learning communities at Seattle academically by helping to facilitate
tional character of student persistence. Central Community College. The study
the development of students’ relation-
Journal of Higher Education, 68(6), investigates the impact on collabora-
ships with faculty and other students
599–623. tive learning strategies on both student
(o’Gara et al., 2008). Moreover, evalua-
persistence and learning.
tion efforts at the Virginia Community
College System examined the impact of
Tinto, V. (1998). Learning communities Details a case study specific to the
a comprehensive approach to student
and the reconstruction of remedial edu- effectiveness of a developmental edu-
orientation that included a half- to
cation in higher education. Prepared cation learning community. Provides a
full-day program (Seeking oppor-
for presentation at the “Conference on description of the program, research
Replacing Remediation in Higher Edu- procedures, and findings. tunities through Academic Recruit-
cation” at Stanford University, January ment [SoAR]), group advising, and an
26–67, 1998, sponsored by the Ford orientation course. Findings indicated
Foundation at the U.S. Department of that the program increased students’
Education. personal adjustment during the
transition process and academic gains
success in college by introducing them Derby & Watson, 2006; Glass & Garrett, among first-semester students. The
to campus facilities, resources, and 1995; Grunder & Hellmich, 1996; Na- orientation course was also found to
services; and/or by enhancing students’ tional Resource Center for the First-Year assist students in developing effective
health or well-being, study skills, time Experience and Students in Transition, study habits, career and academic plan-
management, or learning styles (e.g., 2008; o’Gara, Karp, & Hughes, 2008; ning, and knowledge regarding college
Derby, 2007; Derby & Smith, 2004; Raymond & Napoli, 1998). Programmat- resources (Hollins, 2009).
SPRING 2013 VoLuME | PAGE 27
Resources and support regarding table 4. First-year/Orientation/Success Program References
first-year programs are available to
researchers through the First-Year
References for Designing/ Impact for Institutional
Experience (http://www.sc.edu/fye/).
Assessing Outcomes for First Researchers
Additionally, Table 4 provides a list of
Year/Orientation Success
references specific to designing and as- Programs
sessing outcomes for first-year, orienta-
tion, and student success courses and
Engberg, M. E., & Mayhew, M. J. Examines the impact of a first-year
programs. (2007). The influence of first year program on a variety of outcomes,
“success” courses on student learning including multicultural awareness,
tOOLS FOR and democratic outcomes. Journal of commitment to social justice, and the
College Student Development, 49(2), complexity of attributes. Also includes
ASSESSING StUdENt 95–109. a discussion of connecting theory with
practice.
ENGAGEMENt
Keup, J. R., & Barefoot, B. O. (2005). Utilizes data from the Cooperative In-
Several national surveys are available
Learning how to be a successful stu- stitutional Research Program’s (CIRP)
to institutions interested in assessing
dent: Exploring the impact of first year 2000 freshman survey and the Your
student engagement and/or students’
seminars on student outcomes. Journal First College Year (YFCY) 2001 survey
experiences during college, including
of the First year Experience, 17(1), to study the relationship between taking
the National Survey of Student Engage-
11–47. a first-year course and academic and
ment (NSSE; http://nsse.iub.edu/). This social experiences of students.
survey contains items assumed to mea-
sure different components of student
O’Gara, L. Karp, M. M., & Hughes, K. Investigates the perceived impact of
engagement, including academic chal- L. (2009). Student success courses in taking a success course on individual
lenge (e.g., preparing for class, using the community college: An exploratory benefits such as building relationships
higher-order thinking skills), active and study of student perspectives. Commu- with faculty and developing study skills
collaborative learning (e.g., contribut- nity College Review, 36(3), 195–218. at two urban community colleges in the
ing to class discussions, working with Northeast.
students outside of class), and student
interactions with faculty members (e.g., Schwitzer, A. M., McGovern, T. V., & Presents findings of an evaluation
talking about career plans, working Robbins, S. B. (1991). Adjustment investigating the relationship between
on activities other than coursework) outcomes of a freshman seminar: A participation in a college orientation
utilization-focused approach. Journal seminar and students’ social and aca-
(Kuh, 2004). Additionally, seniors report
of College Student Development, 321, demic adjustment in college.
whether they participated in various
484–489.
programs and on-campus activities,
including learning communities. The
Zeidenberg, M., Jenkins, D., & Cal- Examines the impact of enrolling in
NSSE is typically administered in the
cagno, J. C. (2007). Do student suc- a student success course over the
spring using a paper or online version
cess courses actually help community course of 17 semesters on various stu-
of the survey to a random sample of
college students succeed? Community dent outcomes, controlling for possible
first-year and senior students (Kuh, College Research Center (CCRC) Brief extraneous variables.
Cruce, Shoup, Kinzie, & Gonyea, 2008). No. 36, June.
Variations of the NSSE that measure
engagement of different student measured by the Faculty Survey of Stu- opment during the first year of college
populations are also available, includ- dent Engagement (FSSE) (http://fsse. is the Your First College Year (YFCY)
ing the Community College Survey of iub.edu/). Tips and recommendations survey, developed through collabora-
Student Engagement (CCSSE) (http:// for analyzing and interpreting the NSSE tion between the Higher Education
www.ccsse.org/) and Beginning College survey data are available in a 2009 is- Research Institute (HERI) and the Policy
Survey of Student Engagement (BCSSE) sue of New Directions for Institutional Center on the First Year of College at
(http://bcsse.iub.edu/). Moreover, many Research by Chen and colleagues. Brevard College (http://www.heri.ucla.
institutions compare student responses Another survey available to institutions edu/yfcyoverview.php). This survey
from the NSSE with faculty perceptions interested in assessing students’ devel- allows colleges and universities to
PAGE 28 | SPRING 2013 VoLuME
identify students’ experiences during in student development theory (Dale dent affairs personnel have knowledge
the first year that encourage and sup- & Drake, 2007). Institutional research- of long-standing and professionally
port student involvement, satisfaction, ers should work with faculty as well as accepted student development theory
and learning, as well as other success academic and student affairs personnel (King & Howard-Hamilton, 2000) that is
indicators that enhance first-year to utilize previously validated assess- likely to strengthen assessment efforts
programs. Similar to the NSSE, the YFCY ment tools and survey items that and properly take into account factors
allows for comparisons to national and are grounded in theory, rather than such as ethnicity, culture, and career
institutional peer groups among par- developing home-grown surveys that choice (Pickering & Sharpe, 2000). Stu-
ticipating institutions as well as trend may or may not be accurate measures dent and academic affairs practitioners
and longitudinal analyses. The YFCY is of students’ experiences. Additional and faculty also have ready access and
offered in both paper and web format recommendations specific to using can encourage student participation
and is conducted at the end of the stu- engagement data in assessment and in surveys needed to properly assess
dents’ first academic year (somewhere planning efforts are provided by Banta, student outcomes (Smith & Mather,
between the months of March to June). Pike, and Hansen (2009). 2000). Moreover, institutional research
offices may be able to provide precol-
Third, the Degrees of Preparation sur- When possible, we also strongly lege data to student affairs divisions
vey may also be of interest to institu- encourage the use of experimental to guide and inform the development
tions in measuring ways that college designs that utilize random assignment of programmatic activities (Cole et al.,
experiences are related to various to groups and an experimental (i.e., 2009). Kinzie and Pennipede (2009)
developmental and civic outcomes, students in the program) and control provide further discussion and recom-
including critical thinking skills, career- group (i.e., group of students who do mendations for collaborating with
related experiences, and civic engage- not participate) to assess cause-and- student affairs in using data. Addition-
ment. This survey’s major components effect relationships between program ally, a New Directions article by Nelson
and question descriptions are avail- activities and engagement outcomes. Laird, Smallwood, Niskode-Dossett, and
able in an issue of New Directions for Examples of evaluation work utilizing Garver (2009) offers ideas for involving
Institutional Research (ouimet & Pike, experimental designs are provided in faculty in assessment efforts specific to
2008). A copy of the piloted version of the learning community section of this student engagement.
the survey is available at http://www. article. Because experimental designs
aascu.org/accountability/survey/?u=1. are rarely possible, we also recommend REFERENcES
Additional information regarding the the use of quasi-experimental designs
above-mentioned instruments as well that adequately control for possible
as an inventory of other potentially confounding variables (e.g., matching Astin, A. W. (1984). Student involvement: A
developmental theory for higher education.
relevant surveys and tools used to as- groups). Furthermore, in cases where
Journal of College Student Development,
sess student engagement outcomes is the program is already in place or the
25(4), 297–308.
posted on the Association for Institu- independent variable (i.e., program)
tional Research (AIR) website at http:// cannot be manipulated, we suggest Astin, A. W. (1993). What matters in college:
Four critical years revisited. San Francisco:
applications.airweb.org/surveys/De- the use of nonexperimental designs
Jossey-Bass.
fault.aspx. that adequately control for students’
background characteristics and precol- Astin, A. W. (1999). Student involvement: A
developmental theory for higher education.
cONcLUSIONS ANd lege characteristics that have been
Journal of College Student Development,
previously found to impact student
40(5), 518–529.
REcOMMENdAtIONS outcomes (see discussion by Cole,
Bank, B. J., Slavings, R. L., & Biddle, B. (1990).
Kennedy, & Ben-Avie, 2009). Finally, we
We hope that the information present- Effects of peer, faculty, and parental influ-
suggest that institutional researchers
ed in this article is useful to institutional ence on students’ persistence. Sociology of
consider using qualitative methods
researchers involved with program Education, 63, 208–225.
to answer “how” and “why” questions
planning, assessment, and/or accredi- Banta, T. W., Pike, G. R., & Hansen, M. J.
specific to program assessment.
tation efforts; we offer the following (2009). The use of engagement data in
conclusions and recommendations. accreditation, planning and assessment.
Next, we encourage institutional re-
First, we recommend that student New Directions for Institutional Research, 141,
engagement programs be clearly con- searchers to actively seek out collabo- 21–34.
rations with faculty and student and
nected to the institution’s core mission Bloom, D., & Sommo, C. (2005). Building
academic affairs programs/offices. Stu-
(Culp, 2007) and that they be grounded learning communities: Early results from the
SPRING 2013 VoLuME | PAGE 29
Opening Doors demonstration at Kingsbor- Derby, D. C., & Smith, T. (2004). An orienta- King, P., & Howard-Hamilton, M. (2000). us-
ough Community College. MDRC, New York. tion course and community college reten- ing student development theory to inform
tion. Community College of Research and institutional research. New Directions for
Bordes, V., & Arredondo, P. (2005). Mentor-
Practice, 28, 763–773. Institutional Research, 108, 19–36.
ing and 1st year Latino/a college students.
Journal of Hispanic Higher Education, 4(2), Derby, D. C., & Watson, L. W. (2006). African- Kinzie, J. & Pennipede, B. S. (2009). Convert-
114–133. American retention within a community ing engagement results into action. New Di-
college: Differences in orientation course rections for Institutional Research, 141, 83–96.
Brower, A. M., & Dettinger, K. M. (1998).
enrollment. Journal of College Student Reten-
What is a learning community? Toward a Kuh, G. D. (2004). Forging a new direction:
tion, 8(3), 377–390.
comprehensive model. About Campus, Nov– How uTEP created its own brand of excel-
Dec, 15–21. Donnangelo, F. P., & Santa Rita, E. D. (1982). lence. About campus, 9(5), 9–15.
The effects of two collage orientation
Campbell, T. A., & Campbell, D. E. (1997). Kuh, G. D., Cruce, T. M., Shoup, R., Kinzie, J., &
courses upon the academic performance
Faculty/student mentor programs: Effects Gonyea, R. M. (2008). unmasking the effects
and retention of entering freshmen. Bronx
on academic performance and retention. of student engagement on first-year college
Community College, New York. (ERIC Docu-
Research in Higher Education, 38(6), 727–742. grades and persistence. Journal of Higher
ment Reproduction Service No. ED232747).
Education, 79(5), 540–563.
Chen, P. D., Gonyea, R. M., Sarraf, S. A.,
Erikson, E. (1968). Identity: Youth and crisis.
BrckaLorenz, A., Korkmaz, A., Lambert, A. Kuh, G. D., Kinzie, J., Buckley, J., Bridges, B.,
New York: W. W. Norton.
D., . . . & Williams, J. M. (2009). Analyzing and & Hayek, J. C. (2007). Piecing together the
interpreting NSSE data. New Directions for Freeman, K. (1999). No services needed?: student success puzzle: Research, proposi-
Institutional Research, 141, 35–54. The case for mentoring high-achieving tions, and recommendations. ASHE Higher
African American students. Peabody Journal Education Report, 32(5).
Chickering, A. (1969). Education and identity.
of Education, 74(2), 15–26.
San Francisco: Jossey-Bass. Laden, B. V. (1999). Socializing and mentor-
Girves, J. E., Zepeda, Y., & Gwathmey, J. K. ing college students of color: The Puente
Chickering, A., & Reisser, L. (1993). Educa-
(2005). Mentoring in a post-affirmative Project as an exemplary celebratory social-
tion and identity (2nd ed.). San Francisco:
action world. Journal of Social Issues, 61(3), ization model. Peabody Journal of Education,
Jossey-Bass.
449–479. 74(2), 55–74.
Cole, J. S., Kennedy, M., & Ben-Avie, M.
Glass, J. C. Jr., & Garett, M. S. (1995). Student LaNasa, S. M., olson, E., & Alleman, N. (2007).
(2009). The role of precollege data in assess-
participation in a college orientation The impact of on-campus student growth
ing and understanding student engage-
course, retention, and grade point average. on first-year student engagement and
ment in college. New Directions for Institu-
Community College Journal of Research and success. Research in Higher Education, 48(8),
tional Research, 141, 55–69.
Practice, 19(2), 117–132. 941–966.
Cook, L. P. (1996). A description of new
Grunder, P. G., & Hellmich, D. M. (1996). Mangold, W. D. (2003). Who goes, who stays:
student orientation programs at two-year
Academic persistence and achievement of An assessment of the effect of a freshman
colleges in the united States. (Dissertation
remedial students in a community college’s mentoring and unit registration program
Abstracts International uMI No. 9619361).
college success program. Community Col- on college persistence. Journal of College
Crisp, G. (2009). Conceptualization and lege Review, 24(2), 21–33. Student Retention, 4(2), 95–122.
initial validation of the College Student
Harper, S. R., & Quaye, S. J. (2009). Student National Resource Center for the First-Year
Mentoring Scale (CSMS). Journal of College
engagement in higher education: Theoreti- Experience and Students in Transition.
Student Development, 50(2), 177–194.
cal perspectives and practical approaches (2008). National Resource Center Survey.
Crisp, G. & Cruz, I. (2009). Mentoring college for diverse populations. New York: Rout- Retrieved from http://www.sc.edu/fye/re-
students: A critical review of the literature ledge. search/surveyfindings/
between 1990 and 2007. Research in Higher
Hollins, T. N. (2009). Examining the impact Nelson Laird, T. F., Smallwood, R., Niskode-
Education, 50(6), 525–545.
of a comprehensive approach to student Dossett, A. S., & Garver, A. K. (2009). Effec-
Culp, M. M. (2007). Doing more of what mat- orientation. Journal of the Virginia Commu- tively involving faculty in the assessment
ters: The key to student success. New Direc- nity Colleges, 15–27. of student engagement. New Directions for
tions for Community Colleges, 131, 77–87. Institutional Research, 141, 71–81.
Johnson, C. S. (1989). Mentoring programs.
Dale, P. A., & Drake, T. M. (2007). Connecting In M. L. upcraft & J. Gardner (Eds.), The fresh- Nora, A., & Crisp, G. (2007). Mentoring stu-
academic and student affairs to enhance man year experience: Helping students survive dents: Conceptualizing and validating the
student learning and success. New Direc- and succeed in college (pp. 118–128). San multi-dimensions of a support system. Jour-
tions for Community Colleges, 131, 51–64. Francisco: Jossey-Bass. nal of College Student Retention: Research,
Derby, D. C. (2007). Predicting degree com- Keeling, R. P., Wall, A. F., Underhile, R., & Theory and Practice, 9(3), 337–356.
pletion: Examining the interaction between Dungy, G. J. (2008). Assessment reconsidered: o’Gara, L. Karp, M. M., & Hughes, K. L. (2009).
orientation course participation and ethnic Institutional effectiveness for student success. Student success courses in the community
background. Community College Journal of International Center for Student Success college: An exploratory study of student
Research and Practice, 31(11), 883–894. and Institutional Accountability, uSA. perspectives. Community College Review,
PAGE 30 | SPRING 2013 VoLuME