ERIC EJ1085276: Accelerating Educational Innovation in the MPH Degree Program: What Is the Role of Peer Review of Teaching?

2015·0.47 MB·English

by  ERIC

Journal of Health Education Teaching, 2015; 6(1): 43-56 Copyright: www.jhetonline.com
All rights reserved
Accelerating Educational Innovation in the MPH Degree
Program: What is the Role of Peer Review of Teaching?

Taryn Vian, SM, PhD
Associate Professor
Department of Global Health
Boston University School of Public Health
801 Massachusetts Avenue
Boston, Massachusetts, 02118,
Telephone: [phone]
Email: [email protected]

Paul G. Ashigbie, BPharm, MPH, DrPH(c)
Research Fellow
Department of Global Health
Boston University School of Public Health
801 Massachusetts Avenue
Boston, Massachusetts, 02118
Telephone: [phone]
Email: [email protected]

ABSTRACT
Purpose: The environment of public health practice is rapidly changing, creating the need to adapt graduate
education and accelerate educational innovation. Formative peer review is a strategy designed to promote
critical reflection on teaching and to develop faculty as teachers. Through case study methods, we explore how
peer review of teaching may catalyze reflective practice and contribute to the redesign of public health
education. Methods: We conducted a detailed contextual analysis of a peer review of teaching program
implemented from 2011-2014 in a global health department with approximately 200 students and 26 teaching
faculty. The case study used multiple data sources including peer review feedback reports, a survey of
participating instructors and reviewers, and administrative data. Results: Faculty had favorable attitudes toward
peer review, especially the opportunity to learn by observing others teach. Peer review reports confirmed
existing good teaching practices and suggested ways to improve student engagement and develop teachers in
ways which are aligned with MPH program redesign goals. Both peer reviewers and instructors reported
benefiting from the process. Conclusions: Formative peer review may help to spread innovative teaching
practices among faculty by providing opportunities for reflection, increasing motivation and confidence to adopt
pedagogical changes, and strengthening teaching community. Increased knowledge of other teachers’ courses
and social ties created through peer review can be resources in promoting MPH curriculum integration and
collaboration across disciplines. Recommendations: Schools of public health should consider peer review of
teaching as a pathway to promote greater student engagement and practice-based learning. In planning for peer
review of teaching programs, leaders need to find ways to address time-related concerns and align the peer
review process with other efforts to promote program redesign goals and strengthen teaching community.

Accelerating educational innovation in the MPH degree program Page 43
Journal of Health Education Teaching, 2015; 6(1): 43-56 Copyright: www.jhetonline.com
All rights reserved
INTRODUCTION Beh, 2014). PRT is defined as “informed
colleague judgment about teaching for either
The curriculum for the Masters of Public fostering improvement or making personnel
Health (MPH) degree program is undergoing decisions” (Chism, 2007). Formative PRT
significant changes. In 2011, a task force of the programs are designed to develop faculty
Association of Schools and Programs of Public members’ self-awareness and provide teachers
Health (ASPPH) studied the role of public health with information and an opportunity for critical
education for the 21st century (Petersen & Weist, reflection to improve their teaching practice,
2014). The task force emphasized the need to while evaluative PRT programs seek to compare
adapt education to a rapidly changing teaching to standards to make judgments
environment in public health practice and to related to hiring, promotion, tenure, or salary
accelerate educational innovation. The skills review (Chism, 2007; Thomas et al., 2014). PRT
which students of public health need are can help teachers reflect on weaknesses and
interdisciplinary, interprofessional, and systems build on teaching strengths, select teaching
oriented (Frenk et al., 2010; MPH Expert Panel methods that can best meet the goals for a
of the Framing the Future Task Force, 2014; course, develop more practice-based exercises
Petersen & Weist, 2014). They reflect work-force and assessments, and integrate technology into
competencies from the core disciplines of the classroom (Carter, 2010; Kohut, Burnap, &
biostatistics, epidemiology, health policy and Yon, 2010; Thomas et al., 2014; Yon, Burnap, &
management, environmental health sciences, Kohut, 2010). Faculty peers may be better
and social and behavioral sciences in addition to qualified than students to assess the
cross-cutting skills in communication, change appropriateness of course objectives, quality of
management, informatics, political and cultural course content, whether important points are
sensitivity, and leadership (Bernstein, Jonson, & being emphasized, and the aptness of examples
Smith, 2000; McKee & Tew, 2013). The new used in class (Courneya, Pratt, & Collins, 2008;
focus on applied learning and integration was d'Eon, Overgaard, & Harding, 2000; Nelson,
expected to require major changes in teaching 1998). The Harvard T.H. Chan School of Public
(Frenk, Hunter, & Lapp, 2015). Health includes peer review of teaching as a
component of their new educational agenda,
Research supports the assumption that the suggesting that it can be a vehicle to adapt the
quality of teaching affects student learning, and values of collaborative, team-based research to
continuous development of faculty as teachers is the teaching enterprise (Frenk et al., 2015). Yet,
critically important to successful education little is known about the application of peer
redesign (Bernstein et al., 2000; Clayson, 2009; review of teaching in the context of MPH
McKee & Tew, 2013; MPH Expert Panel of the education. Implementation of PRT programs in
Framing the Future Task Force, 2014). To other disciplines has proven challenging
achieve the goals proposed by the ASPPH task because the goals and format must be adapted
force, course design and teaching must change to institutional context, including the
in ways that promote student engagement and department’s mission, values, student
practice-based learning to build competencies in demographics, resources, and faculty disciplines
public health, with more time devoted to (Chism, 2007; Quinlan & Akerlind, 2000). Other
applying skills such as data analysis or critical obstacles or barriers to successful program
reasoning, and practicing communications and operation include fear on the part of the
leadership (Brame, 2013; Frenk et al., 2015; instructor being reviewed, uncertainty about
Fulton, 2013; Herreid & Schiller, 2013). Yet to what to review, resistance to innovation that
date, most of the literature on redesigning the involves participants doing more work, and
MPH has focused on curriculum choices (what implementation factors such as how reviewers
to teach) rather than teaching methods and are chosen, power differentials, and the amount
professional development of teaching faculty of guidance or flexibility built into the process
(how to teach). (Costello, Pateman, Pusey, & Longshaw, 2001;
Courneya et al., 2008; Frenk et al., 2010; Kell &
Peer review of teaching (PRT) is one method Annetts, 2012; Thomas et al., 2014). We
of promoting professional development of examine issues encountered in using PRT to
teachers, encouraging reflective practice, and help faculty in one department reflect on
motivating faculty to teach from a different teaching practice, and explore what role PRT
perspective (Thomas, Chie, Abraham, Raj, & might play in accelerating educational innovation
Accelerating educational innovation in the MPH degree program Page 44
Journal of Health Education Teaching, 2015; 6(1): 43-56 Copyright: www.jhetonline.com
All rights reserved
in MPH programs across departments and application of public health skills with the
schools of public health. instructor as mentor and coach. Five task forces
were created at the school level to promote the
PURPOSE programmatic changes: core curriculum,
concentrations, practicum, culminating
The purpose of this study is to explore how a experience, and teaching methods and
global health department at a major school of assessment. The teaching methods and
public health implemented PRT in conjunction assessment task force called for competency-
with MPH curriculum redesign, in an effort to based learning to promote skills which would
improve teaching which would reinforce improve students’ job-readiness, and highlighted
competency-based learning. The study the need to develop faculty and provide them
documented the views and perceptions of with support for pedagogical changes.
faculty about the PRT process, the types of
feedback and observations gleaned through The primary objective of GH’s peer review of
PRT, and the challenges in carrying out the teaching policy was to improve the quality of
program. The insight generated from this study teaching. PRT was intended to affirm the
may help envision the role of PRT in promoting importance of teaching, encourage innovation
innovation in MPH education, and inform the and improvement in teaching methods, and
design and implementation of PRT across encourage sharing of teaching techniques. It
departments and in other public health was seen as a form of collaborative analysis that
institutions. would help individual faculty to develop as
teachers while reinforcing their commitment to
Background on the Peer Review of Teaching
teaching. Excellence in teaching would in turn
Policy and Process
improve the competencies of MPH graduates,
The Department of Global Health (GH) especially in light of the move toward
admits 90-110 new MPH students per year. Full- competency-based education. A second
time students generally can complete objective of PRT was to enable GH faculty to
coursework in three semesters including a understand the courses taught by other faculty
practicum experience. The Department adopted members so that faculty could become better
a PRT policy in December 2010. While the advisors, more successfully matching students
number of faculty in the department varies over with course offerings that fit their needs and
time, the department generally has 24-26 interests. Finally, the department hoped that
teaching faculty in a given year, and 6 research PRT would help to create a stronger
faculty. Teaching faculty are expected to teach professional community in order to resist the
at least 8 credits as well as doing research, centrifugal tendencies which are inevitable as
while research faculty are primarily engaged in faculty members pursue individual career goals
research and are not required to teach, though
and funding opportunities (d'Eon et al., 2000).
they may offer to teach a course in exchange for
salary coverage. As the School of Public Health PRT process
does not have tenure, all faculty are non- The process was designed so that teams of
tenured. two faculty members observe each course
assigned for review. In fall and spring
Implementation of peer review of teaching semesters, the Chair or his/her designee
coincided with a school-wide redesign of the develops a list of courses to be reviewed. Every
MPH curriculum started in 2011. The goal of this course is not reviewed every semester, and
redesign was to provide graduates with the reviews are not conducted in summer to ease
knowledge and skill sets needed for the burden on faculty. Typically a course will be
employment, and to assure that the program reviewed every 2-3 times it is offered. The Chair
was adapted to the changing context in which or designee assigns faculty teams to review the
public health is practiced. As many students courses. Factors considered in selecting
were coming straight from undergraduate reviewers are the background and strengths of
studies without much work experience, the reviewers and how they may be helpful to the
redesign aimed to increase applied learning instructor, and the reviewers’ ability to benefit
experiences to simulate work settings. Ideally from the review by gaining ideas for their own
students would be spending less class time courses. Over time, review teams are exposed
listening to lectures and more time practicing the to many different courses. Given the inter-
Accelerating educational innovation in the MPH degree program Page 45
Journal of Health Education Teaching, 2015; 6(1): 43-56 Copyright: www.jhetonline.com
All rights reserved
disciplinary nature of global health, reviewers faculty. The time period examined was January
often do not have content expertise in the topic 2011 to May 2014 (7 semesters). A combination
of the course being reviewed. This is not seen of data collection methods is well suited to
as a problem, as the goal of PRT is to improve understand the relationship between
and share teaching methods, and to provide organizational behavior and its specific context
opportunities for collaborative reflection on (Hartley, 1994). Administrative data sources
teaching. Peer review teams are encouraged to included the school’s registrar office records
meet with the instructor before observing the (courses offered by the department within this
class. During this meeting they discuss specific period, and the instructors who taught the
aspects of the course the instructor may want to courses) and records from the department
focus on, and review the course goals, syllabus, curriculum coordinator’s office (courses
and teaching and assessment methods. This is assigned for review each semester, assigned
meant to provide context for the observation and reviewers, number of reviews which actually
to help the reviewers see how the session to be took place, and availability of review memos). In
observed fits into the instructor’s overall plan. addition, we used the peer review memos to
Before conducting the observation, reviewers determine the number of sessions observed for
may also examine other materials related to the each course, and to help determine which
course including the course web site, e-portfolio faculty members assigned to the review actually
site, or past student course ratings. participated. We also reviewed the content of all
written peer review memos submitted during the
Review teams schedule a time to observe the study time period to elicit themes.
class. The team is expected to attend one
(generally 3-hour) class session and to review To complement the administrative data and
the materials for the class. Given travel peer review memo information, we conducted an
schedules for GH faculty, sometimes reviewers anonymous survey of GH faculty using
split up and review different sessions of the Qualtrics® online software. The survey was sent
course. Following the review, the review team is to 26 full-time teaching faculty with regular
expected to prepare a brief memorandum appointments. The survey was sent in January
summarizing impressions and suggestions. This 2014, and the recall period was 2013. Faculty
document is shared with the instructor, and the answered questions about the time spent in the
reviewers and instructor meet to amplify or peer review process, knowledge, and attitudes
clarify the comments. The memorandum is then or opinions about the program. The knowledge,
submitted to the faculty chair who reviews to attitude, and opinion questions were evaluated
identify and prioritize resources to support on a 5 point Likert-like scale (1=strongly agree, 5
changes in the course or skill development for = strongly disagree). Knowledge of the policy
the instructor. The review does not become part was probed with two questions: “The department
of the faculty member’s personnel file, nor is it uses peer review feedback in the faculty
used for annual performance review. Peer performance review and planning process,” and
review tends to be more effective when the “The department uses peer review feedback in
process is flexible and owned by the the curriculum revision process.” We asked
participating faculty (Thomas et al., 2014). To three questions about usefulness of peer review,
encourage ownership, guide questions and an including “PRT helps me learn new methods of
optional rubric are provided as possible tools, course design, teaching, or assessment,” “PRT
but the review teams can decide which tools increases my knowledge of course offerings,”
they want to use. The optional guide questions and “PRT helps me to get to know other
and rubric focus on aspects of course instructors better.” Finally, we asked opinion
organization, clarity of content, interaction in the questions to probe whether the reviewer thought
classroom, lecture style, discussion the instructor had found the feedback helpful,
management, use of media, and exercises or and whether the instructor found the reviewer’s
projects to promote applied learning. feedback helpful. We did not ask any
demographic information in order to protect
METHODS confidentiality.
We applied case study research methods to
provide a rich description of how the PRT
program is operated and was perceived by
Accelerating educational innovation in the MPH degree program Page 46
Journal of Health Education Teaching, 2015; 6(1): 43-56 Copyright: www.jhetonline.com
All rights reserved
Analysis RESULTS
Administrative data were analyzed in an Administrative Data
Excel spreadsheet. We calculated total and Twenty-six faculty participated in the PRT
average per semester for: courses offered; program as a reviewer for at least one semester
courses assigned for review; reviews which with 11 faculty participating as reviewers 3 or
actually took place; reviews for which a written more times between 2011 and 2014. Each
memo was available. We also calculated the faculty member reviewed 2.5 courses on
average number of reviews per course; average average. The GH department offered 19 courses
number of reviews per teaching faculty member; and scheduled 9 peer reviews per semester
average number of sessions observed per (47%) on average.
review; and proportion of faculty assigned to
review who actually participated. We Over the 7 semesters, 28 unique courses
summarized average length of memos, and the were offered and all were scheduled for at least
proportion of memos which used an evaluation one review. Core courses with sections taught
rubric versus narrative style. by different instructors were often scheduled for
review multiple times. Of the 62 reviews
The content of peer review memos was scheduled, written memos were available for
analyzed using a modified grounded theory only 37 (60%), and verbal feedback was given
approach (Corbin & Strauss, 2008). Our for an additional two reviews (3%). It is assumed
analytical strategy involved constant that the remaining 23 scheduled reviews (37%)
comparison, i.e. looking at observations and probably did not take place due to scheduling
recommendations contained in each memo, and problems or travel. This means that of the 28
comparing it with observations and comments in courses scheduled for review, 4 (14%) were
other memos for similarities and differences. never reviewed, 13 (46%) were reviewed once,
Segments of text which were found to be 9 (32%) were reviewed twice, and 2 (7%) were
conceptually similar were grouped together reviewed three or more times. On average each
under higher-level descriptive concepts (Corbin unique course was actually reviewed 1.3 times
& Strauss, 2008). The lead author, who is an with written results.
experienced researcher trained in qualitative
methods, read each memo and entered sections Some courses scheduled for team review
of text into an Excel spreadsheet. These were actually reviewed by only one faculty
segments of text were then analyzed and member. On average, 85% of faculty who were
assigned codes to indicate different descriptive assigned to a reviewed course contributed to a
themes (Miles & Huberman, 1994). review memo, and review teams observed an
average of 1.3 sessions per course. Review
Survey data were analyzed in Excel. Data memos were generally 1-3 pages long. Eight
were stratified by reviewers and instructors memos (22%) incorporated an optional rubric
where necessary, and described using which rated class organization, lecture style,
proportions and averages. exercises, student engagement, etc. as
“excellent”, “good”, or “needs improvement,”
Institutional Review Board Approval
while the remaining review memos used
The study was reviewed by the university’s narrative style. All review memos included a
Institutional Review Board and found to be summary of observations and suggestions.
exempt because it does not meet the definition
of human subject research. Review Memo Content
Reviewers provided comments on topics
Informed Consent including the syllabus, choice of teaching
The survey used for the study was methods, classroom management, lecture skills,
anonymous and responding to the survey was and discussion management. Feedback
optional. We used de-identified data for this depended on whether the instructor had specific
study and therefore we could not obtain goals which he or she wanted the reviewers to
informed consent of study participants. consider; for example, one instructor asked for
reviewers’ recommendations for how students
could become more engaged during class
sessions while another instructor was interested

Accelerating educational innovation in the MPH degree program Page 47
Journal of Health Education Teaching, 2015; 6(1): 43-56 Copyright: www.jhetonline.com
All rights reserved
in better ways to evaluate group projects. of the overall course goals. They also suggested
Feedback on content was offered if the reviewer additional readings or perspectives connecting
had related expertise, such as an infectious other disciplines: for example, a professor with a
disease doctor reviewing a course on humanities degree, after reviewing a course
controversies in global disease control and session on vaccine refusal taught by a medical
eradication. Recurrent themes in the peer review doctor, ruminated that John Stuart Mill would
feedback included: active learning, making ultimately come down on the side of compulsory
connections between course objectives and vaccination without exclusions because not
activities, and teaching techniques. vaccinating compromises the health and liberty
of others. The reviewer supplied a lengthy quote
Active learning from Mill’s book On Liberty to support this
Reviewers gave praise and constructive argument.
criticism to instructors to enhance active learning
and student engagement, as shown in these Frequently, reviewers observed ways in
quotes: “The session was a great example of the which the instructor could make explicit the
"inside out" organization of teaching where the connections between sessions, summarize key
students are actively engaged in learning. Work points, or anticipate what is coming next as in
outside the session (watching video lectures, these quotes: “It might be good to introduce the
preparing their own group's consulting readings for the day by saying how or why they
assignment, reading another group's report) were chosen, or what the students are meant to
made it possible for this session to be so active. get out of them. Are they linked in any way; an
During class, groups of students evaluated overarching theme?” “I don't usually subscribe
another group's report on a drug management to the philosophy that one needs to hit students
issue, and prepared and gave a 10-minute over the head with the course objectives, but it
presentation with the findings of their might have helped [if you would] place the
evaluation.” “[In this session,] student groups discussion in the overall context of the course
had designed active-learning strategies to and give objectives for the lecture.”
convey nutritional messages. Each group had
just 5 minutes. One group did a charades game, Teaching styles and techniques
another used a polling software to survey class Reviewers observed ways in which the
participants through text messaging, a third instructor set up the classroom to enable active
group had written a song with nutrition discussions, intervened to prevent domination
messages in the lyrics…the students seemed by vocal students, and used techniques such as
happy and engaged.” Active learning sometimes randomly picking students to lead discussion as
involved individual activities such as students a way to motivate students to be more prepared
filling out a worksheet or team activities and therefore more able to actively participate.
(presentations, group projects) inside or outside Reviewers mentioned ways to widen
the classroom. One reviewer acknowledged a participation, suggesting techniques such as
faculty member’s innovation in incorporating coaching student discussion leaders and
tweeting into the classroom while also preparing discussion questions in advance.
questioning how to evaluate the impact of
tweeting on educational outcomes. Other Reviewers gave feedback on systematic
reviewers suggested strategies such as pair- organization of lectures, confidence and
shares, “clickers” (audience response system), command of material, the instructor’s voice and
“minute papers” (a classroom assessment enthusiasm, and the preparation of slides.
technique to get rapid feedback on a main idea Reviewers liked when instructors integrated
introduced in the session), and structured examples and experience into lectures, as
shown in these quotes: “One of the most
discussions to further engage students.
effective elements was when the instructor
Connections and context turned the lecture from conceptual to the specific
Another theme of reviewer comments was by integrating data from recent scientific
helping learners see connections and publications which were thematically linked to
understand the context in which course the student projects.” “The instructor spoke from
concepts can be applied in professional practice. experience, probably the greatest strength of the
Reviewers praised instructors for using class. It really addresses, ‘what is it like to work
techniques which put the session in the context in the field?’” Many reviewers made suggestions
Accelerating educational innovation in the MPH degree program Page 48
Journal of Health Education Teaching, 2015; 6(1): 43-56 Copyright: www.jhetonline.com
All rights reserved
about how to design better presentations, use About three-quarters of instructors believed
questions more effectively, and wait an that peer review was important for faculty
adequate amount of time for students to answer. development and 44% of instructors reported
they had already made changes in course
Other themes design or teaching based on peer review
Reviewers often commented on themes ripe feedback. Faculty shared their positive
for deeper discussion among faculty, such as perceptions, as in these quotes: “I think this
whether instructors should ‘cold call’ on quiet process helps knit the department together.” “I
students, the appropriate role of teaching highly value peer review. I’ve made changes
assistants in lecturing, whether to allow re- based on comments from previous reviews.” “I
submissions of work for a higher grade, how to enjoyed having my colleagues in the classroom.
assure individual accountability in group I always ask them to participate and that adds a
projects, and what to do about students who are nice new voice to the session.” Other faculty
web surfing or emailing in class. While reviewers voiced doubts about the usefulness: “[The
acknowledged the complexity of these issues usefulness] depends on the feedback provided.
and there is likely no one right way, they would Teaching styles vary and not all
often mention their own preferences as these recommendations are helpful.” “The reviewer
reviewers did on the topic of grading: “I wonder if came—but so far has not responded to request
students will be happy doing so much [group] for feedback.” Fifty-three percent of instructors
work and getting few individual grades, but that thought peer review of their course was
may just be my prejudice against group projects. worthwhile or very worthwhile.
I didn't like them as a student and did not find
them useful when I tried them as a teacher.” “If Attitudes toward peer review suggest that the
we are really interested in the intellectual growth reviewers gain as much if not more from the
of our students …then we would want to peer review process as the instructor whose
encourage them to redo work, continue to course is being reviewed (Table 1). Almost
practice, and if that produces a better product, three-quarters of reviewers said that conducting
reward them accordingly.” peer review helped them to learn new design,
teaching and assessment methods. Reviewers
Survey Data also said they gained knowledge of the range of
We received responses from 19 of 26 GH course offerings in their department and got to
faculty surveyed (73% response rate). Fifteen know other faculty better through the peer
(79%) of the respondents had participated in review process, as these quotes suggest: “I
PRT in 2013. Where reviews were not enjoy doing the reviews. I especially like meeting
completed, faculty explained reasons including with the instructors ahead of time and finding out
scheduling problems (being asked to review a what they like or are worried about in their
course that met when the reviewer was courses. I like to see how other instructors
teaching), international travel, and “running out structure their courses.” “I value the opportunity
of time” or forgetting. Reviewers reported to learn from my colleagues and offer feedback.
spending a median of 4 hours while instructors While it can be hard to fit in our busy schedules,
said they spent 0.5 hours on the peer review it’s always worth it.”
(instructors did not count the time being
observed, as they would have been teaching Some respondents didn’t know how their
anyway). feedback had been perceived or suspected that
the feedback they gave to instructors was not
Most of the faculty surveyed said that when welcome. In addition, reviewers were not aware
they had reviewed a course, they had met with of how the department was using the peer
the instructor, reviewed the syllabus, observed a review feedback: only 27% believed the
session, and shared written feedback with the department uses the peer review memos in the
instructor afterwards (Figure 1). Less common curriculum revision process (a stated goal of the
peer review activities were talking to students to peer review policy) while the same percent
get their feedback and reviewing the course web believed that the department used the peer
site or past course evaluations which are review feedback in the faculty performance
available to all faculty and students online. In review and planning process (the policy explicitly
one case, a pair of reviewers had conducted an states that the peer review process is not
online survey to get student feedback. evaluative).
Accelerating educational innovation in the MPH degree program Page 49
Journal of Health Education Teaching, 2015; 6(1): 43-56 Copyright: www.jhetonline.com
All rights reserved
CONCLUSIONS As has been noted in other studies, the PRT
process provided opportunities for faculty
Through this case study, we documented and development not only for the instructor but for
analyzed the implementation of a PRT program the reviewer as well (Kohut et al., 2010;
meant to further MPH curriculum re-design goals Wilkerson & Irby, 1998). In one study, reviewers
to emphasize competency-based learning. We perceived the process more useful for their own
found that the peer review process was teaching than did faculty being reviewed;
generally, though not uniformly, well received by however, the PRT process in that study was
faculty. The proportion of instructors who said being used for evaluative purposes to make
they had already made changes in course tenure and promotion decisions, a different
design or teaching as a result of PRT (44%) was context (Kohut et al., 2010). Observing someone
similar to that found in other studies; for else is a way to self-critique and learn new
example, about one-third of participants in a practice (Chism, 2007), and peer review helps
PRT program at the University of Nebraska faculty interact and learn from one another,
reported making significant changes to some providing collegial support (d'Eon et al., 2000).
components of teaching as a result of the The GH PRT program shows that this was an
program (Bernstein et al., 2000). The content of important outcome: 78% of reviewers and 72%
review memos was consistent with the stated of instructors noted that the peer review process
goals of the policy, i.e. to encourage innovation helped them get to know other faculty as
and improvement in active teaching and learning colleagues. The increased knowledge of other
methods, help faculty develop as teachers and teachers’ courses and the social ties created
advisors, and to create a strong community. through PRT are resources which can be used
in developing MPH curriculum integration and
One theme emphasized by reviewers was making connections between coursework and
the need to include more practice-based public health practice. Over 80% of reviewers
examples in lectures. This suggests that public noted that PRT helped them understand the
health faculty recognize that practice experience Department’s courses, which helps them
is highly valued by graduate students, most of become better advisors to students. In addition,
whom are being trained for practice rather than the GH department has added “teaching
research careers or academia. Integrating innovation” as a standing agenda item at
examples and stories from applied practice monthly faculty meetings. This additional forum
helps students to see the relevance and for communicating about teaching aligns with
possible applications of theoretical concepts. the PRT goals and further reinforces the
While not, in itself, active learning, this may help teaching community. It has helped faculty
make the material more stimulating and identify in-house experts for particular
interesting which is a quality of good teaching innovations, and has helped to emphasize the
(Thomas et al., 2014). Reviewers’ specific
value placed on educational innovation.
criticism of lecture styles also seemed intended
to help instructors increase engagement with In addition to providing feedback to individual
students at their level of understanding teachers, the PRT process highlighted more
(Wilkerson & Irby, 1998). general issues or questions which leaders may
want to address in the context of school-wide
A large number of comments from reviewers MPH redesign. Topics included the value of
related to how instructors could strengthen the competency-based grading (i.e. allowing
connection between the content of the course students to re-submit work until they achieve
and the situations in which students might be competency, and only grading the final product),
expected to use or apply the content as public how to engage introverted or shy students and
health professionals. In active-learning courses, students from diverse cultural backgrounds, and
this is a challenge because the key points are how to minimize technology distraction while
emerging in real-time from the class. Faculty also meeting students where they are, i.e.
members who are innovating with active incorporating social media and other new
learning may not yet be fully competent at aiding technology into the classroom. It would be
students’ sense-making when the activity helpful to provide more opportunities for the
concludes (Ketelaar, Beijaard, den Brok, & faculty as a whole to discuss such observations
Boshuizen, 2013). PRT provides an opportunity and issues. PRT helps to promote active,
to coach faculty in this difficult skill. practice-based learning because it implicitly
Accelerating educational innovation in the MPH degree program Page 50
Journal of Health Education Teaching, 2015; 6(1): 43-56 Copyright: www.jhetonline.com
All rights reserved
recognizes that teaching is not merely a purpose of PRT and taking action to reduce
technical enterprise, but a social practice (d'Eon power imbalances in pairing of reviewers and
et al., 2000). By allowing the reviewer and the instructors is even more important to ensure
instructor to define the process, valuing the acceptance of the process.
process of reviewing as well as receiving
feedback, and providing multiple opportunities to Another factor associated with successful
engage in reflection, PRT helps facilitate the PRT is whether faculty have established norms
adoption of new social practices around of shared inquiry and prior experience working in
learning, such as project-based learning. teams (Thomas et al., 2014). Research has
shown that in disciplines where faculty naturally
Limitations engage in individual research with sole
This study has several limitations. Although authorship, PRT may be less successful
we had a high response rate to the online (Thomas et al., 2014). Our case study was
survey, not all faculty responded and we cannot conducted in a department which was already
know how the perceptions and experience of used to working in interdisciplinary teams for
non-respondents might have differed. The research projects, which may have made it
thematic analysis drew on written peer review easier to apply a team approach to reflect on
memos, which were only provided for 60% of teaching. In situations where team work among
scheduled reviews. Finally, our analysis of a faculty is not the norm, leaders may need to
single case study may not reflect the experience spend more time in forming teams and
of other schools or departments. However, the developing teamwork and reflective practice
framework that resulted may help guide the skills to ensure PRT works well.
design of PRT programs in other settings, as
discussed below. Policy Design
Leadership is essential to motivate faculty
RECOMMENDATIONS especially in public health schools with soft
funding where faculty are juggling the need to
Based on our review of PRT literature and cover their salary by generating research dollars
this case study, we have created a framework and may see teaching as less essential for
which captures our experience and may guide survival. Time and burden on faculty are
other schools of public health considering consistently raised as issues affecting PRT
adopting PRT as part of the redesign of MPH implementation success (Kell & Annetts, 2012;
programs (Table 2). The framework illustrates Siddiqui, Jonas-Dwyer, & Carr, 2007; Thomas et
pre-disposing factors, policy design, and al., 2014). While the model of peer interaction
program implementation features, and how and consultation adopted in this case study was
together they may lead to the intended not very intensive--requiring about a half-day of
outcomes of PRT. each participating faculty member’s time per
semester and minimal administrative support--
Pre-disposing Factors
the program experienced compliance problems:
Positive perceptions of PRT are correlated
40% of review teams either did not submit
with collegiality and good relations among
written memos or did not conduct the assigned
reviewers and observers (Byrne, Brown, &
review, and several comments from faculty
Challen, 2010). Power dynamics and fear can
mentioned the time burden. Leadership is
have an important influence on PRT
needed to signal that committing time to the
implementation and outcomes (Byrne et al.,
process is valued and rewarded.
2010; Kell & Annetts, 2012; Kohut et al., 2010).
Our study was conducted in a non-tenure Leadership is also needed to monitor and
situation, which may have reduced power evaluate how PRT initiatives contribute to the
imbalances as we did not need to consider the outcomes of student learning and professional
social relationships among tenured and non- development. Closer management with more
tenured faculty when assigning reviews or frequent reminders and assistance in scheduling
sharing review feedback. The absence of tenure could also help increase post-observation
may have made it easier for faculty to see the written feedback from reviewers and facilitate
review process as developmental rather than as deeper, more meaningful discussion of
an evaluation tool (Peel, 2005). In a tenure observations. Leadership must also make sure
system, being very clear about the formative that the structure for PRT is focused and
Accelerating educational innovation in the MPH degree program Page 51
Journal of Health Education Teaching, 2015; 6(1): 43-56 Copyright: www.jhetonline.com
All rights reserved
periodically reinvigorated to avoid stagnation that reviewers were influenced by their own
and complacency (Byrne et al., 2010; Kell & perspectives on teaching, e.g. an aversion to
Annetts, 2012; Kohut et al., 2010). For example, group projects. In addition, our finding that some
to re-invigorate a moribund PRT program, one reviewers were uncertain whether their advice
university introduced year-long peer was useful suggests that it might have been
development team projects, where 2-5 teaching helpful to discuss criteria for good teaching to
staff worked together on a project to improve develop a common understanding. This need
practice such as effective use of tutorial support, not reduce the flexibility afforded the faculty in
planning and developing a new course, or the tools and process used for review. In fact,
adapting pedagogy to use interactive initial cycles of PRT could be used to construct
whiteboards (Byrne et al., 2010). Kell and evidence of qualities of effective teaching (Drew
Annetts (2012) found changing teams was & Klopper, 2014).
helpful so conversations didn’t stagnate and
reflection continued to be constructive. Program Implementation
A key question in PRT program
The developmental focus of the PRT implementation is who is qualified to be a peer
program must be clearly communicated in order reviewer (Thomas et al., 2014). One study
to reduce fear and foster development of determined that feedback received from subject
reflective practice (d'Eon et al., 2000). Even specialists and non-specialist assessors was
though PRT in this case study was formative equally reliable and valid (Hanson, 1993). Our
and not evaluative, there were misconceptions investigation provides further evidence that
among faculty. For example, about a third of reviewers with different disciplinary backgrounds
faculty believed the department uses the peer can still provide useful formative feedback and
review feedback in the faculty performance ideas to stimulate critical reflection. While PRT
review process, which is not the case. Ensuring can help assess relevancy of course content,
faculty are clear on the purpose of the PRT may the purpose of formative PRT is much more
increase engagement with the process. about encouraging reflective practice-related
dialogue, sharing ideas about specific pedagogic
In addition, faculty ownership of the process, innovations, and sharing constructive feedback
through opportunities to choose tools and that can lead to new understanding and
approaches and to influence the way reviews improved practice (Kell & Annetts, 2012;
are carried out, increases positive perceptions of Thomas et al., 2014). In order to achieve these
PRT and may enable a more critical and goals, the program needs to assign reviewers to
collegial examination of teaching practice maximize exposure of individual faculty to
(Thomas et al., 2014; Toth & McKey, 2010). We different styles and approaches to teaching.
fostered faculty ownership through opportunities
to choose rubrics, reporting style, specific Training of peer reviewers may also be
session or sessions to attend, and the types of beneficial. Although the GH PRT did not provide
other information to consider (e.g. student specific training to peer reviewers, some studies
evaluations, web sites). have shown that trained reviewers may be more
accurate observers and insightful of their own
A factor which has impeded PRT in some capabilities, thus establishing trust (Carolan &
studies is uncertainty about what should be Wang, 2012; Drew & Klopper, 2014; Kohut et
reviewed (Thomas et al., 2014). Some PRT al., 2010; Thomas et al., 2014). Training should
processes use external criteria, while others recognize that peer review is not just a way to
base the assessment on goals set by the identify mechanical “tips” to improve teaching,
reviewer and instructor (Siddiqui et al., 2007). but rather it is a service activity to foster
The GH department’s PRT policy did not define questioning of beliefs, assumptions, and habitual
“good teaching” criteria in advance other than practices around teaching (d'Eon et al., 2000;
suggesting that improved teaching should Thomas et al., 2014).
promote competencies that MPH graduates
need in the work place. Review teams were Program implementation requires special
allowed to decide which aspects of teaching to attention to create administrative tools to keep
assess, and how to interpret and apply the goal track of reviewer assignments and reports, in
of competency-based learning. As in other order to assure equal coverage of courses, and
studies (Courneya et al., 2008), we observed diversity and breadth of experience for
Accelerating educational innovation in the MPH degree program Page 52