AN ABSTRACT OF THE DISSERTATION OF Richard R. Aman for the degree of Doctor of ...

210 Pages·2009·0.88 MB·English

by  Rick Aman

AN ABSTRACT OF THE DISSERTATION OF

Richard R. Aman for the degree of Doctor of Philosophy in Education presented on May
29, 2009.
Title: Improving Student Satisfaction and Retention with Online Instruction through
Systematic Faculty Peer Review of Courses

Abstract approved:

George H. Copa

The purpose of the study was to determine if online courses that were faculty peer
reviewed based on five factors of quality instruction resulted in increased levels of
student satisfaction and higher rates of student retention when compared with non-
reviewed courses in a post-secondary context. The five factors of quality online
instruction used in the study were: outcomes, assessment, resource materials, student
interaction, and technology. The study utilized a quasi-experimental method with an
online questionnaire administered to 455 student participants from nine community
colleges in Maryland and Oregon. Students were enrolled in 41 online courses that were
either formally faculty peer reviewed or not. The study posed four research questions.
The first two study questions compared peer reviewed and non-peer reviewed courses in
terms of student satisfaction and student retention. The second two questions used a
regression analysis to order the contribution of the factors of quality online instruction
related to student satisfaction and retention in peer reviewed courses.
The results of the study findings were categorized into three areas: (1) student
retention findings, (2) student satisfaction findings, and (3) factors of quality instruction
influencing student satisfaction findings. First, no significance was found in the
relationship of faculty peer review to increased student retention. Second, a significant
relationship was found in the use of the faculty peer review process and increased student
satisfaction. Third, a stepwise regression analysis revealed there was significance found
in the order and contribution of each of the five factors of online quality as predictive of
increased student satisfaction in reviewed courses. This order was: (1) resource materials,
(2) assessment, (3) technology, (4) outcomes, and (5) student interactions. Findings from
the study provided the basis for four recommendations related to educational and
professional practice. These recommendations focused on: accreditation standards
associated with online learning and college self-studies, criteria for institutional funding
models in online course development, use of study questionnaire for instructional quality
assessment purposes, and quality improvement processes related to the Baldridge
Education Criteria.
© Copyright by Richard R. Aman
May 29, 2009
All Rights Reserved
Improving Student Satisfaction and Retention with Online Instruction through
Systematic Faculty Peer Review of Courses

by
Richard R. Aman

A DISSERTATION

submitted to

Oregon State University

in partial fulfillment of
the requirements for the
degree of

Doctor of Philosophy

Presented May 29, 2009
Commencement June 2010
Doctor of Philosophy dissertation of Richard R. Aman
presented on May 29, 2009.

APPROVED:

Major Professor, representing Education

Dean of the College of Education

Dean of the Graduate School

I understand that my dissertation will become part of the permanent collection of Oregon
State University libraries. My signature below authorizes release of my dissertation to
any reader upon request.

Richard R. Aman, Author
ACKNOWLEDGEMENTS

This author expresses sincere appreciation to Dr. George Copa for his extraordinary
wisdom, patience and guidance through this study. Such a task would have been
insurmountable without the efforts and encouragement of Dr. Copa. This study is
dedicated to my wife Linda, who accompanied me on the journey, being a constant
companion and source of support through the many obstacles. Finally, a special thanks to
our sons, Ben and Josh who also patiently supported me in this endeavor.
TABLE OF CONTENTS
Page
CHAPTER 1: FOCUS AND SIGNIFICANCE...................................................................1
Purpose of Study and Research Questions...........................................................................2
Definitions............................................................................................................................4
Significance of Study...........................................................................................................6
Growing Trend in Student Demand for Quality Online Instruction........................6
Credibility of Institutions Offering Online Instruction............................................7
Accrediting Organizations of Post-Secondary Institutions Seek Evidence of
Quality in Online Instruction.................................................................................10
Lack of Systematic Research on Effects of Faculty Peer Review of
Online Instruction..................................................................................................12
Summary of Focus and Significance.................................................................................13
CHAPTER 2: REVIEW OF LITERATURE.....................................................................15
Approach to Review of Literature.....................................................................................15
Organization of the Review of Literature..........................................................................17
Review of Literature..........................................................................................................17
General Factors of Quality Found in Online Instruction.......................................17
Factors Used by Accreditation Commissions in Assessment of
Online Instruction..................................................................................................22
Factors Influencing Student Satisfaction in Online Instruction.............................26
Factors Influencing Student Retention in Online Instruction................................31
Use of Faculty Peer Review of Instruction in Higher Education...........................35
Summary of Review of Literature.....................................................................................37
Factors Directly Related to the Learning Experience in Online Instruction .........38
Factors Indirectly Related to the Learning Experience in Online Instruction.......44
CHAPTER 3: DESIGN OF STUDY.................................................................................48
Philosophical Approach.....................................................................................................48
Positivism...............................................................................................................49
Postpositivism........................................................................................................50
Criteria for Truth....................................................................................................52
TABLE OF CONTENTS (Continued)
Page
Rationale for Use of Postpositivism Approach......................................................53
Researcher Background.........................................................................................54
Method...............................................................................................................................55
Quasi-experimental Design....................................................................................56
Independent Variable.............................................................................................60
Dependent Variables..............................................................................................61
Confounding Variables..........................................................................................64
Data Needed.......................................................................................................................65
Data Collected........................................................................................................66
Data Collection Strategies......................................................................................67
Site Selection.........................................................................................................67
Course Selection....................................................................................................69
Data Collection Processes......................................................................................71
Study Phases......................................................................................................................72
Phase One of Study................................................................................................73
Phase Two of Study...............................................................................................74
Development of Questionnaire..............................................................................75
Methods to Achieve High Rates of Institutional Sponsorship for Questionnaire..77
Methods to Achieve High Rates of Student Response on Questionnaire..............78
Validity of Questionnaire.......................................................................................79
Reliability of Questionnaire...................................................................................81
Pilot Testing of Questionnaire...............................................................................82
Data Analysis Procedures..................................................................................................84
Analysis for Research Question One.....................................................................87
Analysis for Research Question Two....................................................................88
Analysis for Research Question Three..................................................................89
Analysis for Research Question Four....................................................................90
Strategies to Ensure Soundness of Data.............................................................................91
Response to Threats to Internal Validity...............................................................91
TABLE OF CONTENTS (Continued)
Page
Response to Threats of External Validity..............................................................96
Strategies for Protection of Human Subjects.....................................................................98
Summary of Design of Study.............................................................................................99
CHAPTER 4: RESULTS ................................................................................................102
Study Variables................................................................................................................102
Independent Variable...........................................................................................102
Dependent Variables............................................................................................103
Student Satisfaction.................................................................................103
Student Retention.....................................................................................104

Confounding Variables........................................................................................105
Study Colleges.....................................................................................................106
“Reviewed” verses “All Other” Courses.............................................................107
Research Question One -- Overall Satisfaction...............................................................110
Question One: Independent Sample t-Tests.........................................................110
Question One: Controlling for Confounding Variables.......................................112
Question One: Five Factors of Online Quality....................................................114
Research Question Two -- Course Retention...................................................................116
Question Two: Independent sample t-Test for Course Retention........................117
Question Two: ANOVA Analysis Comparing Confounding Variables with
Retention Rates................................................................................................................118
Research Question Three -- Factors of Quality Online Instruction Related to
Student Satisfaction.........................................................................................................120
Question Three: Regression Analysis Ranking Six Factors of Quality...............121
Research Question Four: Factors of Quality Instruction Related To Increased
Retention..........................................................................................................................124
Regression Analysis Ranking Five Factors of Quality Instruction by
Percent Retention.................................................................................................125
Threats to Validity...........................................................................................................126
TABLE OF CONTENTS (Continued)
Page
Summary of Results.........................................................................................................129
CHAPTER 5: SUMMARY, DISCUSSION AND RECOMMENATIONS ...................133
Summary and Discussion of Findings.............................................................................133
Research Question One -- Overall Satisfaction...................................................133
Finding: Overall Satisfaction...................................................................134
Discussion: Overall Satisfaction..............................................................134
Finding: Mean of Factor Means...............................................................135
Discussion: Mean of Factor Means..........................................................136
Finding: Confounding Variables..............................................................136
Discussion: Confounding Variable..........................................................137

Research Question Two -- Student Retention......................................................138
Finding: Student Retention......................................................................139
Discussion: Student Retention.................................................................139

Research Question Three -- Factors of Quality Online Instruction Related to
Satisfaction...........................................................................................................140
Finding: Factors of Learning Outcomes..................................................140
Discussion: Factor of Learning Outcomes...............................................141
Finding: Factor of Assessment.................................................................142
Discussion: Factor of Assessment...........................................................142
Finding: Factor of Resource Materials....................................................143
Discussion: Factor of Resource Materials...............................................143
Finding: Factor of Interaction..................................................................144
Discussion: Factor of Interaction (Instructor-Student and
Student-Student)......................................................................................145
Finding: Factor of Technology................................................................147
Discussion: Factor of Technology...........................................................147

Research Question Four -- Factors of Quality Online Instruction Related to
Retention..............................................................................................................149
Findings: Student Retention and the Factors of Quality Online
Instruction................................................................................................149
Discussion: Student Retention and the Factors of Quality Online
Instruction................................................................................................150

Summary of Findings and Discussion Relating to Study Questions...................151
Recommendations Derived from Study...........................................................................152
Recommendations Related to Educational Policy...............................................153