Modification and Adaptation of the Program Evaluation Standards in Saudi Arabia

138 Pages·2016·1.31 MB·English

WWeesstteerrnn MMiicchhiiggaann UUnniivveerrssiittyy
SScchhoollaarrWWoorrkkss aatt WWMMUU
Dissertations Graduate College
6-2013
MMooddiifificcaattiioonn aanndd AAddaappttaattiioonn ooff tthhee PPrrooggrraamm EEvvaalluuaattiioonn SSttaannddaarrddss
iinn SSaauuddii AArraabbiiaa
Mohammed Alyami
Western Michigan University, [email protected]
Follow this and additional works at: https://scholarworks.wmich.edu/dissertations
Part of the Educational Assessment, Evaluation, and Research Commons, and the International and
Comparative Education Commons
RReeccoommmmeennddeedd CCiittaattiioonn
Alyami, Mohammed, "Modification and Adaptation of the Program Evaluation Standards in Saudi Arabia"
(2013). Dissertations. 157.
https://scholarworks.wmich.edu/dissertations/157
This Dissertation-Open Access is brought to you for free
and open access by the Graduate College at
ScholarWorks at WMU. It has been accepted for inclusion
in Dissertations by an authorized administrator of
ScholarWorks at WMU. For more information, please
contact [email protected].
MODIFICATION AND ADAPTATION OF THE PROGRAM EVALUATION
STANDARDS IN SAUDI ARABIA

by

Mohammed Alyami

A dissertation submitted to the Graduate College
in partial fulfillment of the requirements
for the degree of Doctor of Philosophy
Interdisciplinary Ph.D. in Evaluation
Western Michigan University
June 2013

Doctoral Committee:

Chris Coryn, Ph.D., Chair
Daniela Schroeter, Ph.D.
Lori Wingate, Ph.D.
MODIFICATION AND ADAPTATION OF THE PROGRAM EVALUATION
STANDARDS IN SAUDI ARABIA

Mohammed Alyami, Ph.D.

Western Michigan University, 2013

The Joint Committee on Standards for Educational Evaluation’s Program
Evaluation Standards is probably the most recognized and applied set of evaluation
standards globally. The most recent edition of The Program Evaluation Standards
includes five categories and 30 standards. The five categories are Utility, Feasibility,
Propriety, Accuracy, and Evaluation Accountability. In recent years, evaluation has
grown into a global practice with more than sixty national evaluation associations around
the world. Examples include Japan, Malaysia, Sri Lanka, Mongolia, Russia, Brazil,
Colombia, Peru, South Africa, Zimbabwe, Niger, and New Zealand. Because countries
differ from one another in context and culture, they may need different sets of standards
for better execution and results of evaluation practices. However, this is not the case in
every country.
The cultural adaptation of The Program Evaluation Standards, the growing
economy in Saudi Arabia, and evaluation quality are the key issues framing this study.
The following research questions are addressed: (1) To what extent do The Program
Evaluation Standards fit the Saudi context? (1.1) Which standards are acceptable in their
current form without modification? (1.2) Which standards need modification to be
acceptable? and (1.3) Which standards are unacceptable, with or without modification?
(2)What, if any, aspects of the standards are perceived to be especially problematic for
application in the Saudi context? (3)What modifications are necessary to yield a complete
set of standards for the Saudi context? The findings suggest that The Program Evaluation
Standards needs to be modified for the Saudi context. As a result, a modified set of
standards titled “The Saudi Program Evaluation Standards” will be disseminated,
including four categories and 28 standards.
© 2013 Mohammed Alyami
ACKNOWLEDGMENTS

First of all, I would like to thank Allah (God) for his blessing and seek his help
and guidance; all praise is due to Allah. I would like to humbly dedicate this work to my
father, Saeed Bin Tami, and my mother, Refah Bent Mesfer, for their kindness, support,
and encouragement throughout all of these years.
Having the opportunity to work with my dissertation committee was one of the
most professional, but enjoyable experiences of my life. I wish I could write this
acknowledgment in Arabic to be able to choose the most appropriate appreciation words
to my committee Dr. Chris Coryn, Dr. Daniela Shroeter, and Dr. Lori Wingate. I would
like to thank Dr. Wingate for her help, directions, and encouragement. Also, I would like
to express my thankfulness to Dr. Shroeter for her exceptional professional and ethical
commitment. I was fortunate to have Dr. Coryn as my advisor during the doctorate
program, and as my committee chair. All the difficulties that I faced during this hard
work were dealt with professionally and smoothly because of his thoughtful guidance and
advice.
I would also like to expand my appreciativeness to the expert panel that
participated voluntarily in the study, and provided me with the information I needed to
achieve the final results.

ii
AcknowledgmentsContinued

Last, but not least, to my wife and my daughters I would like to say, “Refah,
Salwa, Leanne, Leila, and Celine - you were the source of energy and happiness during
this long journey.”

Mohammed Alyami
iii
TABLE OF CONTENTS

ACKNOWLEDGMENTS ............................................................................................ ii
LIST OF TABLES ........................................................................................................ viii
LIST OF FIGURES ...................................................................................................... ix
CHAPTER
I. INTRODUCTION ........................................................................................... 1
Background and Issues That Frame This Study ..................................... 6
Overview of Saudi Arabian Context ....................................................... 8
Government ................................................................................... 9
Constitution (Shari'ah Law) ........................................................... 9
Language........................................................................................ 9
Gender............................................................................................ 10
Corruption ...................................................................................... 10
Statement of the Problem ........................................................................ 11
Purpose of the Study ............................................................................... 12
Focal Research Questions ....................................................................... 12
Rationale for and Significance of the Study ........................................... 13
Contributions of the Study ...................................................................... 13
Organization of the Dissertation ............................................................. 14
II. LITERATURE REVIEW ................................................................................ 15
Overview ................................................................................................. 15
iv
Table of ContentsContinued

CHAPTER

Relevant Studies ..................................................................................... 16
Germany ........................................................................................ 17
Switzerland .................................................................................... 23
African Continent .......................................................................... 28
South Korea ................................................................................... 35
Australia and New Zealand ........................................................... 36
Bangladesh ..................................................................................... 37
Japan .............................................................................................. 38
Summary ................................................................................................. 39
III. METHODOLOGY .......................................................................................... 41
Research Questions ................................................................................. 41
Design ..................................................................................................... 41
Focus Group................................................................................... 44
Questionnaire ................................................................................. 46
Setting, Sampling, and Subjects ............................................................. 48
Setting ............................................................................................ 48
Sampling ........................................................................................ 48
Subjects .......................................................................................... 49
Instrumentation ....................................................................................... 49
Focus Group................................................................................... 50
v
Table of ContentsContinued

CHAPTER
Phase 1 Questionnaires .................................................................. 50
Phase 2 Questionnaires .................................................................. 51
Procedures ............................................................................................... 51
Expert Recruitment ........................................................................ 51
Focus Group Procedures ................................................................ 53
Data Processing....................................................................................... 55
Data Analysis .......................................................................................... 55
Timeline .................................................................................................. 56
Limitations and Delimitations ................................................................ 56
IV. RESULTS ........................................................................................................ 58
Focus Group Results ............................................................................... 58
Utility Standards ............................................................................ 59
Feasibility Standards ...................................................................... 63
Propriety Standards ........................................................................ 65
Accuracy Standards ....................................................................... 67
Evaluation Accountability Standards ............................................ 68
Modifications to Program Evaluation Standards Based on
Focus Group Results ............................................................................... 70

Results of the First-Round Questionnaire ............................................... 72
Additional Modifications to Program Evaluation Standards
Based on First-Round Questionnaire Results ......................................... 76
vi