Comparing the Accuracy of Performing Digital and Paper Checklists Using a Feedback Package

172 Pages·2017·2.78 MB·English

by  William Gene Rantz

WWeesstteerrnn MMiicchhiiggaann UUnniivveerrssiittyy
SScchhoollaarrWWoorrkkss aatt WWMMUU
Dissertations Graduate College
4-2009
CCoommppaarriinngg tthhee AAccccuurraaccyy ooff PPeerrffoorrmmiinngg DDiiggiittaall aanndd PPaappeerr
CChheecckklliissttss UUssiinngg aa FFeeeeddbbaacckk PPaacckkaaggee dduurriinngg NNoorrmmaall WWoorrkkllooaadd
CCoonnddiittiioonnss iinn SSiimmuullaatteedd FFlliigghhtt
William Gene Rantz
Western Michigan University
Follow this and additional works at: https://scholarworks.wmich.edu/dissertations
Part of the Cognition and Perception Commons, Cognitive Psychology Commons, and the Industrial
and Organizational Psychology Commons
RReeccoommmmeennddeedd CCiittaattiioonn
Rantz, William Gene, "Comparing the Accuracy of Performing Digital and Paper Checklists Using a
Feedback Package during Normal Workload Conditions in Simulated Flight" (2009). Dissertations. 676.
https://scholarworks.wmich.edu/dissertations/676
This Dissertation-Open Access is brought to you for free
and open access by the Graduate College at
ScholarWorks at WMU. It has been accepted for inclusion
in Dissertations by an authorized administrator of
ScholarWorks at WMU. For more information, please
contact [email protected].
COMPARING THE ACCURACY OF PERFORMING DIGITAL AND PAPER
CHECKLISTS USING A FEEDBACK PACKAGE DURING NORMAL
WORKLOAD CONDITIONS IN SIMULATED FLIGHT
by
William Gene Rantz
A Dissertation
Submitted to the
Faculty of The Graduate College
in partial fulfillment of the
requirements for the
Degree of Doctor of Philosophy
Department of Psychology
Advisor: Dr. Ron Van Houten
Western Michigan University
Kalamazoo, Michigan
April 2009
Copyright by
William Gene Rantz
2009
UMI Number: 3354079
INFORMATION TO USERS
The quality of this reproduction is dependent upon the quality of the copy
submitted. Broken or indistinct print, colored or poor quality illustrations and
photographs, print bleed-through, substandard margins, and improper
alignment can adversely affect reproduction.
In the unlikely event that the author did not send a complete manuscript
and there are missing pages, these will be noted. Also, if unauthorized
copyright material had to be removed, a note will indicate the deletion.
®
UMI
UMI Microform 3354079
Copyright 2009 by ProQuest LLC.
All rights reserved. This microform edition is protected against
unauthorized copying under Title 17, United States Code.
ProQuest LLC
789 E. Eisenhower Parkway
PO Box 1346
Ann Arbor, Ml 48106-1346
ACKNOWLEDGMENTS
I'd like to express my gratitude to the following individuals: To Dr. Ron Van
Houten, my graduate committee chair, who spent countless hours guiding, inspiring, and
reviewing my work. I would also like to thank my other graduate advisors, Dr. Brad
Huitema, Dr. Al Poling, and Dr. Vladimir Risukhin, for keeping me balanced in the
priorities of inferential statistics, experimental design, and developing ideas to formulate
the scope and reach of this study. I will always remember and appreciate their sage
advice. I must also express thanks to my former graduate advisor, Alyce Dickinson, for
helping guide much of the foundation this study builds upon. Notable thanks to Gaurav
Dave for his technical support in the experimentation laboratory and especially the
observation station used for data collection. Great appreciation goes to Gil Sinclair for
arranging simulator resources required to conduct the study. I need to especially thank my
student research assistants, Bryan Hilton and Jared Neterer, for their hard work during
simulator set up, validation, and data collection. Finally, I give thanks for the support
from all my family members and especially my two sons, Garrett and Gavin, who have
been patient with me through this long process.
William Gene Rantz
11
TABLE OF CONTENTS
ACKNOWLEDGMENTS ii
LIST OF TABLES vi
LIST OF FIGURES vii
CHAPTER
I. INTRODUCTION 1
II. METHOD 21
Participants 21
Setting 22
Apparatus 23
Frasca 241, Cirrus SR20 FTD 23
Flight Patterns 25
The Flight Checklists 26
Observation Equipment 26
Dependent Variables 28
Independent Variable 30
Experimental Design 31
Procedures 32
Recruitment 32
Informed Consent Process and Screening 32
iii
Table of Contents—Continued
CHAPTER
Baseline 33
Post-Flight Checklist Feedback 34
Reversal 35
60-90 Day Post-Test Probe 35
Debriefing 35
Data Analysis 35
Inter-Observer Agreement (10A) 39
Independent Variable Integrity 39
in. RESULTS 41
IV. DISCUSSION 69
REFERENCES 81
APPENDICES
A. Participant Eligibility Questionnaire 90
B. Recruitment Flyer 92
C. Recruitment Script 94
D. Informed Consent Form 96
E. Human Subjects Institutional Review Board Letter of Approval 99
F. Technical Flight Pattern Parameters and Narration 101
G. Flight Training Device Simulator Scenario Set Up 114
H. Meteorological Terminal Aviation Weather Reports (METARS)
and Automatic Terminal Information Service (ATIS) 121
iv
Table of Contents—Continued
APPENDICES
I. Flight Checklist 123
J. Observer's Checklist Behavior Protocol 126
K. Observer's Checklist Form 130
L. Digital and Paper Checklist Pre-flight Instructional Script 133
M. Technical Flight Pattern Diagram 140
N. Post-flight Technical Briefing Script 142
O. Checklist Feedback Script and Graphs 144
P. 60-90 Day Post-Test Research Probe Checklist Survey 147
Q. Debriefing Script 152
R. Statistical Analysis 154
S. Consolidated WMU Curriculum Checklist Procedures 157
v
LIST OF TABLES
1. Percentage of Paper Checklist Baseline Trials Participants Performed
Items in Error 52
2. Percentage of Digital Checklist Baseline Trials Participants Performed
Items in Error 56
3. Percentage of Items Omitted in Paper Checklist Baseline 60
4. Percentage of Items Omitted in Digital Checklist Baseline 65
VI
LIST OF FIGURES
1. Frasca 241, Cirrus SR20 Flight Training Device 24
2. Graphic Instructor Station 24
3. Avidyne Entegra EX5000C, Multi Functional Display with Digital
Checklist 27
4. Paper Checklist 27
5. Observation Equipment Station 28
6. Total Number of Paper and Digital Checklist Items Completed Correctly 42
7. Percentage of Paper Checklist Errors for Each Condition Per Participant 48
8. Percentage of Digital Checklist Errors for Each Condition Per Participant 49
9. Percentage of Paper Checklist Items Omitted During Each Experimental
Phase 59
10. Percentage of Digital Checklist Items Omitted During Each Experimental
Phase 63
11. Average Percent Paper Checklist Segment Timing Errors Per Phase 68
12. Average Percent of Digital Segment Timing Errors Per Phase 68
vn