Table Of Content

EFFECTS OF PROCEDURAL AND OUTCOME ACCOUNTABILITY ON INTERVIEW VALIDITY By MARY D. BRTEK A DISSERTATION PRESENTED TO THE GRADUATE SCHOOL OF THE UNIVERSITY OF FLORIDA IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF PHILOSOPHY UNIVERSITY OF FLORIDA ACKNOWLEDGMENTS Completion of this dissertation and my doctoral program would have been impossible without the help and encouragement of many important people in my life. From the very first day of my doctoral program to the present day, Scott Kiker has been there for me to help or lend support whenever needed, which was and continues to be often. He has been extremely patient and loving during the rough spots of my educational journey and throughout my life. To him, I give my heart and my gratitude. I would also like to thank my wonderful parents, Lou and Phyllis, for their tremendous, unending support throughout this program and my life. They taught me never give up, work hard, play hard, and never take life too seriously. I could not have picked better people to call Mom and Dad. I would also like to thank my sister, Michelle, and brothers, Mike and Mark, for always being there for me. I am glad they finally learned, after four years, what degree I was getting and the field in which I was getting it. 11 I owe special thanks to my chair, Steve Motowidlo. His patience, intellectual nurturing, encouragement, and guidance have contributed greatly to this endeavor and to my intellectual development. I am also grateful to my committee members-Henry Tosi, Amir Erez, and Bart Weitz. They were always willing to listen and help make this research project more interesting. I would also like to thank Judy Scully for her constant encouragement and support of my abilities. Finally, I would like to thank two special friends, my classmate, Amy, and my best friend, Nick. Amy has stood by my side throughout my doctoral program-making me laugh, talking me down, and helping me whenever I needed it. Nick has been there to liven up my life and keep a smile on my face. He will always the best and most entertaining friend I've ever had. in TABLE OF CONTENTS page ACKNOWLEDGMENTS ii ABSTRACT vi CHAPTERS 1 INTRODUCTION, LITERATURE REVIEW, AND HYPOTHESES 1 Introduction 1 Literature Review 4 Review of Accountability Studies 19 Accountability and Interview Judgments 29 Hypotheses 32 Exploratory Analyses 32 2 METHODS 37 Introduction 37 Experimental Description 38 Manipulation Checks 42 Primary Study 46 Procedures 47 Instruments 48 Dependent Variable 49 Variables for Additional Analyses 50 3 RESULTS 53 Manipulation Checks 53 Methods of Analysis 58 Tests of Hypotheses 58 Additional Analyses 61 IV 4 DISCUSSION 68 Methodological Issues for Laboratory Studies 68 Findings 70 Conclusion 75 REFERENCES 76 APPENDICES A RATING FORM 83 B INSTRUCTIONS 84 C MANIPULATIONS 85 D MANIPULATION CHECKS 86 E DEMOGRAPHIC SHEET 87 F ATTENTIVENESS RATING SCALE 88 BIOGRAPHICAL SKETCH 89 v . Abstract of Dissertation Presented to the Graduate School of the University of Florida in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy EFFECTS OF PROCEDURAL AND OUTCOME ACCOUNTABILITY ON INTERVIEW VALIDITY By Mary D. Brtek December 1999 Chairperson: Stephan J. Motowidlo Major Department: Management Much research has been conducted on the employment interview, especially on methods for improving its validity and reliability. Results of this early research were discouraging however because both interview reliability and validity were usually found to be quite low. Recently, however, researchers have found that structuring the interview can improve interview reliability and validity dramatically. Unfortunately, in practice interviewers very often deviate from following the structured interview format closely, resulting in a less valid interview over time Accountability might help lessen the problem of interviewers deviating from the structured interview vi . format. Typically, an interviewer is accountable only for the new hire's success, or the outcome of the interview decision (outcome accountability). Accountability of this sort, however, does not necessarily influence the interviewer to follow the structured format exactly. A better solution may be to hold the interviewer accountable not for the outcome of the interview decision, but for following the structured interview format (procedural accountability) The present study sought to apply the distinction of two different types of accountability to the interviewing context. Specifically, this study sought to determine the effects of procedural and outcome accountability on interview validity. Three-hundred thirty-eight participants in one of four experimental conditions formed by crossing two levels of outcome accountability with two levels of procedural accountability watched and rated one of two sets of 30 videotaped interviewees presented in two sessions over a two week period. Validity scores were then computed for each pair of participants. Results showed that procedurally accountable participants were significantly more valid than were participants not held procedurally accountable. Also, outcome accountable participants were significantly less Vll valid than those who were not outcome accountable. These results indicated that interviewers should be accountable for following the interview structure in order to hire the best job performers. When they were held accountable for the new hires' success, the traditional performance standard, interviewers actually chose to hire lower job performers. Additional analyses suggested that procedurally accountable raters were more valid because they were more attentive to the interview information. viii . CHAPTER 1 INTRODUCTION, LITERATURE REVIEW, AND HYPOTHESES Introduction Since 1914 researchers have struggled to describe precisely the procedures required for making the employment interview more reliable and more valid (Eder & Harris, 1999) Despite these efforts, most interview research prior to the early 1980's yielded little in terms of practical advice to organizations in helping them make more valid and reliable selection decisions. It was not until 1980 that this pessimistic view showed signs of changing. In that year, structured interviews were introduced by Latham, Saari, Pursell, and Campion (1980). Since then, many structured interviews have been developed, such as the situational interview (Latham & Saari, 1984; Weekly & Gier, 1987), the past behavior description inteview (Janz, 1982; Janz, Hellervik, & Gilmore, 1986; Orpen, 1985), and the structured behavioral interview (Motowidlo, et al., 1992). Results of several meta-analyses support that structured interviews are more valid and reliable than unstructured interviews (McDaniel, et al., 1994; Wiesner & Cronshaw, 1988; Huffcutt & Arthur, 1994). 1 2 Though the reasons that structured interviews are more valid than unstructured interviews is unclear, Diboye and Gaugler (1993) suggest that one probable reason is that structured interviews constrain an interviewer's discretion, thereby forcing the interviewer to focus on valid sources of information and processing strategies and away from focusing on idiosyncratic cues and invalid processing strategies. Regardless of the cause, it appears that structured interviews will be more valid and reliable if they are followed properly. Unfortunately, Latham and Saari (1984) showed that the introduction of the structured interview into an organization increases interview validity initially, but over time these effects gradually diminish. It seems that, in practice, interviewers very often deviate from following the structured interview format closely, resulting in a less valid interview over time. One possible solution to this problem might be in holding interviewers accountable for following closely the structured interview guidelines. Accountability refers to making the interviewers "answerable to external audiences for performing up to certain prescribed standards thereby fulfilling obligations, duties, expectations, and other charges" (Schlenker, Britt, Pennington, Murphy, & Doherty, 1994, 634). Typically, if an interviewer is held accountable at all, he or she is accountable only for the