Eliciting and Analyzing Expert Judgment: A Practical Guide (ASA-SIAM Series on Statistics and Applied Probability) PDF
Preview Eliciting and Analyzing Expert Judgment: A Practical Guide (ASA-SIAM Series on Statistics and Applied Probability)
Eliciting and Analyzing Expert Judgment ASA-SIAM Series on Statistics and Applied Probability The ASA-SIAM Series on Statistics and Applied Probability is published jointly by the American Statistical Association and the Society for Industrial and Applied Mathematics. The series consists of a broad spectrum of books on topics in statistics and applied probability. The purpose of the series is to provide inexpensive, quality publications of interest to the intersecting membership of the two societies. Editorial Board Richard F. Gunst Sallie Keller-McNulty Southern Methodist University, Editor-in-Chief Los Alamos National Laboratory Robert L. Mason Max D. Morris Southwest Research Institute, Editor-in-Chief Iowa State University Janet P. Buckingham Jane F. Pendergast Southwest Research Institute University of Iowa James A. Calvin Robert N. Rodriguez Texas A&M University SAS Institute, Inc. Gary C. McDonald General Motors R&D Center Smith, P. L., A Primer for Sampling Solids, Liquids, and Cases: Based on the Seven Sampling Errors of Pierre Gy Meyer, M. A. and Booker, J. M., Eliciting and Analyzing Expert Judgment: A Practical Guide Latouche, G. and Ramaswami, V., Introduction to Matrix Analytic Methods in Stochastic Modeling Peck, R., Haugh, L., and Goodman, A., Statistical Case Studies: A Collaboration Between Academe and Industry, Student Edition Peck, R., Haugh, L., and Goodman, A., Statistical Case Studies: A Collaboration Between Academe and Industry Barlow, R., Engineering Reliability Czitrom, V. and Spagon, P. D., Statistical Case Studies for Industrial Process Improvement Eliciting and Analyzing Expert Judgment A Practical Guide Mary A. Meyer Jane M. Booker Statistical Sciences Group Los Alamos National Laboratory Los Alamos, New Mexico ASA Society for Industrial and Applied Mathematics American Statistical Association Philadelphia, Pennsylvania Alexandria, Virginia Copyright ©2001 by the American Statistical Association and the Society for Industrial and Applied Mathematics. This SIAM edition is an unabridged republication of the work first published by Academic Press Limited, London, 1991. 1 0 9 8 7 6 5 4 3 21 All rights reserved. Printed in the United States of America. No part of this book may be reproduced, stored, or transmitted in any manner without the written permission of the publisher. For information, write to the Society for Industrial and Applied Mathematics, 3600 University City Science Center, Philadelphia, PA 19104-2688. Library of Congress Card Number: 00-105923 ISBN 0-89871-474-5 m ZlJLCJLJIL.is a registered trademark. For Dr. Thomas R. Bement, versatile statistician, co-developer of PREDICT, dedicated colleague, and wonderful friend This page intentionally left blank Contents Preface to ASA-SIAM Edition xxi Preface xxiii List of Figures xxv List of Tables xxvii List of Examples xxix PART I: INTRODUCTION TO EXPERT JUDGMENT 1 Introduction 3 What Is Expert Judgment? 3 When Expert Judgment Is Used 4 General Attributes of Expert Judgment 6 Expert Judgment Covered in This Book 7 How Expert Judgment Is Elicited 9 Philosophy Guiding the Elicitation 11 Philosophy Guiding the Analysis 12 How To Use This Book 13 2 Common Questions and Pitfalls Concerning Expert Judgment 17 Summary of Common Questions and Pitfalls 17 List of Common Questions 17 List of Pitfalls 18 Questions 19 What Does It Mean When the Experts Disagree? 19 Is Expert Judgment Valid Data? 20 Is the Gathering and Analyzing of Expert Judgment Scientific? 21 Are Experts Bayesian? 23 Do Experts Provide Better Data? 24 Can Experts Be Fully Calibrated? 26 vii viii CONTENTS Pitfalls 27 Interviewers, Knowledge Engineers, and Analysts Can Introduce Bias 27 Experts Are Limited in the Number of Things That They Can Mentally Juggle 29 The Level of Detail in the Data (Granularity) Affects the Analyses 30 The Conditioning Effect Poses Difficulties in Gathering and Analyzing Expert Data 32 3 Background on Human Problem Solving and Bias 35 Why Is It Necessary to Have an Understanding of Human Problem Solving? 35 What Is Involved in Solving Problems and in Responding? 35 The Four Cognitive Tasks 35 Two Models of Human Memory and Information Processing 37 Bias 38 Two Views of Bias 40 Potential Impact of Bias 41 Sources of Bias 42 Motivational Bias 42 Cognitive Bias 44 Program for Handling Bias 44 Determining Which Steps to Apply 47 The Reason for Focusing on Bias 47 The Selection of the Definition of Bias, Motivational or Cognitive 50 Interest in Particular Types of Bias 51 PART II: ELICITATION PROCEDURES 4 Selecting the Question Areas and Questions 57 Steps Involved in Selecting the Questions 57 CONTENTS ix Illustrations of the Variation in Project Goals, Question Areas, and Questions 58 Sources of Variation 59 Executing the Steps with the Assistance of Sponsors, Project Personnel, and Experts 61 Determining in Which Steps the Advisory or External Experts Will Assist 63 The External Experts Do Not Participate in Question Area Selection, Question Identification, or Refinement, Except Minimally Just Before Having Their Judgments Elicited 63 The External Experts Receive the Question Areas and Develop the Questions and Refine Them, or They Receive the Questions and Refine Them 64 The External Experts Help Select the Question Areas and Assist in the Identification and Refinement of the Questions 64 Checklist for Selected Questions 65 Common Difficulties—Their Signs and Solutions .. 67 Difficulty: Sponsor Cannot Provide Clear Information on the Project's Goal, the Data to Be Gathered, or the Question Areas 67 Difficulty: The Question Developed From the Question Area Remains Too Broad 67 Difficulty: Too Many Questions Have Been Selected for the Amount of Time Available .. 68 5 Refining the Questions 69 Reasons for Structuring the Questions 69 Techniques for Structuring the Questions 71 Presentation of the Question Information 71 Types of Question Information Needed 71 Background 71 Assumptions 71 Definitions 72 Ordering of Information 72 Roles of Project Personnel and Experts 73 Decomposition of the Question 74 Considerations in Question Decomposition 74
Description:The list of books you might like
