ebook img

Handbook of Multimethod Measurement in Psychology PDF

446 Pages·2006·51.075 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Handbook of Multimethod Measurement in Psychology

Handbook of Multimethod Measurement in Psychology Editors: Eid, Michael & Diener, Ed Hardcover: 553 pages Publisher: American Psychological Association (APA) Publication Date: January, 2006 Language: English ISBN-10: 1591473187 ISBN-13: 978-1591473183 Additional Information Categories: Psychology: Assessment, Testing & Measurement Subjects: Research Psychology Summary: Handbook of Multimethod Measurement in Psychology is the first integrative guide to theoretical, methodological, and applied aspects of multimethod measurement in psychological research. Although the multitrait-multimethod analysis and multimethod research strategies have become increasingly important in all areas of psychological research, comprehensible handbooks of multimethod measurement that integrate all phases of the data-gathering and assessment process have been missing. Handbook of Multimethod Measurement in Psychology fills this long-standing gap. Researchers and students of research who want to run a multimethod study or who want to understand a published study no longer must page through methodological journals trying to piece together an understanding of multimethod approaches. This volume provides a single, understandable resource for learning the meaning, advantages, and limitations of different methods. The volume brings together outstanding researchers from all areas of psychology. Contributors give detailed explanations about exactly what has to be considered when conducting a multimethod study, what the strength of the convergent validity and method-specificity in different areas of research is, and what methods should be selected for proving validity in specific research contexts. The volume discusses theoretical concepts, then lays out the most important measurement tools— including, among many others, global self-assessment, informant assessment, observational methods, and nonreactive methods—followed by statistical models for analyzing multitrait-multimethod data. Contributors also explore applications of multimethod research in different areas of psychology as well as broader perspectives on the meaning and purpose of multimethod approaches. Destined to be a classic, Handbook of Multimethod Measurement in Psychology provides an indispensable resource for researchers and students in psychology seeking to enrich their work by using multimethod approaches. CONTENTS Preface I. Theoretical Concepts 1. Introduction: The need for multimethod measurement in psychology 2. Conceptual, theoretical, and historical foundations of multimethod assessment II. Assessment Methods 3. Global self-assessment 4. Informant assessment 5. Momentary capture of real-world data 6. Web based methods 7. Computer-based testing 8. Ability tests 9. Catching the mind in action: Implicit methods in personality research and assessment 10. Sequential observational methods 11. Quantitative text analysis 12. Multilevel Analysis: Physiological and biochemical methods 13. Brain imaging and related methods 14. Nonreactive methods in psychological research 15. Experimental methods of psychological assessment III. Methods of Data Analysis 16. Methodological approaches for analyzing multimethod data 17. Assessing multimethod association with categorical variables 18. Multimethod item response theory 19. Multilevel models for multimethod measurements 20. Structural equation models for multitrait-multimethod data 21. Longitudinal methods IV. Applications 22. Using multiple methods in personality psychology 23. Measuring emotions: Implications of a multi-method perspective 24. The evolution of concepts in research on human memory 25. Applying a multi-method perspective to the study of developmental psychology 26. Multimethod approaches in social psychology: Between and within method replication and multimethod assessment 27. Clinical psychology: Construct validation with multiple sources of information and multiple settings 28. Multimethod Approaches in health psychology 29. Multiple methods in industrial and organizational psychology: Expanding “methods” to include longitudinal designs 30. A Multiple Method Perspective on Self-concept Research in Educational Psychology: A Construct Validity Approach 31. The Finale: Take-home messages from the editors TABLE OF CONTENTS SECTION I: Theoretical Concepts CHAPTER 1 CHAPTER 2 Eid, Michael; Diener, Ed Schmitt, Manfred Introduction: The Need for Conceptual, Theoretical, and Historical Multimethod Measurement in Foundations of Multimethod Psychology Assessment pp 3-8 pp 9-25 SECTION II: Assessment Methods CHAPTER 3 CHAPTER 8 Lucas, Richard E.; Baird, Brendan M. Lubinski, David Global Self-Assessment Ability Tests pp 29-42 pp 101-114 CHAPTER 4 CHAPTER 9 Neyer, Franz J. Robinson, Michael D.; Neighbors, Informant Assessment Clayton pp 43-59 Catching the Mind in Action: Implicit Methods in Personality Research and CHAPTER 5 Assessment Stone, Arthur A.; Litcher-Kelly, pp 115-125 Leighann Momentary Capture of Real-World CHAPTER 10 Data Bakeman, Roger; Gnisci, Augusto pp 61-72 Sequential Observational Methods pp 127-140 CHAPTER 6 Reips, Ulf-Dietrich CHAPTER 11 Web-Based Methods Mehl, Matthias R. pp 73-85 Quantitative Text Analysis pp 141-156 CHAPTER 7 Drasgow, Fritz; Chuah, Siang Chee Computer-Based Testing pp 87-100 CHAPTER 12 CHAPTER 14 Berntson, Gary G.; Cacioppo, John T. Fritsche, Immo; Linneweber, Volker Multilevel Analysis: Physiological and Nonreactive Methods in Psychological Biochemical Measures Research pp 157-172 pp 189-203 CHAPTER 15 CHAPTER 13 Erdfelder, Edgar; Musch, Jochen Zald, David H; Curtis, Clayton Experimental Methods of Psychological Brain Imaging and Related Methods Assessment pp 173-187 pp 205-220 SECTION III: Methods of Data Analysis CHAPTER 16 CHAPTER 19 Eid, Michael Hox, Joop; Maas, Cora Multilevel Methodological Approaches for Models for Multimethod Analyzing Multimethod Data Measurements pp 223-230 pp 269-281 CHAPTER 17 CHAPTER 20 Nussbeck, Fridtjof W. Assessing Eid, Michael; Lischetzke, Tanja; Multimethod Association With Nussbeck, Fridtjof W. Categorical Variables Structural Equation Models for pp 231-247 Multitrait-Multimethod Data pp 283-299 CHAPTER 18 Rost, Jurgen; Walter, Oliver CHAPTER 21 Multimethod Item Response Theory Khoo, Siek-Toon; West, Stephen G; pp 249-268 Wu, Wei; Kwok, Oi-Man Longitudinal Methods pp 301-317 SECTION IV: Applications CHAPTER 22 CHAPTER 27 Roberts, Brent W; Harms, Peter; Burns, G. Leonard; Haynes, Stephen N. Smith, Jennifer L; Wood, Dustin; Clinical Psychology: Construct Webb, Michelle Validation With Multiple Sources of Using Multiple Methods in Personality Information and Multiple Settings Psychology pp 401-418 pp 321-335 CHAPTER 28 CHAPTER 23 Knauper, Barbel; Klein, Rupert Larsen, Randy J; Prizmic-Larsen, Multimethod Approaches in Health Zvjezdana Psychology Measuring Emotions: Implications of a pp 419-427 Multimethod Perspective pp 337-351 CHAPTER 29 Miner, Andrew G.; Hulin, Charles L. CHAPTER 24 Multimethods in Industrial and Benjamin, Aaron S Multimethod Organizational Psychology: Expanding Approaches to the Study of Cognition: "Methods" to Include Longitudinal The Evolution of Concepts in Research Designs on Human Memory pp 429-439 pp 353-369 CHAPTER 30 CHAPTER 25 Marsh, Herbert W.; Martin, Andrew J.; Morris, Amanda Sheffield; Robinson, Hau, Kit-Tai Lara R; Eisenberg, Nancy Applying a A Multimethod Perspective on Self- Multimethod Perspective to the Study Concept Research in Educational of Developmental Psychology Psychology: A Construct Validity pp 371-384 Approach pp 441-456 CHAPTER 26 Smith, Richard H.; Harris, Monica J. CHAPTER 31 Multimethod Approaches in Social Diener, Ed; Eid, Michael Psychology: Between- and Within- The Finale: Take-Home Messages From Method Replication and Multimethod the Editors Assessment pp 457-463 pp 385-400 CHAPTER 1 INTRODUCTION: THE NEED FOR MULTIMETHOD MEASUREMENT IN PSYCHOLOGY Michael Bid and Ed Diener We must measure what can be measured, and make measurable what cannot be measured. —Galileo Galilei (1610) The history of empirical psychology is one of mak- And there are previously popular methods (e.g., ing measurable what most laypeople think cannot unobtrusive and nonreactive measures), which must be measured—emotions, cognitions, motivations, be preserved and revived as valuable measurement personality traits, and so forth. As in all other tools that offer insights not otherwise obtained. empirical and natural sciences, the progress of psy- Although each epoch has its own scientific para- chology is closely and inextricably linked to the digms and methods that fit better than other meth- development of new and more refined methods for ods, it would be unwise to stake scientific insight measuring psychological concepts. New technical on just one. A multimethod approach offers developments (e.g., modern methods of brain imag- insights into scientific phenomena and can con- ing or biochemical analyses) allow deeper insights tribute to confirming psychological theories in a into psychological processes than ever expected at way a single-method approach cannot (Schmitt, this the end of the 19th century when psychology began volume, chap. 2). There are at least two reasons establishing itself as an empirical science. Modern why psychology research and applied work make a computer technology enables traditional psycholog- multimethod research necessary: the multicompo- ical methods (e.g., ability testing, behavior observa- nential structure of psychological phenomena and tion, text analysis, and reaction time measures) to the validity of a research program. come into full flower and makes it possible to real- ize old dreams like measuring individuals in their MULTICOMPONENTIAL AND MULTILEVEL everyday lives far away from the anonymity and STRUCTURE OF PSYCHOLOGICAL artificiality of the psychological laboratory. Modern PHENOMENA communication tools like the Internet make it pos- sible to conduct experiments around the world Psychological phenomena usually consist of many across borders closed to researchers. facets. Emotions, for example, refer not only to the Besides the rapid progress that technological conscious representation of the feeling itself ("I am revolutions cause throughout the sciences, each happy") but also to many other changes in the indi- science has its classical standard measures that with- vidual (Davidson, Goldsmith, & Scherer, 2003; stand the tide of technological progress, almost Kahneman, Diener, & Schwarz, 1999; Larsen & unchanged. Just as a medical doctor does not aban- Prizmic-Larsen, this volume, chap. 23). An individ- don listening to lung sounds with instruments that ual feeling happiness might jump up, feel an inner have not fundamentally changed over the years, vir- ease, and even embrace and kiss passersby. More- tually no psychologist discards the treasure chests of over, muscle changes in the face might accompany self- and informant-reports, even though the way he this feeling, the brain might produce endorphins, or she uses them also remains virtually unchanged. and the individual may likely entertain positive Eid and Diener thoughts. This simple example shows that a psy- For example, while two individuals may feel pride chological phenomenon has many facets. To under- after receiving a compliment, one might jump for stand these emotional reactions it is necessary to joy, while the other quietly sits down. Analyzing have various, appropriate research methods to ana- individual differences in the associations between lyze the diverse facets. A method that measures the components might reveal that the first person muscle movements in the face is not appropriate to grew up in a culture in which pride is a highly assess endorphin levels in the brain. Moreover, a appreciated emotion (e.g., the United States), method for determining the endorphin level is whereas the other was raised in a culture in which probably not useful for assessing subjective feelings. pride is undesirable and should not be expressed, A thorough understanding of an emotional reaction for example in East Asian cultures (e.g., Eid & requires a set of appropriate multimethod tools to Diener, 2001). Hence, combining multimethod measure the different facets of the phenomena. approaches for analyzing individual differences in Multilevel analyses are a related example of the covariation of different components of a multi- multimethod research programs. Berntson and component phenomenon may help us understand Cacioppo (this volume, chap. 12) define multilevel individual and social regulation processes. analyses as a subset of multimethod approaches These simple examples show that a multimethod where the measures, constructs, and theories research program is necessary for a thorough extend across levels of organization—from the psy- description of multicomponent phenomena, as well chological to the physiological to the cellular and as for analyzing the different components of phe- ultimately to the gene and beyond.1 To assess the nomena to detect general and individual rules of different levels, different methods are needed. behavior. A classic example of multimethod Hence, the first aim of using multimethod research strategies is Murray's (1938) famous approaches is the precise description of the multi- Explorations in Personality, where he used such component and multilevel phenomena that are the diverse methods as aptitude tests, projective tests, focus of the behavioral sciences. questionnaires, interviews, and so forth to learn A second aim of multimethod research is provid- more about the different components of the person- ing information for detecting general associations ality of the participants of his study. between different components and levels of a phe- nomenon. For example, to analyze the reasons why VALIDITY happy individuals might be healthier, research must show a link between the feeling component and rel- Validity, one of the key issues of research, concerns evant physiological measures that explain individ- the question whether the inferences drawn from the ual differences in health. Insight into these results of a study are true or not (Shadish, Cook, & processes can be obtained by multimethod research Campbell, 2002). In particular, with respect to programs. However, general relations between the measurement methods, validity represents the different components form only one side of the degree to which the adequacy and appropriateness coin. Beyond general associations individual differ- of inferences and actions based on the results of a ences must be considered because not all individu- measurement device are supported by empirical evi- als behave in the same way. If an emotional reaction dence and theoretical rationales (Messick, 1989). were patterned in a uniform way, measuring one Multimethod research plays a key role in the valida- component would suffice when predicting other tion process. In their groundbreaking article, "Con- components. However, strong individual differences vergent and discriminant validation by the often exist when exploring different components. multitrait-multimethod matrix," Campbell and 'In this handbook the term multilevel analyses will also be used for statistical methods for analyzing nested data (e.g., students nested within classes, measurement occasions nested within individuals, etc). There are strong differences between multilevel analyses as a research program for measuring different determinants of behavior and multilevel analysis as statistical method. However, the appropriate meaning will be clearly deter- mined by the context. Introduction Fiske (1959) described the cornerstones of a multi- or severity errors and halo effects can affect peer rat- trait-multimethod research program regarding the ings, and peer- as well as self-rating might also be validation process. The basic promises of the multi- distorted by social desirability effects (Neyer, this trait-multimethod approach have strongly influ- volume, chap. 4). All these forms of bias and distor- enced the process of exploring validity. First, tion can cause small convergent validity coefficients. Campbell and Fiske pointed out that several meth- Therefore, Westen and Rosenthal (2003) recom- ods are needed to appropriately analyze validity, mend quantifying construct validity by comparing and these different methods should converge in the the observed patterns of correlations with the theo- measurement of the same trait. The convergence of retically expected patterns of correlations. They con- different independent methods indicates convergent tend that if a good theoretical reason for expecting validity. Second, they convincingly argued that dis- lower correlations between multiple measures exists, criminant validity must be shown before introducing and this pattern of correlations can be empirically a new construct into science. Third, Campbell and confirmed, modest degrees of convergence can con- Fiske clarified that a score on a psychological vari- firm construct validity. able not only reflects the psychological construct High convergent validity is not always the goal under consideration, but also reflects systematic of research. Take, for example, a questionnaire method-specific influences. Fourth, they demon- measuring different facets of marital satisfaction. strated the necessity of including at least two differ- Spouses rate their own satisfaction and also their ent methods in psychological studies to separate perception of the satisfaction of their spouse. If the trait from method influences. Hence, for a complete aim of the test construction process was to develop understanding of psychological processes it is nec- a questionnaire that detects deficiencies in essary to apply a multimethod research strategy. intraspouse perception and communication Therefore, the multitrait-multimethod analysis has processes, the items with the lowest convergences become an essential strategy for proving the con- might be the most interesting. In other words, struct validity of psychological measures. method influences are not inevitably unwanted ran- Convergent validity is a core aspect of validity, dom disturbances (e.g., measurement error) but and validation research programs have been focused they can indicate valid and valuable information. A for a long time on seeking high convergent validity deeper understanding of method influences can coefficients. Although high validity coefficients are enlarge our knowledge of the construct under con- desirable many reasons explain why convergent sideration, and this knowledge, in turn, can help validity coefficients are often lower than hoped. For explain method effects, correct for method effects, example, if one compares physiological measures and plan and conduct studies in which method with other measures one must contend with individ- effects are minimized or—depending on the aim of ual response-uniqueness (e.g., Berntson & the study considered—maximized. Beyond the tra- Cacioppo, 2004). Not all individuals react to stimu- ditional search for maximum convergent validity, a lus in the same way, and this response specificity thorough analysis of method influences might tell a can lower convergence when measured with a corre- more interesting story of the construct under con- lation coefficient. Moreover, if one wants to compare sideration. Hence, a multimethod study should a self-rating with a peer-rating, one often uncovers always have two facets: first, the proof of conver- medium-sized correlation coefficients. In comparing gent validity on the basis of theoretical expecta- self- and other-ratings one must recognize rater tions, and second, the analysis of the nature of biases (Hoyt, 2000). Raters may not only interpret method-specific influences. Whereas multimethod scale items differently but might also have opportu- studies intend to meet the first goal, the second nities to observe different behavior, they might use goal is often not considered when planning the different indicators of behavior, and they might link study's design. A careful analysis of method effects the indicators to the response scale in a different requires the inclusion of variables that may explain way (Hoyt, 2000; Kenny, 1991). Moreover, leniency method influences, and that might suppress Eid and Diener method-specific effects to enhance convergent This selection was guided by the goal of presenting validity. This makes a thorough knowledge of meas- methods at the basic level, which can be combined urement methods necessary for all researchers. to understand more complex assessment strategies. For example, you will not find a chapter on inter- AIMS OF THIS HANDBOOK AND OVERVIEW viewing, as an interview situation combines many more basic methods (e.g., self-report, observational Because multimethod research and assessment methods, informant assessment, and text-analysis of strategies are superior to monomethod approaches, the transcript of the interview). The handbook this handbook aims to provide the reader with the focuses more on assessment methods than on necessary knowledge to plan and conduct multi- research methods in general. It follows the tradition method studies and to analyze multimethod data. of Campbell and Fiske's (1959) multimethod We present the most important measurement meth- approach; therefore, we present methods that can ods and show their applications (Part II). Further- be used to measure human behavior, attitudes, and more, we present the most up-to-date methods for feeling. We do not present research methods to test analyzing multimethod data (Part III). Finally, sev- theories without assessing humans (e.g., animal eral chapters discuss applications of multimethod studies and computer simulation techniques). research programs in different areas of psychology These methods may hold importance for multi- (Part IV). These chapters show how multimethod method research in general, but they are less signifi- research programs can be successfully applied and cant for assessment purposes. We use the term discuss problems related to the implementation of multimethod, in most cases, in the sense of applying multimethod strategies. Although these chapters different methods for measuring human beings focus on single subdisciplines of psychology, they although some chapters also refer to multimethod also discuss issues relevant to other fields. Because research programs in the sense of applying different the ways multimethod strategies are implemented research strategies (e.g., experimental vs. nonexper- differ between subdisciplines, we strongly recom- imental research). mend consulting applications chapters in domains Moreover, we focus on widely applied and estab- different from those in which one is primarily inter- lished methods, including more recent develop- ested. Examining methods in other subdisciplines ments (like brain imaging). Some new methods gives readers new ideas about how to improve their may have a high potential for psychological assess- own research and how to develop new and innova- ment and measurement but are less established, tive research programs. with a status more comparable to research methods. Some of these methods include virtual environment Part I: Theoretical Concepts technology (Blascovich et al., 2002) and molecular The first part of the handbook introduces the genetic analysis (e.g., Caspi et al., 2003) and are importance of multimethod assessment in psychol- not considered in this volume. ogy, the philosophical and historical aspects of mul- The handbook covers the most basic assessment timethod research strategies, and the different methods that are relevant for a thorough under- concepts of consistency and method specificity. standing of human behavior, attitudes, and feelings. These include self-report (Lucas & Baird, this vol- Part II: Assessment Methods ume, chap. 3), informant assessment (Neyer, this The second part presents the current state of the volume, chap. 4), ability tests (Lubinski, this vol- art—the most important measurement methods in ume, chap. 8), implicit methods (Robinson & modern psychology, which comprise traditional Neighbors, this volume, chap. 9), observational methods like self-report as well as the most recent methods (Bakeman & Gnisci, this volume, chap. developments in brain imaging and Web-based 10), physiological and biochemical methods methods. Not all methods applied in psychology (Berntson & Cacioppo, this volume, chap. 12), can be presented, and selections had to be made. functional neuroimaging (Zald & Curtis, this vol-

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.