Political Science Research Methods Eighth Edition Janet ButtolphJohnson H. T. Reynolds Jason D. Mycoff University of Delaware c!!!s ($)SAGE I Los Angeles I London I New Delhi Singapore I Washington DC Iii ($)SAGEI COPRESS Los AngelesI London I New Delhi SingaporeI W ashingtonD C FORI NFORMATION: Copyright© 2016 by CQ Press, an imprint of SAGE Publications, Inc. CQ Press is a registered trademark of Congressional Quarterly Inc. ' CQP ress An Imprinto f SAGEP ublicationsIn, c. All rights reserved. No part.of this book may be reproduced or 2455T ellerR oad utilized in any form or by any means, electronic or mechanical, ThousandO aks,C alifornia9 1320 including photocopying, recording, or by any information E-mailo: [email protected] storage and retrieval system, without permission in writing from the publisher. SAGEP ublicationLst d. 1 Oliver'sY ard 55 CityR oad LondonE C1Y1 S P UnitedK ingdom Printed in Canada SAGEP ublicationIsn diaP vt. Ltd. B 1/11M ohanC ooperativeIn dustriaAl rea MathuraR oad,N ewD elhi1 100 44 India SAGEP ublicationAs sia-PacificP te.L td. 3 ChurchS treet ISBN 978-1-5063-0782-4 #10-04S amsungH ub Singapore0 49483 AcquisitionEs ditor: SarahC alabi This book is printed on acid-free paper. DevelopmenEtd itor: NancyM atuszak eLearningE ditor: AllisonH ughes EditoriaAl ssistant: RaqueCl hristie ProductionE ditor: OliviaW eber-Stenis CopyE ditor: TaliaG reenberg Typesetter:C &MD igitals( P)L td. Proofreader:T heresaK ay Indexer: SheilaB odell CoverD esigner: GailB uschman MarketingM anager: AmyW hitaker 15 16 17 1819 10 9 8 7 6 5 4 3 2 1 ----• Brief Contents Tables, Figures, and Features xi Prefa ce xvii About the Authors xxiii Chapter 1 • Introduction 1 Chapter 2 • ·The Empirical Approach to Political Science 46 Chapter 3 • Beginning the Research Process: Identifying a Research Topic, Developing Research Questions, and Reviewing the Literature 7 4 Chapter 4 • The Building Blocks of Social Scientific Research: Hypotheses, Concepts, and Variables 104 Chapter 5 • The Building Blocks of Social Scientific Research: Measurement 128 Chapter 6 • Research Design: Making Causal Inferences 166 Chapter 7 • Sampling 212 Chapter 8 • Making Empirical Observations: Firsthand Observation 244 Chapter 9 • Document Analysis: Using the Written Record 268 Chapter 10 • Survey Research and Interviewing 294 Chapter 11 • Making Sense of Data: First Steps 348 Chapter 12 • Statistical Inference 388 Chapter 13 • Investigating Relationships between Two Variables 412 Chapter 14 • Multivariate Analysis 516 Chapter 15 • The Research Report: An Annotated Example 584 Appendixes 605 Glossary 617 Index 625 ----• Detailed Contents Tables,F igures,a nd Features xi Self-Reflection and Individuality 61 Preface xvii Is Political Science Trivial or Irrelevant? 63 Aboutt he Authors xxiii Competing Points of View 64 Interpretation 65 Chapter 1 • Introduction 1 Constructionism and Critical Theory 67 Conclusion 71 Researcho n Inequality 6 Terms Introduced 71 Who Votes?W ho Doesn't? 14 SuggestedR eadings 72 Politics and the Gender Gap 18 Repressiono f Human Rights 22 Chapter 3 • Beginning the A Look into Judicial Decision Research Process: Identifying Making and Its Effects 25 a Research Topic, Developing Influencing Bureaucracies 31 Research Questions, and Effects of CampaignA dvertising Reviewing the Literature 74 on Voters 36 Researcho n Public Support for Specifying the ResearchQ uestion 74 US Foreign Involvement 40 Sourceso f Ideas for ResearchT opics 78 Conclusion 44 Why Conduct a Literature Review? 82 Terms Introduced 45 CollectingS ourcesf or a LiteratureR eview8 4 Identifying the Relevant Scholarly Chapter 2 • The Empirical Literature 85 Approach to Political Science 46 Managing Citations 91 Identifying Useful Popular Sources 91 Elementso f Empiricism 48 Reading the Literature 93 The Importanceo f Theory 54 Writing a Literature Review 94 An Example: Proximity ·Theory of Anatomyo f a Literature Review 97 Voting 54 Conclusion 101 The Explanatory Range of Theories 58 Terms Introduced 102 A Brief Overviewo f the Empirical SuggestedR eadings 102 ResearchP rocess 58 Development of an Idea to Investigate Chapter 4 • The Building Blocks or a Problem to Solve 59 of Social Scientific Research: Hypothesis Formation 59 Hypotheses, Concepts, and "Data" Collection 59 Variables 104 Interpretation and Decision 60 Modification and Extension 60 ProposingE xplanations 104 Reactionst o the Empirical Approach: Formulating Hypothes~s 109 Practical Objections 60 Characteristics of Good Hypotheses 111 Specifying Units of Analysis 116 Multiple-Group Design 182 Cross-Level Analysis: Ecological Randomized Field Experiments 183 Inference and Ecological Fallacy 117 NonrandomizedD esigns: Defining Concepts 121 Quasi-Experiments 185 Conclusion 126 Natural Experiments 188 Terms Introduced 126 Intervention Analysis 191 Suggested Readings 127 ObservationalS tudies 195 Small-N Designs 196 Chapter 5 • The Building Cross-Sectional Designs: Blocks of Social Scientific Surveys and Aggregate Analysis 203 Research: Measurement 128 Longitudinal (Time Series) Designs 206 Devising MeasurementS trategies 130 Conclusion 209 Terms Introduced 210 Exampleso f Political Measurements: Getting to Operationalization 132 SuggestedR eadings 211 The Accuracy of Measurements 135 Chapter 7 • SampliQg 212 Reliability 135 Validity 138 The Basics of Sampling 213 Problems with Reliability and Validity Population or Sample? 215 in Political Science Measurement 143 Fundamental Con.cepts 216 The Precision of Measurements 146 Typeso f Samples 219 Levels of Measurement 146 Simple Random Samples 220 Working with Precision: i:oo Little or Systematic Samples 221 Too Much 150 Stratified Samples 223 Multi-Item Measures 151 Cluster Samples 225 Indexes 151 Nonprobability Samples 228 Scales 155 What Can Be Learned from a Conclusion 162 Sample of a Population 230 Terms Introduced 163 Expected Values 232 Suggested Readings 164 Measuring the Variability of the Estimates: Standard Errors 234 Chapter 6 • Research Design: Sample Size 237 Making Causal Inferences 166 Conclusion 240 Terms Introduced 241 Verifying Causal Assertions 167 SuggestedR eadings 242 Causal versus Spurious Relationships 167 ' . The Classical Randomized Chapter 8 • Making Empirical Experiment 171 Observations: Firsthand The Power of Random Assignment 176 Observation 244 Internal Validity 176 External Validity 179 Typeso f Data and Collection / Other Randomized Experiments 179 Techniques. 244 Posttest Design 180 Choosing among Data·Collection Repeated-Measurement Design 181 Methods 245 Firsthand, Direct Obsen(ation 248 Characteristics of Surveys 304· Direct Observation in a Natural Setting 249 Response Quality 316 Structured and Unstructured Observation 254 Survey Type and Response Quality 320 Covert and Overt Observation 256 Interlude: The Health Care Debate and Note-Taking as Data Collection 257 Why Question Wording Matters 322 Direct Observation in a Laboratory Question Wording 327 Setting 258 Question Type 330 Firsthand, Indirect Observation 259 Question Order 334 Physical Trace Measures 260 Questionnaire Design 336 Validity Problems with Indirect Using Archived Surveys 337 Observation 261 Advantages of Using Archived Ethical Issues in Observation 262 Surveys 337 Conclusion 265 Interviewing 338 Terms Introduced 266 The Ins and Outs of Interviewing 340 Suggested Readings 267 Conclusion 344 Terms Introduced 345 Chapter 9 • Document Analysis: Suggested Readings 345 Using the Written Record 268 Chapter 11 • Making Sense Content Analysis 270 of Data: First Steps 348 Content Analysis Procedures 270 The Data Matrix 349 A Simple Computer Content Analysis 275 Data Description and Exploration 351 Types of Written Records 275 Frequency Distributions, Proportions, The Running Record 275 and Percentages 352 The Episodic Record 279 Descriptive Statistics 355 The Running Record and Episodic Measures of Central Tendency 356 Record Compared 284 Measures of Variability or Presidential Job Approval 286 Dispersion 361 Advantages and Disadvantages of the Graphs for Presentation and Written Record 287 Exploration 369 Conclusion 292 Designing and Drawing Graphs 371 Terms Introduced 293 Bar Charts 375 Suggested Readings 293 ' Exploratory Graphs 375 What's Next? 384 Chapter 10 • Survey Research Terms Introduced 385 and Interviewing 294 Suggested Readings 386 Fundamentals: Ensuring Validity and Chapter 12 • Statistical Inference 388 Reliability 296 Survey Research 299 Two Kinds of Inferential Activities 389 Types of Surveys 299 Hypothesis Testing 389 American Association for Public Opinion Steps for Hypothesis Testing 391 Research's Transparency Initiative 300 Significance Tests of a Mean 393 Testing Hypotheses about Proportions 401 Inference for Regression Parameters: Confidence Intervals and Confidence Tests and Confidence Intervals 499 Levels 405 Case Studies in Two-Variable Conclusion 409 Regression 502 Terms Introduced 410 Conclusion 511 SuggestedR eadings 410 Terms Introduced 513 SuggestedR eadings 515 Chapter 13 • Investigating Chapter 14 • Multivariate Analysis 516 Relationships between Two Variables 412 Holding a Variable Constant 518 Multivariate Analysis of Categorical The Basics of Identifying and Measuring Relationships 415 Data 518 Level of Measurement 415 Linear Models 528 Types of Relationships 416 Multiple Regression Analysis 528 Strength of Relationship 418 Interpretation of Parameters 531 Numerical and Graphical Examining Residuals 533 Procedures 419 Statistical Tests 534 Table Summaries of Categorical CategoricaVl ariablesa nd Linear Models 541 Variable Associations 420 Does a Model Fit? ANOVA and Dummy Variable Regression 547 MeasuringS trength of Relationships Models with Quantitative and Categorical in Tables 426 Variables: Interaction and Direction of a Relationship 428 Analysis of Covariance (ANCOVA) 548 Coefficients for Ordinal Variables 431 Interaction 550 A Coefficient for Nominal Data 438 Standardized Regression Coefficients 553 Testing a Cross-Tabulation for Measuring the Goodness of Fit, R2 558 Statistical Significance 441 Logistic Regression 559 The Relationship between a Ways of Thinking about Dichotomous Categorical DependentV ariable Dependent Variables 563 and a Quantitative Variable 449 Estimating the Model's Coefficients 568 Difference of Means and Measures of Fit 570 Effect Sizes 450 Significance Tests 571 Difference of Proportions 465 An Alternative Interpretation of Logistic Analysis of Variance CANOVA) 467 Regression Coefficients 575 RegressionA nalysis 478 Logits 581 Basic Assumptions 479 Conclusion 581 Scatterplots 481 Terms Introduced 582 The Linear Regression Model 483 SuggestedR eadings 582 Regression Basics 484 Measuring the Fit of a Regression Chapter 15 • The Research Line: R2 490 Report: An Annotated Example 584 The Correlation Coefficient 492 Standardized Regression Coefficients 496 Annotated ResearchR eport Example 584 . . Appendixes 605 Appendix D: F Distribution 611 Appendix A: Normal Curve Tail Glossary 617 Probabilities 605 Index 625 Appendix B: Critical Values from t Distribution 607 Appendix C: Chi-Squared Distribution Values for Various Right-Tail Probabilities 609 ----• Tables, Figures, and Features Tables 6-5 Mail Survey Incentive Experiment 183 6-6 Results from Hypothetical Quasi- 1-1 Independent Expenditures, 2000-2014 39 Experiment 188 1-2 Casualty Sensitivity by Mission 6-7 Haiti and Dominican Republic Type and Inequality Cue 43 Compared 190 6-8 Natural Experiment "Results" 191 2-1 Methodological Perspectives in 6-9 Randomized and Quasi-Experiments Political Science 70 Compared 195 4-1 Voting by Democrats, Republicans, 6-10 Some Observational and Statistical and Others for a Sales Tax Increase 120 Designs 197 4-2 Concept Development: The 6-11 Mills Method of Difference 201 Relationship between Economic Development and Democracy 124 7-1 Fifty Random Numbers 221 7-2 An Abbreviated List of Supreme Court 5-1 Correlations of Leadership Power Nominees, 1787-2011 222 Measures 142 7-3 Stratified Sample of Student Majors 226 5-2 Interitem Association Validity Test 7-4 Properties of Samples of Different Sizes 239 of a Measure of Liberalism 143 5-3 Hypothetical Index for Measuring 9-1 Policy Agendas Project Policy Topics 280 Freedom in Countries 152 9-2 Policy Agendas Project Datasets 281 5-4 Provisional Likert Scale to Measure 9-3 Presidential Personality Types 284 Concept of Liberalism 156 5-5 Guttman Scale of Attitudes toward Abortion 158 10-1 Types and Characteristics of Surveys 302 5-6 Items Measuring Landowner Attitudes toward Riparian Buffers on Their 11-1 Inequality and Socioeconomic Measures Streamside Property, Sorted into for Twenty-One Developed Democracies 350 Dimensions Using Factor Analysis 161 11-2 Frequency Distribution: Beliefs about Power in the United States 352 6-1 Voting Intention by Ad Exposure 169 11-3 The Effect of Missing Data on 6-2 Results of Hypothetical Media Percentages 354 Experiment 175 11-4 The Effect of Missing Data on 6-3 Pre- and Posttest Scores in Non- Percentages: Large Effect 355 Repeated-Measurement Experiment 181 11-5 Hypothetical Incomes in Two 6-4 Pre- and Posttest Scores in Repeated- Communities 357 Measurement Experiment 182 11-6 Calculation. of the Median 359 xi .. xii POLITICALS CIENCER ESEARCHM ETHODS 11-7 Deviations from the Mean 364 13-13 2008 Presidential Vote by Party 436 11-8 Calculation of the Variation and 13-14 Gays in the Military: A Gender Gap? 437 Standard Deviation for Three 13-15 Voter Turnout by Gender 443 Hypothetical Sample Populations 365 13-16 Voter Turnout by Social Class 443 11-9 Summary of Descriptive Statistjcs 368 13-17 Opinion on Civil Liberties: 11-10 Typical Presentation and Tolerance of Dissent 444 Exploratory Graphs 384 13-18 Observed and Expected Values under Hypothesis of Independence 446 12-1 Types of Inferential Errors 391 13-19 Relationship between X and Y Based 12-2 Large-Sample 2-Tailed Test of a Mean 398 · on Sample of 300 447 12-3 Large-Sample Test of a Proportion 403 13-20 Relationship between X and Y Based 12-4 Confidence Intervals Calculated for on Sample of 3,000 448 Four Different Confidence Levels 408 13-21 Large-Sample Difference-of-Means 12-5 Confidence Intervals for Various Test (Gender by Attitudes toward Sample Sizes at 95 Percent Level the Military) 456 of Confidence 408 13-22 Small-Sample Difference-of-Means Test 460 13-1 Levels of Measurement and Statistical . 13-23 Difference of Proportions Test Procedures: A Summary 416 ( Gender by Attitudes toward 13-2 Level of Partisanship: Donating "Feminists") 466 Money 421 13-24 Measurements on Y within 13-3 Level of Partisanship: Finding Three Categories of X 469 Compromise versus Sticking to 13-25 Typical ANOVAT able Format 472 Your Convictions 424 13-26 ANOVA Results 473 13-4 Cross-Tabulation of Gender by 13-27 ANOVAT able and F-Test 477 Party Identification 425 13-28 Predicted Gini Scores 489 13-5 Row Percentages Are Not the 13-29 Regression Sums of Square and Same as Column.Percentages 426 R-Squared and r 492 13-6 Example of a Nil Relationship 13-30 Raw and Standardized Union Density between Region and Opinions about Scores 498 Comprehensive Immigration Reform 427 13-31 Example of Raw Scores Converted 13-7 Example of a Perfect Relationship to Standardized Values 499 between Region and Comprehensive 13-32 Regression of Gini on Union Density 501 Immigration Reform 428 13-33 Summary Statistics for GDP and 13-8 Attitudes toward Gun Control by Literacy in Ninety-Seven Countries 503 Liberalism 429 13-34 Regression Results: Literacy on Income 505 13-9 Table with Concordant, Discordant, 13-35 Regression Estimates for Two Groups 507 and Tied Pairs 432 13-36 Supreme Court Justices 512 13-10 Perfect Positive and Negative 13-37 The Effects of Case Deletion 513 Relationships 434 13-11 Perfect Monotonic Relationship 435 14-1 Relationship between Attitudes 13-12 Perfect bul Not Monotonic toward Government Spending and Relationship 435 Presidential Vote 520