Understanding Elections through Statistics: Polling, Prediction, and Testing

226 Pages·2020·10.662 MB·English

by  Ole J. Forsberg

Understanding
Elections through
Statistics
Chapman & Hall/CRC
Statistics in the Social and Behavioral Sciences Series

Series Editors
Jeff Gill, Steven Heeringa, Wim J. van der Linden, Tom Snijders

Recently Published Titles

Multilevel Modelling Using Mplus
Holmes Finch and Jocelyn Bolin

Applied Survey Data Analysis, Second Edition
Steven G. Heering, Brady T. West, and Patricia A. Berglund

Adaptive Survey Design
Barry Schouten, Andy Peytchev, and James Wagner

Handbook of Item Response Theory, Volume One: Models
Wim J. van der Linden

Handbook of Item Response Theory, Volume Two: Statistical Tools
Wim J. van der Linden

Handbook of Item Response Theory, Volume Three: Applications
Wim J. van der Linden

Bayesian Demographic Estimation and Forecasting
John Bryant and Junni L. Zhang

Multivariate Analysis in the Behavioral Sciences, Second Edition
Kimmo Vehkalahti and Brian S. Everitt

Analysis of Integrated Data
Li-Chun Zhang and Raymond L. Chambers

Multilevel Modeling Using R, Second Edition
W. Holmes Finch, Joselyn E. Bolin, and Ken Kelley

Modelling Spatial and Spatial-Temporal Data: A Bayesian Approach
Robert Haining and Guangquan Li

Measurement Models for Psychological Attributes
Klaas Sijtsma and Andries van der Ark

Handbook of Automated Scoring: Theory into Practice
Duanli Yan, André A. Rupp, and Peter W. Foltz

Interviewer Effects from a Total Survey Error Perspective
Kristen Olson, Jolene D. Smyth, Jennifer Dykema, Allyson Holbrook, Frauke Kreuter,
and Brady T. West

Understanding Elections through Statistics: Polling, Prediction, and Testing
Ole J. Forsberg

Big Data and Social Science: Data Science Methods and Tools for Research and Practice,
Second Edition
Ian Foster, Rayid Ghani, Ron S. Jarmin, Frauke Kreuter and Julia Lane

Analyzing Spatial Models of Choice and Judgment, Second Edition
David A. Armstrong II, Ryan Bakker, Royce Carroll, Christopher Hare, Keith T. Poole,
and Howard Rosenthal

For more information about this series, please visit: https://www.routledge.com/Chapman--
HallCRC-Statistics-in-the-Social-and-Behavioral-Sciences/book-series/CHSTSOBESCI

Information Classification: General
Understanding
Elections through
Statistics
Polling, Prediction, and Testing
Ole J. Forsberg, PhD
Knox College
Department of Mathematics
Galesburg, Illinois, USA
Firsteditionpublished2021
byCRCPress
6000BrokenSoundParkwayNW,Suite300,BocaRaton,FL33487-2742
andbyCRCPress
2ParkSquare,MiltonPark,Abingdon,Oxon,OX144RN
(cid:13)c 2021Taylor&FrancisGroup,LLC
CRCPressisanimprintofTaylor&FrancisGroup,LLC
Reasonableeffortshavebeenmadetopublishreliabledataandinformation,buttheauthor
and publisher cannot assume responsibility for the validity of all materials or the conse-
quences of their use. The authors and publishers have attempted to trace the copyright
holdersofallmaterialreproducedinthispublicationandapologizetocopyrightholdersif
permissiontopublishinthisformhasnotbeenobtained.Ifanycopyrightmaterialhasnot
beenacknowledgedpleasewriteandletusknowsowemayrectifyinanyfuturereprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted,
reproduced, transmitted, or utilized in any form by any electronic, mechanical, or other
means,nowknownorhereafterinvented,includingphotocopying,microfilming,andrecord-
ing,orinanyinformationstorageorretrievalsystem,withoutwrittenpermissionfromthe
publishers.
For permission to photocopy or use material electronically from this work, access
www.copyright.com or contact the Copyright Clearance Center, Inc. (CCC), 222 Rose-
wood Drive, Danvers, MA 01923, 978-750-8400. For works that are not available on CCC
[email protected]
Trademark Notice: Product or corporate names may be trademarks or registered trade-
marks,andareusedonlyforidentificationandexplanationwithoutintenttoinfringe.
Library of Congress Cataloging-in-Publication Data
Names:Forsberg,OleJ.,author.
Title:Understandingelectionsthroughstatistics:polling,prediction,
andtesting/OleJ.Forsberg.
Description:Firstedition.|BocaRaton,FL:CRCPress,2021.|Series:
Chapman&Hall/CRCstatisticsinthesocialandbehavioralsciences|
Includesbibliographicalreferencesandindex.|Contents:Polling--
Polling--Combiningpolls--In-depthanalysis:Brexit--Digittests--
Differentialinvalidation--Consideringgeography--In-depth
analysis:SriLankasince1994.|Summary:“Writtenforthosewithonly
abriefintroductiontostatistics,thisbooktakesyouonastatistical
journeyfromhowpollsaretakentohowtheycan-andshould-beusedto
estimatecurrentpopularopinion.Onceanunderstandingoftheelection
processisbuilt,weturntowardstestingelectionsforevidenceof
unfairness.Whileholdingelectionshasbecomethedefactoproofof
governmentlegitimacy,thoseelectoralprocessesmayhidethedirty
littlesecretofthegovernmentillicitlyensuringafavorableelection
outcome”--Providedbypublisher.
Identifiers:LCCN2020025312|ISBN9780367895358(paperback)|ISBN
9780367895372(hardback)|ISBN9781003019695(ebook)
Subjects:LCSH:Elections--Statistics.|Votingresearch.
Classification:LCCJQ1694.F672021|DDC323.6072/7--dc23
LCrecordavailableathttps://lccn.loc.gov/2020025312
ISBN:9780367895372(hbk)
ISBN:9780367895358(pbk)
ISBN:9781003019695(ebk)
TypesetinLMRoman
byNovaTechsetPrivateLimited,Bengaluru&Chennai,India
I dedicate this book to my family.

Contents
Preface ix
Acknowledgments xiii
About the Author xv
Part I Estimating Electoral Support 1
1 Polling 101 3
1.1 Simple Random Sampling . . . . . . . . . . . . . . . . . . . . 5
1.2 One Estimator of π: The Sample Proportion . . . . . . . . . 6
1.3 Reasonable Values of π . . . . . . . . . . . . . . . . . . . . . 12
1.4 A Second Estimator of π: Agresti-Coull . . . . . . . . . . . . 21
1.5 SRS without Replacement . . . . . . . . . . . . . . . . . . . 26
1.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
1.7 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
1.8 Chapter Appendix . . . . . . . . . . . . . . . . . . . . . . . . 30
2 Polling 399 33
2.1 Stratified Sampling . . . . . . . . . . . . . . . . . . . . . . . 35
2.2 The Mathematics of Estimating π . . . . . . . . . . . . . . . 38
2.3 Confidence Intervals . . . . . . . . . . . . . . . . . . . . . . . 49
2.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.5 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.6 Chapter Appendix . . . . . . . . . . . . . . . . . . . . . . . . 54
3 Combining Polls 57
3.1 Simple Averaging of Polls . . . . . . . . . . . . . . . . . . . . 59
3.2 Weighted Averaging of Polls . . . . . . . . . . . . . . . . . . 61
3.3 Averaging of Polls over Time . . . . . . . . . . . . . . . . . . 64
3.4 Looking Ahead . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.5 South Korean 2017 Presidential Election . . . . . . . . . . . 73
3.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77
3.7 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
3.8 Chapter Appendix . . . . . . . . . . . . . . . . . . . . . . . . 79
vii
viii Contents
4 In-Depth Analysis: Brexit 2016 83
4.1 Knowing Your Data . . . . . . . . . . . . . . . . . . . . . . . 85
4.2 Combining the Polls . . . . . . . . . . . . . . . . . . . . . . . 89
4.3 Discussion: What Went Wrong? . . . . . . . . . . . . . . . . 90
4.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
Part II Testing Election Results 94
5 Digit Tests 95
5.1 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
5.2 The Benford Test . . . . . . . . . . . . . . . . . . . . . . . . 99
5.3 The Generalized Benford Test . . . . . . . . . . . . . . . . . 108
5.4 Using the Generalized Benford Distribution . . . . . . . . . . 112
5.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.6 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
5.7 Chapter Appendix . . . . . . . . . . . . . . . . . . . . . . . . 121
6 Differential Invalidation 125
6.1 Differential Invalidation . . . . . . . . . . . . . . . . . . . . . 127
6.2 Regression Modeling . . . . . . . . . . . . . . . . . . . . . . . 132
6.3 Examining Côte d0Ivoire . . . . . . . . . . . . . . . . . . . . 141
6.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
6.5 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.6 Chapter Appendix . . . . . . . . . . . . . . . . . . . . . . . . 145
7 Considering Geography 149
7.1 Detecting Spatial Correlation . . . . . . . . . . . . . . . . . . 151
7.2 The Spatial Lag Model . . . . . . . . . . . . . . . . . . . . . 158
7.3 Casetti’s Spatial Expansion Model (SEM) . . . . . . . . . . . 161
7.4 Geographically Weighted Regression . . . . . . . . . . . . . . 164
7.5 The Spatial Lagged Expansion Method . . . . . . . . . . . . 169
7.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172
7.7 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
7.8 Chapter Appendix . . . . . . . . . . . . . . . . . . . . . . . . 174
8 In-Depth Analysis: Sri Lanka since 1994 177
8.1 Differential Invalidation . . . . . . . . . . . . . . . . . . . . . 179
8.2 Methods and Data . . . . . . . . . . . . . . . . . . . . . . . . 179
8.3 Results by Election . . . . . . . . . . . . . . . . . . . . . . . 182
8.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192
8.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
Bibliography 195
Index 207
Preface
Elections hold a special place in my heart. As a political scientist, I see elec-
tions as the outward manifestation of the hopes, dreams, and aspirations of a
people toward their government, their society, and their futures. As a statis-
tician, I see them as a random process producing reams of data that should
be describable, predictable, and testable.
In both cases, elections tend to fall short of those ideals. They tend to
be expressions of our fears instead of our hopes. They tend to be heavily
influenced, both indirectly in the form of the media, social and not — and
directly in the form of the governments that hold the elections, count the
ballots,andreporttheresults.AsNicaraguanleaderAnastasioSomozastated
in an interview with the London Guardian [43]:
Indeed, you won the elections, but I won the count.
Electionsare randomvariables,buttheyarerandomvariableswithoutknown
(or knowable?) distributions except in the simplest cases with the strongest
assumptions. This makes testing elections for direct government intervention
difficult, to say the least.
And yet, here I am writing a book dedicated to the proposition that elec-
tions can be statistically understood, with that understanding giving us a
deeper insight into ourselves and what we want our future to look like.
Audience for this Book
Toachievemygoalsinthisbook,IdecidedonasbroadanaudienceasIcould
reach.Tobeclear,thisisnotagraduate-leveltextbook.Myaudienceisthose
who have had some experience with statistics. This exposure could be from
anadvancedhighschoolcourseoranintroductorycollegecourseinstatistics.
It may also come from extensive experience in employment, such as through
professional journalism covering elections and polls.
Thus, I envision this book to be accessible to those who have already had
anintroductiontostatisticalthinkingintermsoftheideasbehindhypothesis
testing. This means having an exposure to test statistics and the p-value.
Thepointofthebookistoobtainabetterunderstandingofelections,from
the polls to the predictions to the testing. This requires some mathematics.
However, the amount of mathematics required depends on the goals of the
reader. The more you want to understand polling, the more mathematics you
ix