Just Plain Data Analysis: Finding, Presenting, and Interpreting Social Science Data, 2nd Edition

211 Pages·2012·6.92 MB·English

by  Gary M. Klass

Just Plain Data Analysis
JUST PLAIN
DATA ANALYSIS
Finding, Presenting, and
Interpreting Social Science Data
SECOND EDITION
G M. K
ARY LASS

ROWMAN & LITTLEFIELD PUBLISHERS, INC.
Lanham • Boulder • New York • Toronto • Plymouth, UK
Published by Rowman & Littlefield Publishers, Inc.
A wholly owned subsidiary of The Rowman & Littlefield Publishing Group, Inc.
4501 Forbes Boulevard, Suite 200, Lanham, Maryland 20706
www.rowman.com
10 Thornbury Road, Plymouth PL6 7PP, United Kingdom
Copyright © 2012 by Rowman & Littlefield Publishers, Inc.
All rights reserved. No part of this book may be reproduced in any form or by any electronic or mechanical
means, including information storage and retrieval systems, without written permission from the publisher,
except by a reviewer who may quote passages in a review.
British Library Cataloguing in Publication Information Available
Library of Congress Cataloging-in-Publication Data
Klass, Gary M., 1952-
Just plain data analysis : finding, presenting, and interpreting social science data / Gary M. Klass. —
2nd ed.
p. cm.
ISBN 978-1-4422-1507-8 (cloth : alk. paper) — ISBN 978-1-4422-1508-5
(pbk. : alk. paper) — ISBN 978-1-4422-1509-2 (electronic)
1. Statistics. 2. Social sciences—Statistics. 3. Social sciences—Statistical methods.
I. Title.
HA29.K58 2012
519.5—dc23 2012004619
The paper used in this publication meets the minimum requirements of American National Standard
for Information Sciences—Permanence of Paper for Printed Library Materials, ANSI/NISO Z39.48-1992.
Printed in the United States of America
To the thousands of Illinois State and
Illinois Wesleyan University students
who have spent their Saturday mornings
building eighteen Habitat for Humanity homes since 1995
Preface
What Is Just Plain Data Analysis?
“JUST PLAIN data analysis” is, simply, the compilation and presentation of
numerical evidence to support and illustrate arguments about politics and
public affairs.
There is a realm of public debate about society’s most contentious issues
where arguments are grounded in hard evidence and sound reasoning. Oft en this
evidence comes in the form of numerical measures of social conditions and of
the effectiveness of public policies and governing institutions. When contending
sides advance their causes by finding, presenting, and interpreting such evidence
with clear thinking, the quality of public debate and the chances of devising
effective solutions to society’s problems are greatly increased.
The contending sides in such debate are rarely dispassionate and oft en
present misleading evidence and deceptive reasoning, but the shortcomings of
such arguments are transparent to those who can apply critical thinking skills to
the evidence. This is oft en not the case in other realms of public debate,
prevalent in today’s broadcast media and, increasingly, in academic discourse,
where competing anecdotes and malign characterizations of the other side’s
motives are all too common.
Just plain data analysis is the most common form of quantitative social
science methodology, although the statistical literacy skills and knowledge it
entails are oft en not presented, or presented well, in social science research
methods and statistics textbooks. These skills involve finding, presenting, and
interpreting numerical information in the form of commonly used social,
political, and economic indicators. They are skills that students will find of
considerable practical use, both in their subsequent coursework and in their
future careers.
Just plain data analysis differs from what is commonly regarded as
quantitative social science methodology in that it usually does not involve
formal tests of theories, hypotheses, or null hypotheses. Rather than relying on
statistical analysis of a single dataset, just plain data analysis, at its best, involves
compiling all the relevant evidence from multiple data sources. Where
conventional approaches to quantitative social science analysis stress the
statistical analysis of data to model and test narrowly defined theories, just plain
data analysis stresses presenting and critically evaluating statistical data to
support arguments about social and political phenomena.
The best examples of just plain data analysis are found in many books that
advance comprehensive and data-based arguments about social issues written by
public intellectuals for a broad public audience. Often these works shape public
debate about critical public policy issues. Charles Murray’s 1984 book Losing
Ground, for example, presented evidence of rising welfare caseloads and
spending, undiminished poverty, and the breakdown of the two-parent family,
which shaped conservative attacks on American welfare programs and
eventually led to the dramatic welfare reform policies during the Clinton
administration.1 Employing much of the same method of analysis, Jeffrey
Sachs’s The End of Poverty, coming from a decidedly different ideological
perspective, addresses issues of global poverty and may serve much the same
role in spurring progressive solutions to the intractable poverty of third world
nations.2
At times, both sides of a public debate use the same evidence to draw
different conclusions. Thus, the annual State of the World3 report published by
the environmentalist organization Worldwatch Institute regularly describes the
deterioration of the world’s environment on a wide range of ecological
indicators. Bjorn Lomborg’s critique of that report in The Skeptical
Environmentalist4 counters withevidence that long-term trends in deforestation
and in food, energy, and raw material production generally do not support the
environmentalists’ dire predictions.
In The Politics of Rich and Poor, historian Kevin Phillips argued that the
Reagan administration policies were producing a new era of accelerating
concentration of wealth, paralleling that of the Gilded Age and the Roaring
Twenties. In tables and charts, Phillips presents statistic after statistic
demonstrating that the United States has the highest inequalities of wealth and
income in the developed world, the inequalities of wealth and income are
steadily increasing, the divergence in pay between corporate executives and their
employees is widening, and the rich are much richer and the poor and middle
class poorer.5 The theme, using the same Gilded Age metaphor and fifteen more
years of evidence, is repeated with fewer tables and no charts, but often with a
much more careful analysis of the statistical evidence in Paul Krugman’s The
Conscience of a Liberal.6
Robert Putnam’s Bowling Alone, arguing that America faces a critical decline
in social capital, is a classic example of just plain data analysis. Almost all of
Putnam’s analysis is grounded in quantitative data, from a wide variety of
sources, presented in charts and graphs. Putnam describes his strategy as
attempting to “triangulate among as many independent sources of information as
possible” based on the “core principle” that “no single source of data is flawless,
but the more numerous and diverse the sources, the less likely that they could all
be influenced by the same flaw.”7 Legions of social scientists applied Putnam’s
core ideas to many fields of scholarly research, and public officials regularly cite
his work in advancing new approaches to public issues.
No politician is as adept at the use of statistical evidence to support political
arguments as is former president Bill Clinton. Clinton’s most recent book is a
broad critique of Republican anti–government policy agendas, an accounting of
his own administration’s success (and some of Obama’s), and a detailed agenda
for addressing current economic conditions.8 Barely a page passes without the
presentation of some numerical evidence, including many of the same statistical
comparisons of the United States with other developed democracies contained in
this book. His selection, presentation, and interpretation of the data are both self-
congratulatory and partisan, but the effort challenges those who would disagree
to counter with alternative data and explanations of their own.
Works such as these and others addressing public issues as diverse as gun
control, the death penalty, racial and gender discrimination, national health care,
school vouchers, and immigration advance the argument on one side of the
public debate and oft en set the research agenda for additional social science
research.
Students in almost every field of study encounter just plain data analysis all
the time in the charts and tables presented in their textbooks and reading
assignments. It is a primary mode of communication in government and is found
in the studies, annual reports, and PowerPoint presentations of almost every
governmental agency and advocacy group.
Just plain data analysis also plays an important role in the private sector,
where compiling reliable and valid performance measures, effectively
communicating numerical information to customers and employees, and
evaluating data-based claims and conclusions are critical management talents.
This, combined with having the confidence to base decisions on good data
analysis even when tradition and conventional wisdom say otherwise, led to the
remarkable achievements of the general manager of the Oakland Athletics, Billy
Beane, depicted in Michael Lewis’s book (and movie) Moneyball.9
Because Beane went straight from high school to a mediocre career playing
professional baseball, he never had the opportunity to take a college statistics
course. Yet Beane taught himself the science of sabermetrics, essentially the
application of just plain data analysis to baseball statistics. As general manager
of the Oakland Athletics, he used the statistics to identify undervalued trade and
draft prospects (high school prospects were overvalued, walks were
undervalued) and to make decisions traditionally left up to the team’s manager
(sacrifice bunts and intentional walks are often a bad idea). In his eleven years as
general manager, Oakland achieved the fifth-best record in major league baseball
despite having one of the lowest payrolls.
FINDING, PRESENTING, AND INTERPRETING THE DATA
There are three tasks and skills involved in doing just plain data analysis that
traditional research methods courses and textbooks oft en neglect: finding,
presenting, and interpreting numerical evidence.
Finding the Data
With the advances in information technology over the past decade, there has
been a revolution in the amount and availability of statistical indicators provided
by governments and nongovernmental public and private organizations. In
addition to the volumes of data provided by the U.S. Census Bureau, many
federal departments now have their own statistics agency, such as the National
Center for Education Statistics, the Bureau of Justice Statistics, the National
Center for Health Statistics, the Bureau of Labor Statistics, and the Bureau of
Transportation Statistics, providing convenient online access to comprehensive
data collections and statistical reports. In recent years, the greatest growth in the
sheer quantity of statistical indicators has been in the field of education. The
mandated testing under the No Child Left Behind Act and the expansion of the
Department of Education’s National Assessment of Educational Progress ( The
Nation’s Report Card ) have produced massive databases of measures of the
performance of the nation’s schools that, for better or worse, have fundamentally
transformed the administration of educational institutions.
There has also been a significant growth in the quantity and quality of
comparative international data. The Organisation for Economic Co-operation
and Development (OECD) now provides a comprehensive range of
governmental, social, and economic data for developed nations. For developing
nations, the World Bank’s development of poverty indicators and measures of
business and economic conditions and the UN’s Millennium Development Goals
database have contributed greatly to public debate and analysis of national and
international policies affecting impoverished people around the world. With the
Trends in International Mathematics and Science Study (TIMSS) and the
Programme in International Student Assessment (PISA) international
educational achievement tests, rich databases of educational system conditions
and student performance are now easily accessible.
A similar growth has taken place in the availability of social indicator data
derived from nongovernmental public opinion surveys that offer consistent times
series and cross-national measures of public attitudes and social behaviors. Time
series indicators can be readily obtained online from the U.S. National Elections
Study and the National Opinion Research Center’s annual General Social
Survey, and comparative cross-national data indicators can be accessed from
Comparative Study of Electoral Systems, the International Social Survey
Programme, and World Values Survey.
Finding the best data relevant to the analysis of contemporary social and
political issues requires a basic familiarity with the kinds of data likely to be
available from these sources. Social science research methods courses oft en
give short shrift to this crucial stage of the research process, which involves
skills and expertise usually acquired by years of experience in specific fields of
study. Too oft en, the data are a given: the instructor gives a dataset to the
students and asks them to analyze it. Finding the best data to address a research
question requires that one understand the kinds of data that are likely to be
available, who collects the data, and where they can be found.
Presenting the Data
Good data presentation skills are to data-based analysis what good writing is to
literature, and some of the same basic principles apply to both. Poor graphical
and tabular presentations oft en lead both readers and writers to draw erroneous
conclusions from the data and obscure facts that better presentations would
reveal. Some of these practices involve deliberate distortions of data, but more
commonly they involve either unintentional distortions or simply ineffective
approaches to presenting numerical evidence.
The past two decades have seen the development of a substantial literature on
the art and science of data presentation, much of it following Edward R. Tufte’s
path-breaking work The Visual Display of Quantitative Information.10 With his
admonitions to “show the data,” “minimize the ink-to-data ratio,” and avoid
“ChartJunk,” Tufte established many of the basic rules and principles of data