Standard deviations: flawed assumptions, tortured data, and other ways to lie with statistics

474 Pages·2.121 MB·English

by  Smith, Gary

MORE ADVANCE PRAISE FOR STANDARD
DEVIATIONS
“Gary Smith’s Standard Deviations is both a statement
of principles for doing statistical inference correctly and
a practical guide for interpreting the (supposedly) data-
based inferences other people have drawn. Cleverly
written and engaging to read, the book is full of concrete
examples that make clear not just what Smith is saying
but why it matters. Readers will discover that lots of
what they thought they’d learned is wrong, and they’ll
understand why.”
—BENJAMIN M. FRIEDMAN
William Joseph Maier Professor of Political Economy,
Harvard University
“Standard Deviations shows in compelling fashion why
humans are so susceptible to the misuse of statistical
evidence and why this matters. I know of no other book
that explains important concepts such as selection bias in
such an entertaining and memorable manner.”
—RICHARD J. MURNANE
Thompson Professor of Education and Society, Harvard
Graduate School of Education
“We all learn in school that there are three kinds of lies:
lies, damn lies, and statistics. Gary Smith’s new book
imparts true substance to this point by setting forth
myriad examples of how and why statistics and data-
crunching at large are susceptible to corruption. The
2
great risk today is that the young will forget that
deductive logic is vastly more powerful than inductive
logic.”
—HORACE “WOODY” BROCK
President, Strategic Economic Decisions, Inc.
“Statistical reasoning is the most used and abused form
of rhetoric in the field of finance. Standard Deviations is
an approachable and effective means to arm oneself
against the onslaught statistical hyperbole in our modern
age. Professor Smith has done us all a tremendous
service.”
—BRYAN WHITE
Managing Director, BlackRock, Inc.
“It’s entertaining, it’s gossipy, it’s insightful—and it’s
destined to be a classic. Based on a lifetime of
experience unraveling the methodical blunders that
remain all too frequent, this book communicates Gary
Smith’s wisdom about how not to do a data analysis.
Smith’s engaging rendering of countless painful
mistakes will help readers avoid the pitfalls far better
than merely mastering theorems.”
—EDWARD E. LEAMER
Distinguished Professor and Chauncey J. Medberry
Chair in Management, UCLA
“Standard Deviations will teach you how not to be
deceived by lies masquerading as statistics. Written in an
entertaining style with contemporary examples, this
3
book should appeal to everyone, whether interested in
marriages or mortgages, the wealth of your family, or the
health of the economy. This should be required reading
for everyone living in this age of (too much?)
information.”
—ARTHUR BENJAMIN
Professor of Mathematics, Harvey Mudd College and
author of Secrets of Mental Math
4
STANDARD DEVIATIONS
5
STANDARD
DEVIATIONS
FLAWED ASSUMPTIONS,
TORTURED DATA, AND
OTHER WAYS TO LIE
WITH STATISTICS
GARY SMITH
DUCKWORTH OVERLOOK
6
This eBook edition 2014
First published in 2014 by
Duckworth Overlook
LONDON
30 Calvin Street, London E1 6NW
T: 020 7490 7300
E: [email protected]
www.ducknet.co.uk
For bulk and special sales, please contact
[email protected]
or write to us at the address above
NEW YORK
141 Wooster Street, New York, NY 10012
www.overlookpress.com
© 2014 by Gary Smith
All rights reserved. No part of this publication may be
reproduced, stored in a retrieval system, or transmitted,
in any form or by any means, electronic, mechanical,
photocopying, recording or otherwise, without the prior
permission of the publisher.
The right of Gary Smith to be identified as the Author of
the Work has been asserted by him in accordance with
the Copyright, Designs and Patents Act 1988.
A catalogue record for this book is available from the
British Library
7
ISBNs
Paperback: 978-0-7156-4914-5
Mobi: 978-0-7156-4973-2
ePub: 978-0-7156-4974-9
Library PDF: 978-0-7156-4975-6
8
To my wife Margaret and my children Josh, Jo, Chaska,
Cory, Cameron, and Claire
9
CONTENTS
INTRODUCTION
1. Patterns, Patterns, Patterns
2. Garbage In, Gospel Out
3. Apples and Prunes
4. Oops!
5. Graphical Gaffes
6. Common Nonsense
7. Confound It!
8. When You’re Hot, You’re Not
9. Regression
10.Even Steven
11.The Texas Sharpshooter
12.The Ultimate Procrastination
13.Serious Omissions
14.Flimsy Theories and Rotten Data
15.Don’t Confuse Me With Facts
16.Data Without Theory
17.Betting the Bank
18.Theory Without Data
19.When to Be Persuaded and When to Be Skeptical
SOURCES
INDEX
10