Evidence-based Technical Analysis: Applying the Scientific Method and Statistical Inference to Trading Signals

523 Pages·2007·2 MB·English

by  David R. Aronson

Evidence-Based
Technical
Analysis
Applying the Scientific
Method and
Statistical Inference
to Trading Signals
DAVID R. ARONSON
John Wiley &Sons, Inc.
Copyright © 2007 by David R. Aronson. All rights reserved.
Published by John Wiley & Sons, Inc., Hoboken, New Jersey.
Published simultaneously in Canada.
Library of Congress Cataloging-in-Publication Data:
Aronson, David R., 1945–
Evidence-based technical analysis : applying the scientific method and
statistical inference to trading signals / David R. Aronson.
p. cm.—(Wiley trading series)
Includes bibliographical references and index.
ISBN-13: 978-0-470-00874-4 (cloth)
ISBN-10: 0-470-00874-1 (cloth)
1. Investment analysis. I. Title. II. Series.
HG4529.A77 2007
332.63'2042—dc22
2006014664
Printed in the United States of America
Contents
Acknowledgments ix
About the Author xi
Introduction 1
PART I Methodological, Psychological,
Philosophical, and Statistical
Foundations
CHAPTER 1 Objective Rules and Their Evaluation 15
CHAPTER 2 The Illusory Validity of Subjective
Technical Analysis 33
CHAPTER 3 The Scientific Method and Technical
Analysis 103
CHAPTER 4 Statistical Analysis 165
CHAPTER 5 Hypothesis Tests and Confidence Intervals 217
CHAPTER 6 Data-Mining Bias: The Fool’s Gold
of Objective TA 255
CHAPTER 7 Theories of Nonrandom Price Motion 331
PART II Case Study: Signal Rules
for the S&P 500 Index
CHAPTER 8 Case Study of Rule Data Mining
for the S&P 500 389
CHAPTER 9 Case Study Results and the Future of TA 441
APPENDIX Proof That Detrending Is Equivalent to
Benchmarking Based on Position Bias 475
Notes 477
Index 517
Introduction
T
echnical analysis (TA) is the study of recurring patterns in financial
market data with the intent of forecasting future price movements.1
It is comprised of numerous analysis methods, patterns, signals, indi-
cators, and trading strategies, each with its own cheerleaders claiming
that their approach works.
Much of popular or traditional TA stands where medicine stood before
it evolved from a faith-based folk art into a practice based on science. Its
claims are supported by colorful narratives and carefully chosen (cherry
picked) anecdotes rather than objective statistical evidence.
This book’s central contention is that TA must evolve into a rigorous
observational science if it is to deliver on its claims and remain relevant.
The scientific method is the only rational way to extract useful knowledge
from market data and the only rational approach for determining which
TA methods have predictive power. I call this evidence-based technical
analysis (EBTA). Grounded in objective observation and statistical infer-
ence (i.e., the scientific method), EBTA charts a course between the magi-
cal thinking and gullibility of a true believer and the relentless doubt of a
random walker.
Approaching TA, or any discipline for that matter, in a scientific man-
ner is not easy. Scientific conclusions frequently conflict with what seems
intuitively obvious. To early humans it seemed obvious that the sun cir-
cled the earth. It took science to demonstrate that this intuition was
wrong. An informal, intuitive approach to knowledge acquisition is espe-
cially likely to result in erroneous beliefs when phenomena are complex
or highly random, two prominent features of financial market behavior.
Although the scientific method is not guaranteed to extract gold from the
mountains of market data, an unscientific approach is almost certain to
produce fool’s gold.
This book’s second contention is that much of the wisdom comprising
the popular version of TA does not qualify as legitimate knowledge.
2 INTRODUCTION
KEY DEFINITIONS: PROPOSITIONS AND CLAIMS,
BELIEF AND KNOWLEDGE
I have already used the terms knowledge and belief but have not rigor-
ously defined them. These and several other key terms will be used re-
peatedly in this book, so some formal definitions are needed.
The fundamental building block of knowledge is a declarative state-
ment, also known as a claimor a proposition. A declarative statement is
one of four types of utterances that also include exclamations, questions,
and commands. Declarative statements are distinguished from the others
in that they have truth value. That is to say, they can be characterized as
either true or false or probably true or probably false.
The statement “Oranges are on sale at the supermarket for five cents a
dozen” is declarative. It makes a claim about a state of affairs existing at
the local market. It may be true or false. In contrast, the exclamatory
statement “Holy cow, what a deal,” the command “Go buy me a dozen,” or
the question “What is an orange?” cannot be called true or false.
Our inquiry into TA will be concerned with declarative statements,
such as, “Rule X has predictive power.” Our goal is to determine which of
these declarative statements warrant our belief.
What does it mean to say, “I believe X.”? “With regard to states of af-
fairs in general (i.e., ‘matters of fact’ or ‘what will happen’) believing X
amounts to expecting to experience X if and when we are in a position to
do so.”2 Therefore, if I believe the claim that oranges are on sale for five
cents a dozen, it means that I expect to be able to buy oranges for five
cents a dozen if I go to the store. However, the command to buy some or-
anges or the exclamation that I am happy about the opportunity, set up no
such expectation.
What does all this all means for us? For any statement to even be con-
sidered as a candidate for belief, it must “assert some state of affairs that
can be expected.3 Such statements are said to have cognitive content—
they convey something that can be known. “If the statement contains
nothing to know then there is nothing there to be believe.”4
Although all declarative statements presumably have cognitive con-
tent, not all actually do. This is not a problem if the lack of cognitive
content is obvious, for example, the declaration “The square root of
Tuesday is a prime number.”5 This utterance is, on its face, nonsense.
There are other declarative statements, however, whose lack of cogni-
tive content is not so obvious. This can be a problem, because such
statements can fool us into thinking that a claim has been made that
sets up an expectation, when, in fact, no claim has really been put for-
ward. These pseudo-declarative-statements are essentially meaningless
claimsor empty propositions.
Introduction 3
Although meaningless claims are not valid candidates for belief, this
does not stop many people from believing in them. The vague predictions
made in the daily astrology column or the nebulous promises made by
promoters of bogus health cures are examples of meaningless claims.
Those who believe these empty propositions simply do not realize that
what they have been told has no cognitive content.
A way to tell if a statement has cognitive content and is, thus, a valid
candidate for belief is the discernible-difference test6 described by Hall.
“Utterances with cognitive content make claims that are either true or
false; and whether they are true or false makes a difference that can be
discerned. That is why these utterances offer something to believe and
why there is no point in trying to believe an utterance that makes no such
offer”7In other words, a proposition that passes the discernible-difference
test sets up an expectation such that the state of affairs, if the statement
were true, is recognizably different from the state of affairs, if the state-
ment were false.
The discernible-difference criterion can be applied to statements pur-
porting to be predictions. A prediction is a claim to know something about
the future. If a prediction has cognitive content, it will be clearly dis-
cernible in the outcome if the prediction was accurate or not. Many, if not
most, of the forecasts issued by practitioners of popular TA are devoid of
cognitive content on these grounds. In other words, the predictions are
typically too vague to ever determine if they were wrong.
The truth or falsity of the claim oranges are on sale for five cents a
dozen will make a discernible difference when I get to the market. It is
this discernible difference that allows the claim to be tested. As will be de-
scribed in Chapter 3, testing a claim on the basis of a discernible differ-
ence is central to the scientific method.
Hall, in his book Practically Profound, explains why he finds
Freudian psychoanalysis to be meaningless when examined in light of the
discernible-difference test.
“Certain Freudian claims about human sexual development are com-
patible with all possible states of affairs. There is no way to confirm or dis-
confirm either ‘penis envy’ or ‘castration complex’ because there is no
distinguishable difference between evidence affirming and evidence deny-
ing these interpretations of behavior. Exactly opposite behaviors are
equally predictable, depending on whether the alleged psychosexual
stress is overt or repressed.” The requirement of “cognitive content rules
out all utterances that are so loose, poorly formed or obsessively held
(e.g., conspiracy theories) that there is no recognizable difference be-
tween what would be the case if they were so, and what would be the case
if they were not.”8 In a like vein, the Intelligent Design Theory carries no
cognitive freight in the sense that no matter what life form is observed it is
4 INTRODUCTION
consistent with the notion that it manifests an underlying form specified
by some intelligent designer.9
What then is knowledge?Knowledge can be defined as justified true
belief. Hence, in order for a declarative statement to qualify as knowl-
edge, not only must it be a candidate for belief, because it has cognitive
content, but it must meet two other conditions as well. First, it must be
true (or probably true). Second, the statement must be believed with jus-
tification. A belief is justified when it is based on sound inferences from
solid evidence.
Prehistoric humans held the false belief that the sun moved across
the sky because the sun orbited the earth. Clearly they were not in pos-
session of knowledge, but suppose that there was a prehistoric person
who believed correctly that the sun moved across the sky because of
the earth’s rotation. Although this belief was true, this individual could
not be described as possessing knowledge. Even though they believed
what astronomers ultimately proved to be true, there was no evidence
yet to justify that belief. Without justification, a true belief does
not attain the status of knowledge. These concepts are illustrated in
Figure I.1.
f A l l Statements
e
i
l
e Empty Claims
B

r
o
f ds
ates Co m man Question
d s
i Well Formed
d
n Declarative Statements
a (True or False)
C
True
Statements
True
Beliefs
Justified Knowledge
by Evidence
& Inference
FIGUREI.1 Knowledge: justified true belief.
Introduction 5
From this it follows that erroneous beliefs or false knowledge fail to
meet one or more of the necessary conditions of knowledge. Thus, an er-
roneous belief can arise either because it concerns a meaningless claim or
because it concerns a claim that, though meaningful, is not justified by
valid inferences from solid evidence.
Still, even when we have done everything right, by drawing the best
possible inference from sound evidence, we can still wind up adopting
erroneous beliefs. In other words, we can be justified in believing a
falsehood, and honestly claim to know something, if it appears to be
true according to logically sound inferences from the preponderance of
available evidence. “We are entitled to say ‘I know’ when the target
of that claim is supported beyond reasonable doubt in the network
of well-tested evidence. But that is not enough to guarantee that we
do know.”10
Falsehoods are an unavoidable fact of life when we attempt to know
things about the world based on observed evidence. Thus, knowledge
based on the scientific method is inherently uncertain, and provisional,
though less uncertain than knowledge acquired by less formal methods.
However, over time, scientific knowledge improves, as it comes to de-
scribe reality in a progressively more accurate manner. It is a continual
work in progress. The goal of EBTA is a body of knowledge about market
behavior that is as good as can be had, given the limits of evidence gather-
ing and the powers of inference.
ERRONEOUS TA KNOWLEDGE: THE COST
OF UNDISCIPLINED ANALYSIS
To understand why the knowledge produced by the popular version of TA
is untrustworthy, we must consider two distinct forms of TA: subjective
and objective. Both approaches can lead to erroneous beliefs, but they do
so in distinct ways.
Objective TA methods are well defined repeatable procedures that is-
sue unambiguous signals. This allows them to be implemented as comput-
erized algorithms and back-tested on historical data. Results produced by
a back test can be evaluated in a rigorous quantitative manner.
Subjective TA methods are not well-defined analysis procedures. Be-
cause of their vagueness, an analyst’s private interpretations are required.
This thwarts computerization, back testing, and objective performance
evaluation. In other words, it is impossible to either confirm or deny a
subjective method’s efficacy. For this reason they are insulated from evi-
dentiary challenge.
6 INTRODUCTION
From the standpoint of EBTA, subjective methods are the most prob-
lematic. They are essentially meaningless claims that give the illusion of
conveying cognitive content. Because the methods do not specify how
they are to be applied, different analysts applying it to the same set of
market data can reach different conclusions. This makes it impossible to
determine if the method provides useful predictions. Classical chart pat-
tern analysis,11hand-drawn trend lines, Elliott Wave Principle,12Gann pat-
terns, Magic T’s and numerous other subjective methods fall into this
category.13 Subjective TA is religion—it is based on faith. No amount of
cherry-picked examples showing where the method succeeded can cure
this deficiency.
Despite their lack of cognitive content and the impossibility of ever
being supported by sound evidence, there is no shortage of fervent believ-
ers in various subjective methods. Chapter 2 explains how flaws in human
thinking can produce strong beliefs in the absence of evidence or even in
the face of contradictory evidence.
Objective TA can also spawn erroneous beliefs but they come about
differently. They are traceable to faulty inferences from objective evi-
dence. The mere fact that an objective method has been profitable in a
back test is not sufficient grounds for concluding that it has merit. Past
performance can fool us. Historical success is a necessary but not a suffi-
cient condition for concluding that a method has predictive power and,
therefore, is likely to be profitable in the future. Favorable past perfor-
mance can occur by luck or because of an upward bias produced by one
form of back testing called data mining. Determining when back-test
profits are attributable to a good method rather than good luck is a ques-
tion that can only be answered by rigorous statistical inference. This is
discussed in Chapters 4 and 5. Chapter 6 considers the problem of data-
mining bias. Although I will assert that data mining, when done correctly,
is the modern technician’s best method for knowledge discovery, special-
ized statistical tests must be applied to the results obtained with data
mining.
HOW EBTA IS DIFFERENT
What sets EBTA apart from the popular form of TA? First, it is restricted
to meaningful claims—objective methods that can be tested on historical
data. Second, it utilizes advanced forms of statistical inference to deter-
mine if a profitable back test is indicative of an effective method. Thus,