Intelligent Systems and Financial Forecasting

232 Pages·1997·7.661 MB·English

by  Jason Kingdon PhD, MSc, BSc (auth.)

To Jim and Jean Kingdon
Perspectives in Neural Computing
Springer
London
Berlin
Heidelberg
New York
Barcelona
Budapest
Hong Kong
Milan
Paris
Santa Clara
Singapore
Tokyo
Also in this series:
J.G. Taylor, E.R. Caianiello, R.M.J. Cotterill and J.W. Clark (Eds)
Neural Network Dynamics, Proceedings of the Workshop on Complex
Dynamics in Neural Networks, June 17-211991 at IIASS, Vierri, Italy
3-540-19771-0
J.G. Taylor (Ed.)
Neural Network Applications (NCM91)
3-540-19772-9
J.G. Taylor
The Promise of Neural Networks
3-540-19773-7
Maria Marinaro and Roberto Tagliaferri (Eds)
Neural Nets - WIRN VIETRI-96
3-540-76099-7
Adrian Shepherd
Second-Order Methods for Neural Networks: Fast and Reliable
Training Methods for Multi-Layer Perceptrons
3-540-76100-4
Jason Kingdon
Intelligent Systems and
Financial Forecasting
, Springer
To Jim and Jean Kingdon
Jason Kingdon, PhD, MSc, BSc
Director, Searchspace Limited, 83 Charlotte Street, London WIP lLB, UK
[email protected]
Series Editor
J.G. Taylor, BA, BSc, MA, PhD, FlnstP
Centre for Neural Networks,
Department of Mathematics, Kings College,
Strand, London WC2R 2LS, UK
ISBN-13: 978-3-540-76098-6 Springer-Verlag Berlin Heidelberg New York
British Library Cataloguing in Publication Data
Kingdon, Jason
Intelligent systems and financial forecasting. - (Perspectives in neural computing)
1.Finance - Forecasting - Data processing 2.Artificial intelligence
I.Title
332'.028563
ISBN·13: 978·3·540·76098·6 e·ISBN·13: 978·1·4471·0949·5
DOl: 10.1007/978·1·4471·0949·5
Library of Congress Cataloging-in-Publication Data
Kingdon, J. (Jason)
Intelligent systems and financial forecasting / Jason Kingdon.
p. em. --(Perspectives in neural computing)
Includes bibliographical references (p. ).
ISBN 3-540-76098-9 (pbk.: alk. paper)
1. Finance--Decision making--Data processing. 2. Finance--Mathematical models.
3. Time-series analysis. 4. Artificial intelligence. 5. Neural networks (Computer science).
6. Genetic algorithms. 7. Fuzzy logic. I. Title. II. Series.
HG4515.5.K56 1997
332'.0285'63--dc21 96-49037
Apart from any fair dealing for the purposes of research or private study, or criticism or
review, as permitted under the Copyright, Designs and Patents Act 1988, this publication
may only be reproduced, stored or transmitted, in any form or by any means, with the prior
permission in writing of the publishers, or in the case of reprographic reproduction in
accordance with the terms oflicences issued by the Copyright Licensing Agency. Enquiries
concerning reproduction outside those terms should be sent to the publishers.
© Springer-Verlag London Limited 1997
The use of registered names, trademarks etc. in this publication does not imply, even in the
absence of a specific statement, that such names are exempt from the relevant laws and
regulations and therefore free for general use.
The publisher makes no representation, express or implied, with regard to the accuracy of
the information contained in this book and cannot accept any legal responsibility or liability
for any errors or omissions that may be made.
Typesetting: Ian Kingston Editorial Services, Nottingham
34/3830-543210 Printed on acid-free paper
Preface
A fundamental objective of Artificial Intelligence (AI) is the creation ofi n
telligent computer programs. In more modest terms AI is simply con
cerned with expanding the repertoire of computer applications into new
domains and to new levels of efficiency. The motivation for this effort
comes from many sources. At a practical level there is always a demand for
achieving things in more efficient ways. Equally, there is the technical
challenge of building programs that allow a machine to do something a
machine has never done before. Both of these desires are contained within
AI and both provide the inspirational force behind its development. In
terms of satisfying both of these desires there can be no better example
than machine learning. Machines that can learn have an in-built effi
ciency. The same software can be applied in many applications and in
many circumstances. The machine can adapt its behaviour so as to meet
the demands of new, or changing, environments without the need for
costly re-programming. In addition, a machine that can learn can be ap
plied in new domains with the genuine potential for innovation. In this
sense a machine that can learn can be applied in areas where little is
known about possible causal relationships, and even in circumstances
where causal relationships are judged not to exist. This last aspect is of
major significance when considering machine learning as applied to fi
nancial forecasting. Traditional economic theory has held that financial
price series are random and that no underlying predictive model of such
series exists. The lack of a priori models on which to base a computational
solution for financial modelling therefore provides one of the hardest
tests oflearning system technology.
There are a number of reasons why the intelligent systems community
has taken such an interest in financial forecasting. Clearly, one aspect is
the possibility of financial reward. There is no doubt that many applica
tions of intelligent systems in finance have been profit driven. How suc
cessful this has been is still very open. The overall lack of controls and
explicit experimental procedure within much of the published literature
means that replication, or validation, of the results is almost impossible.
This point is not just a criticism of sloppy experimental procedure it goes
some way to the heart of the modelling problem. How do you validate a
model? How can you have confidence in the results? What does good his
toric performance imply? All of these issues are made worse when it is
vii
viii Preface
considered that many intelligent system techniques rely on random ini
tialisation. What impact does this have on the stability of the results at
tained? This book does not answer all of these questions. Instead, it
attempts to provide a pragmatic analysis and guide to solving some of
these issues as and when they arise in the course of developing an auto
mated adaptive financial time series modelling system. In the process of
designing such a system a number of new techniques are introduced, both
in terms of new styles of genetic algorithm, new methods for neural net
work pruning and analysis, and new methods for examining the perform
ance of a neural net time series model.
In more technical terms this book presents an automated system for fi
nancial time series modelling. In it, formal and applied methods are inves
tigated for combining feed-forward neural networks and genetic
algorithms into a single adaptive/learning system for automated time se
ries forecasting. Four new intelligent system techniques arise from this in
vestigation: 1) novel forms of genetic algorithm are introduced which are
designed to counter the representational bias associated with the conven
tional Holland genetic algorithm, 2) an experimental methodology for
validating neural network architecture design strategies is introduced, 3)
a new method for network pruning is introduced, and 4) an automated
method for inferring network complexity for a given learning task is de
vised. These methods provide a general-purpose applied methodology for
developing neural network applications and are tested in the construction
of an automated system for financial time series modelling. It is shown
that the system developed isolates a deterministic signal within a Gilt Fu
tures prices series, to a confidence level of over 99 per cent, yielding a pre
diction accuracy of over 60 per cent on a single run of 1000 out-of-sample
experiments.
Acknowledgements
I would like to thank all my colleagues from both Searchspace Ltd and the
Computer Science Department at UCL. In particular, I would like to thank
Laura Dekker and Dr Suran Goonatilake for helpful comments and sug
gestions on earlier drafts of this work, and to give special thanks to Laura
Dekker for helping to proofread later versions and donating the image of
the 4-dimensional hypercube. I would also like to thank members of the
Intelligent Systems Lab for joint work and collaborations that were under
taken during my time at UCL. In particular, my thanks go to Dr Ugur
Bilge, Konrad Feldman, Dr Sukhdev Khebbal, Anoop Mangat, Dr Mike
Recce, Dr Jose Ribeiro, John Taylor and Professor Philip Treleaven. Fi
nally I would also like to thank Jerry Severwright and Professor John G.
Taylor for an extremely valuable discussion of the work.
Contents
1 From Learning Systems to Financial Modelling ........ . 1
1.1 Introduction .......................... . 1
1.2 Adaptive Systems and Financial Modelling ........ . 3
1.2.1 Financial Modelling: The Efficient Markets
Hypothesis ....................... . 3
1.2.2 Learning Systems ................... . 4
1.2.3 Technical Issues .................... . 5
1.3 Time Series Analysis .................... . 8
1.3.1 Fundamentals of Time Series Forecasting and
Learning ........................ . 8
1.4 Brief History of Neural Networks .............. . 10
1.4.1 The Development of Neural Net Techniques 11
1.4.2 More Recent Issues .................. . 13
1.5 Book Overview . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.5.1 Research Objectives .................. . 14
1.5.2 Book Structure . . . . ............. . 14
1.6 Summary ......... . 17
2 Adaptive Systems and Financial Modelling . . . . . . . . . . . . 19
2.1 Financial Modelling ...................... . 19
2.2 The Problems with Financial Modelling ......... . 20
2.2.1 Fuzzy Rationality and Uncertainty ......... . 21
2.2.2 Efficient Markets and Price Movement . . . . . . . . 22
2.3 Evidence Against the Efficiency Hypothesis ......... 23
2.4 An Adaptive Systems Approach .... . . . . . . . . . . .. 25
2.5 Neural Nets and Financial Modelling . . . . . . . . . . . .. 26
2.5.1 Comparisons Between Neural Nets and Other
Time Series Methods ................. 29
2.6 Genetic Algorithms in Finance ............... 31
2.6.1 The Genetic Algorithm Search Technique ..... 31
2.6.2 Applications of Genetic Algorithms . : . . . . . . .. 34
2.7 Summary ............................. 34
ix
x Contents
3 Feed-Forward Neural Network Modelling . . . . . . . . . . . .. 37
3.1 Neural Net Search ........................ 37
3.2 MLP Training: The Model ................... 38
3.3 MLP: Model Parameters .................... 40
3.4 The Data ............................. 41
3.5 MLP: Training Parameters .., . . . . . . . . . . . . . . .. 42
3.5.1 Architecture ....................... 42
3.5.2 Activation Function . . . . . . . . . . . . . . . . . .. 45
3.5.3 Learning Rules, Batch and On-Line Training .... 45
3.6 Network Performance . . . . . . . . . . . . . . . . . . . . .. 46
3.6.1 Convergence ......... . . . . . . . . . . . . .. 46
3.6.2 Network Validation and Generalisation ....... 49
3.6.3 Automated Validation ................. 51
3.7 Summary ............................. 52
4 Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . .. 55
4.1 Using Genetic Algorithms ................... 55
4.2 Search Algorithms . . . . . . . . . . . . . . . . . . . . . . .. 56
4.2.1 The GA Search Process: The Simple GA ....... 57
4.2.2 Schema Analysis . . . . . . . . . . . . . . . . . . . .. 58
4.2.3 Building Blocks Under Review. . . . . . . . . . . .. 60
4.3 GA Parameters . . . . . . . . . . . . . . . . . . . . . . . . .. 61
4.3.1 The Shape of Space ................... 61
4.3.2 Population Encodings ................. 66
4.3.3 Crossover, Selection, Mutation and Populations .. 68
4.4 A Strategy for GA Search: Transmutation .......... 73
4.4.1 Five New Algorithms: Morphic GAs (MGAs) 75
4.5 Summary ............................. 79
5 Hypothesising Neural Nets ..................... 81
5.1 System Objectives ........................ 81
5.2 Hypothesising Neural Network Models . . . . . . . . . . .. 82
5.3 Occam's Razor and Network Architecture .......... 83
5.3.1 Existing Regulisation and Pruning Methods . . . .. 83
5.3.2 Why use Occam's Razor? . . . . . . . . . . . . . . .. 84
5,4 Testing Occam's Razor ..................... 85
5.4.1 Generating Time Series . . . . . . . . . . . . . . . .. 85
5.4.2 Artificial Network Generation (ANG) ........ 86
5.4.3 ANG Results ....................... 87
5.4.4 Testing Architectures . . . . . . . . . . . . . . . . .. 87
5.5 Strategies using Occam's Razor ................ 91
5.5.1 Minimally Descriptive Nets .............. 92
5.5.2 Network Model ..................... 93
5.5.3 Network Regression Pruning (NRP) ......... 94
5.5.4 Results ofNRP on ANG Series . . . . . . . . . . . .. 96
Contents xi
5.5.5 Interpretation of the Pruning Error Profiles . . . .. 98
5.5.6 Determining Topologies ................ 100
5.6 Validation ............................ 101
5.7 GA-NN Hybrids: Representations ............... 102
5.7.1 Fitness Measures for GA-NN Hybrids ........ 103
5.7.2 Neural Networks and GAs: Fitness Measure for
Generalisation ................ . . . . . . 104
5.8 Summary ............................. 105
6 Automating Neural Net Time Series Analysis .......... 107
6.1 System Objectives ........................ 107
6.2 ANTAS .............................. 108
6.2.1 Stage I: Primary Modelling ............... 109
6.2.2 Stage II: Secondary Modelling . . . . . . . . . . . . . 111
6.2.3 Stage III: System Modelling .............. 111
6.3 Primary Modelling ....................... 111
6.3.1 Automating the use of Neural Nets .......... 112
6.3.2 GA Rule-Based Modelling ............... 114
6.4 Secondary Modelling ...................... 115
6.4.1 Generating Secondary Models ............. 115
6.4.2 Model Integration .................... 115
6.4.3 Model Performance Statistics ............. 117
6.5 Validation Modules ....................... 118
6.6 Control Flow ........................... 119
6.6.1 Neural Net Control ................... 119
6.6.2 GA Control ........................ 121
6.7 Summary ............................. 122
7 The Data: The Long Gilt Futures Contract . . . . . . . . . . . . . 125
7.1 The Long Gilt Futures Contract ................ 125
7.2 The LGFC Data . . . . . . . . . . . . . . . . . . . . . . . . . . 126
7.2.1 Time Series Construction ............... 126
7.3 Secondary Data ......................... 128
7.4 Data Preparation ........................ 130
7.4.1 LGFC Data Treatment ................. 130
7.4.2 Using Moving Averages ................ 131
7.5 Data Treatment Modules .................... 133
7.5.1 Moving Average Modules ............... 133
7.6 Efficient Market Hypothesis and the LGFC ......... 135
7.7 Summary ............................. 136
8 Experimental Results ......................... 137
8.1 Experimental Design ...................... 137
8.2 Phase I - Primary Models ................... 138
8.2.1 NN Hypothesis Modules (Phase I) .......... 138
8.2.2 Results for GA-NN Module .............. 140