Principles of Neural Model Identification, Selection and Adequacy: With Applications to Financial Econometrics

193 Pages·1999·7.74 MB·English

by  Achilleas Zapranis BSc

Perspectives in Neural Computing
Springer-Verlag London Ltd.
Also in this series:
Maria Marinaro and Roberto Tagliaferri (Eds)
Neural Nets - WIRN VIETRI-96
3-540-76099-7
Adrian Shepherd
Second-Order Methods for Neural Networks
3-540-76100-4
Dimitris C. Dracopoulos
Evolutionary Learning Algorithms for Neural Adaptive Control
3-540-76161-6
John A. Bullinaria, David W. Glasspool and George Houghton (Eds)
4th Neural Computation and Psychology Workshop, London, 9-11
April1997: Connectionist Representations
3-540-76208-6
Maria Marinaro and Roberto Tagliaferri (Eds)
Neural Nets - WIRN VIETRI-97
3-540-76157-8
Gustavo Deco and Dragan Obradovic
An Information-Theoretic Approach to Neural Computing
0-387 -94666-7
Thomas Lindblad and Jason M. Kinser
Image Processing using Pulse-Coupled Neural Networks
3-540-76264-7
L. Nik1asson, M. Boden and T. Ziemke (Eds)
ICANN98
3-540-76263-9
Maria Marinaro and Roberto Tagliaferri (Eds)
Neural Nets - WIRN VIETRI-98
1-85233-051-1
Amanda J.C. Sharkey (Ed.)
Combining Artificial Neural Nets
1-85233-004-X
Dietmar Heinke, Glyn W. Humphreys and Andrew Olson (Eds)
Connectionist Models in Cognitive Neuroscience
The 5th Neural Computation and Psychology Wo rkshop, Birmingham,
8-10 September 1998
1-85233-052-X
Dirk Husmeier
Neural Networks for Conditional Probability Estimation
1-85233-095-3
Achilleas Zapranis and
Apostolos-Paul Refenes
Principles of Neural
Modelldentification,
Selection and Adequacy
With Applications to
Financial Econometrics
, Springer
Achilleas Zapranis, BSc, MSc, PhD
Apostolos-Paul N. Refenes, BSc, PhD
London Business School, Sussex Place, Regents Park, London
NW14SA, UK
Series Editor
J.G. Taylor, BA, BSc, MA, PhD, FlnstP
Centre for Neural Networks, Department of Mathematics, King's
College, Strand, London WC2R 2LS, UK
ISBN 978-1-85233-139-9
British Library Cataloguing in Publication Data
Zapranis, A. D.
Principles of neural model identification, selection and adequacy
: with applications to financial econometrics. -
(perspectives in neural computing)
1. Neural networks (Computer science) 2. Finance -Mathematical models
3. Econometrics
I. Title II. Refenes, Apostolos-Paul
332'.0285632
ISBN 978-1-85233-139-9
Library of Congress Cataloging-in-Publication Data
Zapranis, A. D., 1965-
Principles of neural model identification, selection and adequacy :
with applications to financial econometrics./ A.d. Zapranis and A-P. N. Refenes.
p. cm. --( perspectives in neural computing)
Includes bibliographical references and index.
ISBN 978-1-85233-139-9 ISBN 978-1-4471-0559-6 (eBook)
DOI 10.1007/978-1-4471-0559-6
1. Neural networks (Computer science) 2. Econometrics-Data processing.
3. Finance-Data processing. I. Refenes, Apostolos-Paul. II. Title. III. Series.
QA76.87.Z371999 98-51734
006.3'2--dc21 CIP
Apart from any fair dealing for the purposes of research or private study, or criticism or
review, as permitted under the Copyright, Designs and Patents Act 1988, this publication
may only be reproduced, stored or transmitted, in any form or by any means, with the prior
permission in writing of the publishers, or in the case of reprographic reproduction in
accordance with the terms oflicences issued by the Copyright Licensing Agency. Enquiries
concerning reproduction outside those terms should be sent to the publishers.
© Springer-Verlag London 1999
Originally published by Springer-Verlag London Limited in 1999
The use of registered names, trademarks etc. in this pubIication does not imply, even in the
absence of a specific statement, that such names are exempt from the relevant laws and
regulations and therefore free for general use.
The publisher makes no representation, expr.ess or implied, with regard to the accuracy of
the information contained in this book and cannot accept any legal responsibiIity or liabiIity
for any errors or omissions that may be made.
Typesetting: lan Kingston Editorial Services, Nottingham
34/3830-543210 Printed on acid-free paper SPIN 10707793
Preface
Neural networks are receiving much attention because of their powerful
universal approximation properties. They are essentially devices for
non-parametric statistical inference, providing an elegant formalism for
unifying different non-parametric paradigms, such as nearest neigh
bours, kernel smoothers and projection pursuit. Neural networks have
shown considerable successes in a variety of disciplines, ranging through
engineering, control and financial modelling. However, a major weakness
of neural modelling is the lack of established procedures for performing
tests for misspecified models and tests of statistical significance for the
various parameters that have been estimated. This is a serious disadvan
tage in applications where there is a strong culture for testing not only the
p.redictive power of a model or the sensitivity of the dependent variable to
changes in the inputs but also the statistical significance of the finding at a
specified level of confidence. This is very important in the majority of
financial applications, where the data-generating processes are domi
nantly stochastic and only partially deterministic.
In this book we investigate a broad range of issues arising with relation
to their use as non-parametric statistical tools, including controlling the
bias and variance parts of the estimation error, eliminating parameter and
explanatory variable redundancy, assessing model adequacy and esti
mating sampling variability. Based upon the latest, most significant
developments in estimation theory, model selection and the theory of
misspecified models, this book develops neural networks into an
advanced financial econometrics tool for non-parametric modelling. It
provides the theoretical framework and displays, through a selected case
study and examples, the efficient use of neural networks for modelling
complex financial phenomena.
The majority of existing books on neural networks and their applica
tion to finance concentrate on some of the intricate algorithmic aspects of
neural networks, the bulk of which is irrelevant to practitioners in this
field. They use terminology which is incomprehensible to professional
financial .engineers, statisticians and econometricians, who are the
natural readership in this subject. Neural networks are essentially statis
tical devices for non-linear, non-parametric regression analysis, but most
of the existing literature discusses neural networks as a form of artificial
intelligence. In our opinion this work meets an urgent demand for a
v
vi Preface
textbook illustrating how to use neural networks in real-life financial
contexts and provides methodological guidelines on how to develop
robust applications which work from a platform of statistical insight.
Achilleas D. Zapranis
Apostolos-Paul N. Refenes
Jan uary 1999
to our wives
Fotini and Tina
Contents
1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . .. 1
1.1 Overview ............................. 1
1.2 Active Asset Management, Neural Networks and Risk ... 2
1.2.1 Factor Analysis ..................... 6
1.2.2 Estimating Returns ................... 7
1.2.3 Portfolio Optimization ................. 8
1.3 Non-Parametric Estimation with Neural Networks . . . .. 9
1.3.1 Sources of Specification Bias . . . . . . . . . . . . .. 10
1.3.2 Principles of Neural ModelIdentification ...... 11
1.4 Overview ofthe Remaining Chapters . . . . . . . . . . . .. 17
2 Neural Modelldentification . . . . . . . . . . . . . . . . . . . .. 19
2.1 Overview ............................. 19
2.2 Neural Model Selection ..... . . . . . . . . . . . . . . .. 22
2.2.1 Model Specification . . . . . . . . . . . . . . . . . .. 22
2.2.2 Fitness Criteria ..................... 25
2.2.3 Parameter Estimation Procedures . . . . . . . . . .. 26
2.2.4 Consistency and the Bias Variance Dilemma .... 28
2.3 Variable Significance Testing ................. 31
2.3.1 Relevance Quantification . . . . . . . . . . . . . . .. 32
2.3.2 Sampling Variability Estimation ........... 32
2.3.3 Hypothesis Testing ................... 33
2.4 Model Adequacy Testing .................... 34
2.5 Summary ............................. 35
3 Review of Current Practice in Neural Model Identification 37
3.1 Overview ............................. 37
3.2 Current Practice in Neural Model Selection ......... 37
3.2.1 Regularization .. . . . . . . . . . . . . . . . . . . .. 39
3.2.2 Topology-Modifying Algorithms ........... 40
3.2.3 The Structural Risk Minimization (SRM) Principle 42
3.2.4 The Minimum Description Length (MDL) Principle 43
3.2.5 The Maximum A Posteriori Probability (MAP)
Principle . . . . . . . . . . . . . . . . . . . . . . .. 43
3.2.6 The Minimum Prediction Risk (MPR) Principle 44
vii
viii Contents
3.3 Variable Significance Testing ................. 46
3.3.1 Common Relevance Criteria . . . . . . . . . . . . .. 46
3.3.2 Sampling Variability and Bias Estimation With
Bootstrap . . . . . . . . . . . . . . . . . . . . . . . .. 50
3.3.3 Hypothesis Tests for Variable Selection ....... 54
3.4 Model Adequacy Testing: Misspecification Tests ...... 56
3.5 Summary ............................. 57
4 Neural Model Selection: the Minimum Prediction Risk
Principle .............................. 59
4.1 Overview ............................. 59
4.2 Algebraic Estimation of Prediction Risk ........... 62
4.3 Estimating Prediction Risk With Resampling Methods .. 64
4.3.1 The Bootstrap and Jackknife Methods for
Estimating Prediction Risk ............. 65
4.3.2 Cross-Validatory Methods for Estimating
Prediction Risk .................... 66
4.4 Evaluation of the Model Selection Procedure ........ 68
4.4.1 Experimental Setup ................... 68
4.4.2 Algebraic Estimates . . . . . . . . . . . . . . . . . .. 69
4.4.3 Bootstrap Estimates . . . . . . . . . . . . . . . . . .. 69
4.4.4 Discussion ........................ 72
4.5 Summary ............................. 73
5 Variable Significance Testing: a Statistical Approach ...... 75
5.1 Overview ............................. 75
5.2 Relevance Quantification . . . . . . . . . . . . . . . . . . .. 79
5.2.1 Sensitivity Criteria ................... 79
5.2.2 Model-Fitness Sensitivity Criteria . . . . . . . . . .. 81
5.3 Sampling Variability Estimation ............... 85
5.3.1 Local Bootstrap for Neural Models .......... 85
5.3.2 Stochastic Sampling from the Asymptotic
Distribution of the Network's Parameters
(Parametric Sampling) . . . . . . . . . . . . . . .. 88
5.3.3 Evaluation of Bootstrap Schemes for Sampling
Variability Estimation ................ 89
5.4 Hypothesis Testing ....................... 104
5.4.1 Confidence Intervals .................. 104
5.4.2 Evaluating the Effect of a Variable's Removal .... 105
5.4.3 Stepwise Variable Selection .............. 107
5.5 Evaluation of Variable Significance Testing ......... 109
5.6 Summary ............................. 112
6 Model Adequacy Testing . . . . . . . . . . . . . . . . . . . . . . . 113
6.1 Overview ............................. 113
Contents Ix
6.2 Testing for Serial Correlation in the Residuals ....... 114
6.2.1 The Correlogram .................... 114
6.2.2 The Box-Pierce Q-Statistic ............... 115
6.2.3 The Ljung-Box LB-Statistic .............. 115
6.2A The Durbin-Watson Statistic ............. 116
6.3 An F-test for Model Adequacy ................. 116
6.4 Summary ............................. 117
7 Neural Networks in Tactical Asset Allocation: a Case Study .. 119
7.1 Overview ............................. 119
7.2 Quantitative Models for Tactical Asset Allocation ..... 121
7.3 Data Pre-Processing ....................... 125
7.4 Forecasting the Equity Premium with Linear Models 129
7A.l Model Estimation .................... 129
7A.2 ModelAdequacyTesting ................ 131
7 A.3 Variable Selection . . . . . . . . . . . . . . . . . . . . 13 7
7.5 Forecasting the Equity Premium with Neural Models ... 139
7.5.1 Model Selection and Adequacy Testing ....... 141
7.5.2 Variable Selection .................... 143
7.6 Comparative Performance Evaluation ............ 146
7.7 Summary ............................. 153
8 Conclusions 157
Appendices
A Computation of Network Derivatives . . . . . . . . . . . . . 161
B Generating Random Normal Deviates ............ 175
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
Index .................................... 183
1. Introduction
1.1 Overview
Because of their inductive nature, neural networks have the ability to infer complex
non-linear relationships between an asset price and its determinants. Although this
approach can potentially lead to better non-parametric estimators, neural networks
are not always easily accepted in the financial economics community, mainly
because there do not exist established procedures for testing the statistical signifi
cance of the various aspects of the estimated model. The primary aim of this book is
to provide a coherent set of methodologies for developing and assessing neural
models, with a strong emphasis on their practical use in the capital markets. Partly a
tutorial, partly a review, this chapter gives an introduction to investment manage
ment, positions neural networks and finally gives an introductory exposure to a
novel neural model identification procedure, which is synergetic rather than
competitive to theory formulation.
Modern investment management models, such as the Arbitrage Pricing Theory,
rely on the assumption that asset returns can be explained in terms of a set of factors.
The usual assumption is that the return of an asset is a linear combination of the
asset's exposure to these factors. Such theories have been very useful in expanding
our understanding of the capital markets, but many financial anomalies have
remained unexplainable. Here we divide the problem of investment management
into three parts, factor analysis, estimating returns and portfolio optimization, and
show that neural learning can playa part in each.
Neural networks form a field of research which has enjoyed rapid expansion and
increasing popularity in both the academic and industrial research communities.
Neural networks are essentially statistical devices for performing inductive infer
ence. From the statistician's point of view they are analogous to non-parametric,
non-linear regression models. The novelty about neural networks lies in their
ability to model non-linear processes with few (if any) a priori assumptions about
the nature of the generating process. This is particularly useful in investment
management, where much is assumed and little is known about the nature of the
processes determining asset prices.
A. Zapranis et al., Principles of Neural Model Identification, Selection and Adequacy
© Springer-Verlag London 1999