Guide to Neural Computing Applications (Hodder Arnold Publication)

151 Pages·1998·7.08 MB·English

by  Lionel Tarassenko

A Guide to Neural
A Guide to Neural
This Page Intentionally Left Blank
A Guide to Neural
Computing Applications
Lionel Tarassenko
Professor of Electrical and Electronic Engineering
University of Oxford
UK
EI~EVIER
Copubllshed in North, Central and South America by
John Wiley & Sons Inc., New York (cid:12)9T oronto
Elsevier Ltd.
Linacre House, Jordan Hill, Oxford OX2 8DP
200 Wheeler Road, Burlington, MA 01803
First published in Great Britain in 1998 by
Arnold, a member of the Hodder Headline Group,
338 Euston Road, London NWI 3BH
http://www.arnoldpublishers.eom
Copublished in North, Central and South America by
John Wiley & Sons Inc.,
605 Third Street,
New York, NY 10158-0012
Transferred to digital printing 2004
(cid:14)9 1998 Neural Computing Applications Forum (NCAF)
All rights reserved. No part of this publication may be reproduced or
transmitted in any form or by any means, electronically or mechanically,
including photocopying, recording or any information storage or retrieval
system, without either prior permission in writing from the publisher or a
licence permitting restricted copying. In the United Kingdom such licences
are issued by the Copyright Licensing Agency: 90 Tottenham Court Road,
London W 1P 9HE.
Whilst the advice and information in this book is believed to be true and
accurate at the date of going to press, neither the author nor the publisher
can accept any legal responsibility or liability for any errors or omissions
that may be made.
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
Library of Congress Cataloging-in-Publication Data
A catalog record for this book is available from the Library of Congress
ISBN 0 340 70589 2
ISBN 0 471 25550 5 (Wiley)
Publisher: Eliane Wigzell
Production Editor: James Rabson
Production Controller: Priya Gohil
Cover designer: Terry Griffiths
Typeset in 10/12 pt Times by Focal Image, Torquay
CONTENTS
Foreword ix
Introduction
1.1 Neural computing-today's perspective
1.2 The purpose of this book
1.3 A brief overview
1.4 Acknowledgements
Mathematical background for neural computing 5
2.1 Introduction 5
2.2 Why neural networks? 5
2.3 Brief historical background 6
2.4 Pattern recognition 8
2.5 Pattern classification 8
2.6 The single-layer perceptron 9
2.7 From the 1960s to today: multi-layer networks 12
2.8 Multi-layer perceptrons and the error back-propagation
algorithm 14
2.9 Training a multi-layer perceptron 16
2.10 Probabilistic interpretation of network outputs 18
2.11 Unsupervised learning-the motivation 19
2.12 Cluster analysis 20
2.13 Clustering algorithms 21
2.14 Data visualisation-Kohonen's feature map 24
2.15 From the feature map to classification 27
2.16 Radial Basis Function networks 28
2.17 Training an RBF network 30
2.18 Comparison between RBF networks and MLPs 31
2.19 Auto-associative neural networks 32
2.20 Recurrent networks 33
2.21 Conclusion 35
vi Contents
3 Managing a neural computing project 37
3.1 Introduction 37
3.2 Neural computing projects are different 37
3.3 The project life cycle 38
3.4 Project planning 39
3.5 Project monitoring and control 42
3.6 Reviewing 43
3.7 Configuration management 44
3.8 Documentation 45
3.9 The deliverable system 46
4 Identifying applications and assessing their feasibility 49
4.1 Introduction 49
4.2 Identifying neural computing applications 50
4.3 Typical examples of neural computing applications 51
4.4 Preliminary assessment of candidate application 53
4.5 Technical feasibility 53
4.6 Data availability and cost of collection 54
4.7 The business case 55
4.8 Conclusion 57
Neural computing hardware and software 59
5.1 Introduction 59
5.2 Computational requirements 59
5.3 Platforms for software solutions 61
5.4 Special-purpose hardware 64
5.5 Deliverable system 66
Collecting and preparing data 67
6.1 Introduction 67
6.2 Glossary 67
6.3 Data requirements : 68
6.4 Data collection and data understanding 71
7 Design, training and testing of the prototype 77
7.1 Introduction 77
7.2 Overview of design 77
7.3 Pre-processing 79
7.4 Input/output encoding 82
7.5 Selection of neural network type 87
7.6 Selection of neural network architecture 88
7.7 Training and testing the prototype 89
7.8 From prototype to deliverable system 94
7.9 Common problems in training and/or testing the prototype 95
Contents vii
8 The case studies 99
8.1 Overview of the case studies 99
8.2 Benchmark results 102
8.3 Application of data visualisation to the case studies 104
8.4 Application of MLPs to the case studies 109
8.5 Application of RBF networks to the case studies 116
8.6 Conclusions 119
More advanced topics 121
9.1 Introduction 121
9.2 Data visualisation 121
9.3 Multi-layer perceptrons 123
9.4 On-line learning 125
9.5 Introduction to Netlab 126
Appendix A: The error back-propagation algorithm for weight updates
in an MLP 129
Appendix B: Use of Bayes' theorem to compensate for different prior
probabilities 131
References 133
Index 137
This Page Intentionally Left Blank
Foreword
Neural networks are fascinating. A few simple algorithms will learn relation-
ships between cause and effect or organise large volumes of data into orderly
and informative patterns. The prospects for commercial exploitation have been
shown to be real over the past couple of years. It is easy to be carried away
and begin to overestimate their capabilities. The usual consequence of this is,
hopefully, no more serious than an embarrassing failure with concomitant mut-
terings about black boxes and excessive hype. Neural networks cannot solve
every problem. Traditional methods may be better. Nevertheless, neural net-
works, when they are used wisely, usually perform at least as well as the most
appropriate traditional method and in some cases significantly better.
In one sense neural networks are no more than just another statistical tech-
nique, just another systems identification tool, or another clustering algorithm
or mapping function. In another sense they are revolutionary. They put the
tool in the hands of the applications expert rather than the data analyst. In the
terms of late 1980s business jargon, they 'enable the end user'. It is almost
invariably much quicker for the engineer or scientist to learn how to use neu-
ral networks than it is for the data analyst to learn quantum electrodynamics.
Of course the data analyst who does know quantum electrodynamies can, by
adding neural networks to her armoury of techniques, quickly produce compact
empirical models of high dimensional non-linear systems.
The Neural Computing Applications Forum commissioned this book for its
members, who are a group of people interested in the effective application of
neural networks. The book was inspired by a set of guidelines produced for
the United Kingdom Government's Department of Trade and Industry as part
of their Neural Computing Learning Solutions Initiative. Assignment of the
copyright of the text of the original book is gratefully acknowledged. In taking
account of the lessons learned over the very active years since the DTI guidelines
were published, Professor Tarassenko has largely rewritten the original text and
has provided new sample applications which are described in detail.
This book provides a set of guidelines which will help everyone make best