Mathematics of Neural Networks. Models, Algorithms and Applications

423 Pages·2016·40.93 MB·English

by  Ellacott Stephen W.

MATHEMATICS OF NEURAL
NETWORKS
Models, Algorithms and Applications
OPERATIONS RESEARCH/COMPUTER SCIENCE
INTERFACES
SERIES
Ramesh Sharda, Series Editor
ConocolDuPont Chair of Management of Technology
Oklahoma State University
Stillwater, Oklahoma U.S.A.
Other published titles in the series:
Brown, Donald/Scherer, William T.
University of Virginia
Intelligent Scheduling Systems
Nash, Stephen G'/Sofer, Ariela
George Mason University
The Impact ofE merging Technologies on Computer Science
and Operations Research
Barth, Peter
Max-Planck-Institut fur Infonnatik, Gennany
Logic-Based 0-/ Constraint Programming
Jones, Christopher V.
University of Washington
Visualization and Optimization
Barr, Richard S./ Helgason, Richard V./ Kennington, Jeffery L.
Southern Methodist University
Interfaces in Computer Science and Operations Research: Advances in
Metaheuristics, Optimization, and Stochastic Modeling Technologies
MATHEMATICS OF
NEU RAL N ETWORKS
Models, Aigorithms and Applications
EDITED SY
Stephen W ELLACOTT
University of Srighton
United Kingdom
•
John C MASON
University of Huddersfield
United Kingdom
•
lain J ANDERS ON
University of Huddersfield
United Kingdom
SPRINGER SCIENCE+BUSINESS MEDIA, LLC
Library of Congress Cataloging-in-Publication Data
A C.I.P. Catalogue record for this book is available from the Library of Congress.
ISBN 978-1-4613-7794-8 ISBN 978-1-4615-6099-9 (eBook)
DOI 10.1007/978-1-4615-6099-9
© Springer Science+Business Media New York 1997
Originally published by Kluwer Academic Publishers 1997
Softcover reprint of the hardcover 1s t edition 1997
All rights reserved. No part of this publication may be reproduced, stored in a
retrieval system or transmitted in any form or by any means, mechanical, photo
copying, recording, or otherwise, without the prior written permission of the
publisher, Springer Science+Business Media, LLC
Printed on acid-free paper.
CONTENTS
PREFACE
XXI
Part I INVITED PAPERS 1
1 N-TUPLE NEURAL NETWORKS
N. M. Allinson and A. R. Kolcz 3
2 INFORMATION GEOMETRY OF NEURAL
NETWORKS -AN OVERVIEW-
Shun-ichi Amari 15
3 Q-LEARNING: A TUTORIAL AND
EXTENSIONS
George Cybenko, Robert Gray and Katsuhiro Moizumi 24
4 ARE THERE UNIVERSAL PRINCIPLES OF
BRAIN COMPUTATION?
Stephen Grossberg 34
5 ON-LINE TRAINING OF MEMORY-DRIVEN
ATTRACTOR NETWORKS
Morris W. Hirsch 41
6 MATHEMATICAL PROBLEMS ARISING
FROM CONSTRUCTING AN ARTIFICIAL
BRAIN
J. G. Taylor 47
Part II SUBMITTED PAPERS 59
7 THE SUCCESSFUL USE OF PROBABILITY
DATA IN CONNECTIONIST MODELS
J. R. Alexander Jr. and J. P. Coughlin 61
8 WEIGHTED MIXTURE OF MODELS FOR
ON-LINE LEARNING
P. Edgar An 67
9 LOCAL MODIFICATIONS TO RADIAL BASIS
NETWORKS
I. J. Anderson 73
v
VI MATHEMATICS OF NEURAL NETWORKS
10 A STATISTICAL ANALYSIS OF THE
MODIFIED NLMS RULES
E.D. Aved'yan, M. Brown and C.J. Harris 78
11 FINITE SIZE EFFECTS IN ON-LINE
LEARNING OF MULTI-LAYER NEURAL
NETWORKS.
David Barber, Peter Sollich and David Saad 84
12 CONSTANT FAN-IN DIGITAL NEURAL
NETWORKS ARE VLSI-OPTIMAL
V. Beiu 89
13 THE APPLICATION OF BINARY ENCODED
2ND DIFFERENTIAL SPECTROMETRY IN
PREPROCESSING OF UV- VIS ABSORPTION
SPECTRAL DATA
N Benjathapanun, W J 0 Boyle and K T V Grattan 95
14 A NON-EQUIDISTANT ELASTIC NET
ALGORITHM
Jan van den Berg and Jock H. Geselschap 101
15 UNIMODAL LOADING PROBLEMS
Monica Bianchini, Stefano Fanelli, Marco Gori and Marco Protasi 107
16 ON THE USE OF SIMPLE CLASSIFIERS FOR
THE INITIALISATION OF ONE-HIDDEN-
LAYER NEURAL NETS
Jan C. Bioch, Robert Carsouw and Rob Potharst 113
17 MODELLING CONDITIONAL PROBABILITY
DISTRIBUTIONS FOR PERIODIC VARIABLES
Christopher M Bishop and Ian T Nabney 118
18 INTEGRO-DIFFERENTIAL EQUATIONS
IN COMPARTMENTAL MODEL
NEURODYNAMICS
Paul C. Bressloff 123
19 NONLINEAR MODELS FOR NEURAL
NETWORKS
Susan Brittain and Linda M. Haines 129
20 A NEURAL NETWORK FOR THE
TRAVELLING SALESMAN PROBLEM WITH A
WELL BEHAVED ENERGY FUNCTION
Marco Budinich and Barbara Rosario 134
Contents
Vll
21 SEMIPARAMETRIC ARTIFICIAL NEURAL
NETWORKS
Enrico Capobianco 140
22 AN EVENT-SPACE FEEDFORWARD
NETWORK USING MAXIMUM ENTROPY
PARTITIONING WITH APPLICATION TO
LOW LEVEL SPEECH DATA
D.K. Y. Chiu, D. Bockus and J. Bradford 146
23 APPROXIMATING THE BAYESIAN DECISION
BOUNDARY FOR CHANNEL EQUALISATION
USING SUBSET RADIAL BASIS FUNCTION
NETWORK
E.S. Chng, B. Mulgrew, S. Chen and G. Gibson 151
24 APPLICATIONS OF GRAPH THEORY TO
THE DESIGN OF NEURAL NETWORKS
FOR AUTOMATED FINGERPRINT
IDENTIFICATION
Carol G. Crawford 156
25 ZERO DYNAMICS AND RELATIVE DEGREE
OF DYNAMIC RECURRENT NEURAL
NETWORKS
A. Delgado, C. Kambhampati and K. Warwick 161
26 IRREGULAR SAMPLING APPROACH
TO NEUROCONTROL: THE BAND-AND
SPACE-LIMITED FUNCTIONS QUESTIONS
Andrzej Dzielinski and Rafal Zbikowski 166
27 UNSUPERVISED LEARNING OF TEMPORAL
CONSTANCIES BY PYRAMIDAL-TYPE
NEURONS.
Michael Eisele 171
28 NUMERICAL ASPECTS OF MACHINE
LEARNING IN ARTIFICIAL NEURAL
NETWORKS
S. W. Ellacott and A. Easdown 176
29 LEARNING ALGORITHMS FOR RAM-BASED
NEURAL NETWORKS
Alistair Ferguson, Laurence C Dixon and Hamid Bolouri 181
30 ANALYSIS OF CORRELATION MATRIX
MEMORY AND PARTIAL MATCH-
IMPLICATIONS FOR COGNITIVE
Vlll MATHEMATICS OF NEURAL NETWORKS
PSYCHOLOGY
Richard Filer and James Austin 186
31 REGULARIZATION AND REALIZABILITY IN
RADIAL BASIS FUNCTION NETWORKS
Jason A.S. Freeman and David Saad 192
32 A UNIVERSAL APPROXIMATOR
NETWORK FOR LEARNING CONDITIONAL
PROBABILITY DENSITIES
D. Husmeier, D. Allen and J. G. Taylor 198
33 CONVERGENCE OF A CLASS OF NEURAL
NETWORKS
Mark P. Joy 204
34 APPLICATIONS OF THE COMPARTMENTAL
MODEL NEURON TO TIME SERIES ANALYSIS
S. K asderidis and J. G. Taylor 209
35 INFORMATION THEORETIC NEURAL
NETWORKS FOR CONTEXTUALLY GUIDED
UNSUPERVISED LEARNING
Jim Kay 215
36 CONVERGENCE IN NOISY TRAINING
Petri K oistinen 220
37 NON-LINEAR LEARNING DYNAMICS WITH
A DIFFUSING MESSENGER
Barl Krekelberg and John G. Taylor 225
38 A VARIATIONAL APPROACH TO
ASSOCIATIVE MEMORY
Abderrahim Labbi 230
39 TRANSFORMATION OF NONLINEAR
PROGRAMMING PROBLEMS INTO
SEPARABLE ONES USING MULTILAYER
NEURAL NETWORKS
Bao-Liang Lu and Koji Ito 235
40 A THEORY OF SELF-ORGANISING NEURAL
NETWORKS
S P Luttrell 240
41 NEURAL NETWORK SUPERVISED TRAINING
BASED ON A DIMENSION REDUCING
METHOD
G.D. Magoulas, M.N. Vrahatis, T.N. Grapsa and G.S. Androulakis 245
Contents
IX
42 A TRAINING METHOD FOR DISCRETE
MULTILAYER NEURAL NETWORKS
G.D. Magoulas, M.N. Vrahatis, T.N. Grapsa and G.S. Androulakis 250
43 LOCAL MINIMAL REALISATIONS OF
TRAINED HOPFIELD NETWORKS
S. Manchanda and G.G.R. Green 255
44 DATA DEPENDENT HYPERPARAMETER
ASSIGNMENT
Glenn Marion and David Saad 259
45 TRAINING RADIAL BASIS FUNCTION
NETWORKS BY USING SEPARABLE AND
ORTHOGONALIZED GAUSSIANS
J. C. Mason, 1. J. Anderson, G. Rodriguez and S. Seatzu 265
46 ERROR BOUNDS FOR DENSITY
ESTIMATION BY MIXTURES
Ronny Meir and Assaf J. Zeevi 270
47 ON SMOOTH ACTIVATION FUNCTIONS
H. N. Mhaskar 275
48 GENERALISATION AND REGULARISATION
BY GAUSSIAN FILTER CONVOLUTION OF
RADIAL BASIS FUNCTION NETWORKS
Christophe Molina and Mahesan Niranjan 280
49 DYNAMICAL SYSTEM PREDICTION: A LIE
ALGEBRAIC APPROACH FOR A NOVEL
NEURAL ARCHITECTURE
Yves Moreau and Joos Vandewalle 285
50 STOCHASTIC NEURODYNAMICS AND THE
SYSTEM SIZE EXPANSION
Toru Ohira and Jack D. Cowan 290
51 AN UPPER BOUND ON THE BAYESIAN
ERROR BARS FOR GENERALIZED LINEAR
REGRESSION
Cazhaow S. Qazaz, Christopher K. I. Williams and Christopher M.
Bishop 295
52 CAPACITY BOUNDS FOR STRUCTURED
NEURAL NETWORK ARCHITECTURES
Peter Rieper, Sabine Kroner and Reinhard Moratz 300
53 ON-LINE LEARNING IN MULTILAYER
NEURAL NETWORKS
David Saad and Sara A. Solla 306