Neural Network Dynamics: Proceedings of the Workshop on Complex Dynamics in Neural Networks, June 17–21 1991 at IIASS, Vietri, Italy

377 Pages·1992·12.04 MB·English

by  Günther Palm, Friedrich T. Sommer (auth.), J. G. Taylor BA, BSc, MA, PhD, FlnstP, E. R. Caianiello, R. M. J. Cotterill BSc, MS, PhD, DSc, FlnstP, C.Phys., J. W. Clark (eds.)

PERSPECTIVES
IN
NEURAL COMPUTING
Also in this series
J.G. Taylor and C.L.T. Mannion (Eds.)
Theory and Applications of Neural Networks (NCM90), 3-540-19650-1
J.G. Taylor and C.L.T. Mannion (Eds.)
Coupled Oscillating Neurons (CN090), 3-540-19744-3
J.G. Taylor
The Promise of Neural Networks, 3-540-19773-7
J.G. Taylor (Ed.)
Neural Network Applications (NCM91), 3-540-19772-9
J.G. Taylor, E.R. Caianiello,
R.M.J. Cotterill and J.W. Clark (Eds.)
NEURAL NETWORK
DYNAMICS
Proceedings of the Workshop on Complex
Dynamics in Neural Networks, June 17-21 1991 at
IIASS, Vietri, Italy
- i - -
Springer-Verlag
London Berlin Heidelberg New York
Paris Tokyo Hong Kong
Barcelona Budapest
J. G. Taylor, BA, BSe, MA, PhD, FlnstP
Director, Centre for Neural Networks, Department of Mathematics,
King's College, Strand, London WC2R 2LS, UK
E.R. Caianiello
Dipartimento di Fisica Teorica, Universita di Salerno,
Via S. Allende, 84081 Baronissi (SA), Italy
R.M.J. Cotterill, BSc, MS, PhD, DSe, FlnstP, C.Phys.
Department of Biophysics, The Technical University of Denmark,
Building 307, DK-2800 Lyngby, Denmark
J,W. Clark
Department of Physics, Washington University,
St Louis, Missouri 63130, USA
Series Editors
J. G. Taylor, BA, BSe, MA, PhD, FlnstP
Director, Centre for Neural Networks, Department of Mathematics,
King's College, Strand, London WC2R 2LS, UK
C. L. T. Mannion, BSe, PhD, MlnstP
Department of Electrical Engineering, University of Surrey, Guildford,
Surrey, GU2 5XH, UK
ISBN-13: 978-3-540-19771-3 e-ISBN-13: 978-1-4471-2001-8
DOl: 10.1007/978-1-4471-2001-8
British Ubrary Cataloguing in Publication Data
A catalogue record for this book is available from the British Ubrary
Ubrary of Congress Cataloging-in-Publication Data
A catalog record for this book is available from the Ubrary of Congress
Apart from any fair dealing for the purposes of research or private study, or criticism
or review, as permitted under the Copyright, Designs and Patents Act 1988, this
publication may only be reproduced, stored or transmitted, in any form or by any
means, with the prior permission in writing of the publishers, or in the case of
reprographic reproduction in accordance with the terms of licences issued by the
Copyright Ucensing Agency. Enquiries concerning reproduction outside those terms
should be sent to the publishers.
(Q Springer-Verlag London Umited 1992
The use of registered names, trademarks etc. in this publication does not imply, even
in the absence of a specific statement, that such names are exempt from the relevant
laws and regulations and therefore free for general use.
The publisher makes no representation, express or implied, with regard to the
accuracy of the information contained in this book and cannot accept any legal
responsibility or liability for any errors or omissions that may be made.
Typesetting: Camera ready by author
34/3830-543210 Printed on acid-free paper
PREFACE
The papers in this volume cover a unique range of topics in which neural
networks playa fundamental role. Firstly there are problems raised in the
general analysis of neural network activity. Various types of network
architectures and nodes are described, as are the training rules. The
resultant framework of theory is important in looking at problems of
application of such networks.
Secondly there is the new paradigm of oscillatory networks. These
range from applications to early visual processing to higher level cognitive
activity, and involve networks using correlations of neuronal activity to
those which are coupled relaxation oscillators. The dynamics range from
that of simple S' oscillators to more complex integrate-and-fire neurons
with sets of ionic currents of quite general time courses. The activity in
this area is part of the important bridge between neural computation (as
a branch of artificial intelligence) and computational neuroscience (as a
branch of neurobiology).
Thirdly there are contributions on the new paradigm of doing science
by neural networks. In particular there are essays on the determination of
structural properties of proteins, on nuclear phenomenology, and on
resonance searches in high energy physics, as well as on the investigation
of information storage. Finally there are more direct biological applications
of neural networks, especially in modelling various aspects of visual
cortical architecture and of visual processing.
The contributions are up-dated versions of papers given at the
Workshop on Complex Dynamics in Neural Networks, June 17-21 1991
at "ASS, Vietri, Italy. We would like to thank the Director of "ASS and his
staff for making the meeting possible, and for providing the beautiful
environment of Vietri in which the meeting was held.
January 1992 J.G. Taylor
CONTENTS
Contributors.................................................................................... ix
Foundations of Neural Net Dynamics
Information and Pattern Capacities in Neural Associative Memories
with Feedback for Sparse Memory Patterns
G. Palm and F. T. Sommer............................................................... 3
Associative Reinforcement Training Using Probabilistic RAM Nets
D. Gorse.......................................................................................... 19
Unlearning and its Relevance to REM Sleep: Decorrelating
Correlated Data
J.L. van Hemmen and N. Klemmer.................................................. 30
Optimal Architectures and High-Order Networks
K.E Kurten ...................................................................................... 44
Deterministic Networks with Ternary Neurons
M. Marinaro..................................................................................... 57
Nets, Structure, Hierarchy
E.R. Caianiello ................................................................................. 75
Training Random Asymmetric 'Neural' Networks Towards Chaos
A Progress Report
P. C. McGuire, G. C. Littlewort, C. Pershing and J. Ratelski............... 90
Complex Dynamics of a Discrete Time Model of a Neuron
P.C. Bressloff .................................................................................. 103
Oscillatory Neural Networks
Correlated Neuronal Firing: A Clue to the Integrative Functions of
Cortex?
A.K. Engel, P. Konig and W. Singer ........ ............ .............. ....... ........ 125
Two-Layered Physiology-Oriented Neuronal Network Models that
Combine Dynamic Feature Linking via Synchronization with a
Classical Associative Memory
M. Arndt, P. Dicke, M. Erb, R. Eckhom and H.J. Reitboeck ............. 140
Theoretical Framework for Analysing the Behaviour of Real and
Simulated Neural Networks
A. V. Holden, J. V. Tucker, B. C. Thompson, D. Withington
and H. Zhang.................................................................................. 155
viii Contents
Coupled Neuronal Oscillatory Systems
J. Hyde ........................................................................................... 170
Gamma-Band Oscillations in a Cortical Model
R.M.J. Cotterill and C. Nielsen......................................................... 180
Information Processing by Dynamical Interaction of Oscillatory
Modes in Coupled Cortical Networks
B. Baird................. ..................... ..................................................... 191
Analysis of Oscillatory Regimes of a Coupled Neural Oscillator
System with Application to Visual Cortex Modeling
G.N. Borisyuk, R.M. Borisyuk, and A. I. Khibnik ................................ 208
Systems of Relaxation Oscillators with Time-Delayed Coupling
E. Niebur, H.G. Schuster and D.M. Kammen ................. .................. 226
Cortical Coherent Activity Induced by Thalamic Oscillations
A. Destexhe and A. Babloyantz ....................................................... 234
A 'Microscopic' Model of Collective Oscillations in the Cortex
J.L. van Hemmen, W. Gerstner and R. Ritz...................................... 250
Temporal Processing in Brain Activity
J.G. Taylor....................................................................................... 258
Scientific Applications of Neural Networks
Structural Properties of Proteins Predicted by Neural Networks
H. Bohr ........................................................................................... 293
Nuclear Phenomenology with Neural Nets
J. W. Clark, S. Gazula and H Bohr................................................... 305
Applying Neural Networks to Resonance Search in High Energy
Physics
J. Rau, B. Muller and R.G. Palmer ................................................... 323
Visual Comparison of Information Storage in Various Neural
Network Models
C. Aberger, R. Folk, K.E. Kurten and H Schweng .................... ....... 330
Biological Applications of Neural Networks
Activation Dynamics of Space-Variant Continuous Networks
HA. Mallot and F. Giannakopoulos ................................................. 341
Hierarchical Neural Representations by Synchronized Activity:
A Concept for Visual Pattem Recognition
G. Hartmann ................................................................................... 356
Author Index.................................................................................... 370
CONTRIBUTORS
Aberger, C.
Institut fOr Theoretische Physik, Johannes-Kepler-UniversitEit, A-4040 Unz, Austria
Arndt, M.
Department of Biophysics, Philipps-University Marburg, Renthof 7, D-3550 Marburg,
Germany
Babloyantz, A.
Universite Ubre de Bruxelles, CP 231 -Campus de la Plaine, Boulevard du
Triomphe, B-1 050 Bruxelles, Belgium
Baird, B.
Department of Mathematics, and Department of Molecular and Cell Biology,
University of California Berkeley, Berkeley, CA 94720, USA
Bohr, H.
Noyes Laboratory, and School of Chemical Sciences, University of Illinois at
Urbana-Champaign, Urbana, IL 61801, USA
Borisyuk, G.N.
Research Computing Center of the USSR Academy of Sciences, Pushchino,
Moscow 142292, USSR
Borisyuk, A.M.
Research Computing Center of the USSR Academy of Sciences, Pushchino,
Moscow 142292, USSR
Bressloff, P.C.
GEC-Marconi Ud, Hirst Research Centre, East Lane, Wembley, HA9 7PP, UK
Caianiello, E.R.
Dipartimento di Rsica Teorica, Universita di Salerno, Via S. Allende,
84081 Baronissi (SA), Italy
Clark, J.W.
McDonnell Center for the Space Sciences, and Department of Physics,
Washington University, St. Louis, MO 63130, USA
Cotterill, A.M.J.
Division of Biophysics, Technical University of Denmark, DK-2800 Lyngby, Denmark
Destexhe, A.
Universite Ubre de Bruxelles, CP 231 -Campus de la Plaine, Boulevard du
Triomphe, B-1050 Bruxelles, Belgium
x Contributors
Dicke, P.
Department of Biophysics, Philipps-University Marburg, Renthof 7, D-3550 Marburg,
Germany
Eckhorn, R.
Department of Biophysics, Philipps-University Marburg, Renthof 7, D-3550 Marburg,
Germany
Engel, A.K.
Max-Planck-Institut fOr Hirnforschung, D-6000 FrankfurtlMain, Germany
Erb, M.
Department of Biophysics, Philipps-University Marburg, Renthof 7, D-3550 Marburg,
Germany
Folk, R.
Institut fOr Theoretische Physik, Johannes-Kepler-Universitiit, A-4040 linz, Austria
Gazula, S.
McDonnell Center for the Space Sciences, and Department of Physics,
Washington University, St. Louis, MO 63130, USA
Gerstner, W.
Physik-Department der Technische Universitiit Munchen, D-8046 Garching
bei Munchen, Germany
GiannakopoulOs, F.
Mathematisches Institut der Universitiit zu Koln, D-5000 Koln, Germany
Gorse, D.
Department of Computer Science, University College, Gower Street,
London, WC1 E 6BT, UK
Hartmann, G.
Fachbereich 14 Elektrotechnik Universitiit-Gesamthochschule-Paderborn,
Pohlweg 47-49, D-4790 Paderborn, Germany
Holden, A.V.
Department of Physiology, and Centre for Nonlinear Studies, University of Leeds,
Leeds, LS2 9JT, UK
Hyde, J.
Department of Applied Mathematical Studies, and Department of Physiology,
University of Leeds, Leeds, LS2 9JT, UK
Kammen, D.M.
Department of Physics, Harvard University, Cambridge, MA 02138, USA
Khibnik, A.I.
Research Computing Center of the USSR Academy of Sciences, Pushchino,
Moscow, 142292, USSR
and
Department of Computer Science, Katholieke Universiteit Leuven, B-3001 Leuven,
Belgium
Contributors xi
Klemmer, N.
Physik-Department der Technische Universitat MOnchen, 0-8046 Garching
bei MOnchen, Germany
Konig, P.
Max-Planck-Institut fOr Hirnforschung, 0-6000 FrankfurtlMain, Germany
KOrten, K.E.
Institut fOr Theoretische Physik, Johannes-Kepler-Universitat, A-4040 Linz, Austria
and
Institut fOr Neuroinformatik, Ruhr-Universitat-Bochum, 0-4630 Bochum, Germany
Uttlewort, G.C.
Department of Physics, University of Arizona, Tucson, AZ 85721, USA
and
Department of Cognitive Science, University of California, San Diego, La Jolla,
CA 92093-0515, USA
Mallot, HA
Institut fOr Neuroinformatik, Ruhr-Universitat-Bochum, 0-4630 Bochum, Germany
Marinaro, M.
Dipartimento di Rsica Teorica, Universita di Salerno, Via S. Allende,
84081 Baronissi (SA), Italy
McGuire, P.C.
Department of Physics, University of Arizona, Tucson, AZ 85721, USA
MOiler, B.
Department of Physics, Duke University, Durham, NC 27706, USA
Niebur, E.
Computation and Neural Systems Program, California Institute of Technology,
Pasadena, CA 91125, USA
Nielsen, C.
Division of Biophysics, Technical University of Denmark, DK-2800 Lyngby, Denmark
Palm,G.
C.u.O. Vogt Institut fOr Hirnforschung, MoorenstraBe 5, University of DOsseldorf,
0-4000 DOsseldorf 1 , Germany
Palmer, R.G.
Department of Physics, Duke University, Durham, NC 27706, USA
Pershing, C.
Department of Physics, University of Arizona, Tucson, AZ 85721, USA
Rafelski, J.
Department of Physics, University of Arizona, Tucson, AZ 85721, USA
Rau, J.
Department of Physics, Duke University, Durham, NC 27706, USA