Introduction to Neural Dynamics and Signal Transmission Delay

192 Pages·2001·6 MB·English

by  Jianhong Wu

de Gruyter Series in Nonlinear Analysis and Applications 6
Editors
A. Bensoussan (Paris)
R. Conti (Florence)
A. Friedman (Minneapolis)
K.-H. Hoffmann (Munich)
L. Nirenberg (New York)
A. Vignoli (Rome)
Managing Editors
J. Appell (Würzburg)
V. Lakshmikantham (Melbourne, USA)

Jianhong Wu
Introduction to Neural Dynamics
and
Signal Transmission Delay
W
DE
G_
Walter de Gruyter · Berlin · New York 2001
Author
Jianhong Wu
Department of Mathematics and Statistics
York University
4700 Keele Street
North York, Ontario M3J 1P3
Canada
Mathematics Subject Classification 2000: 34-02; 34K20, 34K15, 92B20
Keywords: Neural networks, Delay, Dynamics, Associative memory
© Printed on acid-free paper which falls within the guidelines of the ANSI
to ensure permanence and durability.
Library of Congress Cataloging-in-Publication Data
Wu, Jianhong.
Introduction to neural dynamics and signal transmission
delay / by Jianhong Wu.
p. cm.
Includes bibliographical references and index.
ISBN 3-11-016988-6
1. Neural networks (Neurobiology) - Mathematical mo-
dels. I. Title.
QP363.3 ,W8 2001
573.8'5—dc21 2001017248
Die Deutsche Bibliothek — Cataloging-in-Publication Data
Wu, Jianhong:
Introduction to neural dynamics and signal transmission delay /
Jianhong Wu. - Berlin ; New York : de Gruyter, 2001
(De Gruyter series in nonlinear analysis and applications ; 6)
ISBN 3-11-016988-6
ISSN 0941-813 X
© Copyright 2001 by Walter de Gruyter & Co. KG, 10785 Berlin, Germany.
All rights reserved, including those of translation into foreign languages. No part of this book
may be reproduced in any form or by any means, electronic or mechanical, including photocopy,
recording, or any information storage and retrieval system, without permission in writing from
the publisher.
Printed in Germany.
Typesetting using the authors' TgX files: I. Zimmermann, Freiburg.
Printing and binding: WB-Druck GmbH & Co., Rieden/Allgäu.
Cover design: Thomas Bonnie, Hamburg.
To Ming

Preface
In the design of a neural network, either for biological modeling, cognitive simulation,
numerical computation or engineering applications, it is important to describe the
dynamics (also known as evolution) of the network. The success in this area in the
early 1980's was one of the main sources for the resurgence of interest in neural
networks, and the current progress towards understanding neural dynamics has been
part of exhaustive efforts to lay down a solid theoretical foundation for this fast growing
theory and for the applications of neural networks.
Unfortunately, the highly interdisciplinary nature of the research in neural net-
works makes it very difficult for a newcomer to enter this important and fascinating
area of modern science. The purpose of this book is to give an introduction to the
mathematical modeling and analysis of networks of neurons from the viewpoint of
dynamical systems. It is hoped that this book will give an introduction to the basic
knowledge in neurobiology and physiology which is necessary in order to understand
several popular mathematical models of neural networks, as well as to some well-
known results and recent developments in the theoretical study of the dynamics of the
mathematical models.
This book is written as a textbook for senior undergraduate and graduate students
in applied mathematics. The level of exposition assumes a basic knowledge of matrix
algebra and ordinary differential equations, and an effort has been made to make the
book as self-contained as possible.
Many neural network models were originally proposed to explain observations in
neurobiology or physiology. Also, understanding human behavior and brain function
is still one of the main motivations for neural network modeling and analysis. It is
therefore of prime importance to understand the basic structure of a single neuron as
well as a network of neurons and to understand the main mechanisms of the neural
signal transmission. This necessary basic knowledge in neuroscience will be collected
in Chapter 1.
Chapter 2 will start with the derivation of general models of biological neural net-
works. In particular, the additive and shunting equations will be presented and a brief
description of the popular signal functions will be given. An abstract formulation,
which is suitable for both biological and artificial networks and treats a network as a
labeled graph, will be provided as well. Two types of network architectures: feedfor-
ward and feedback networks will be identified and various connection topologies will
be described.
In Chapter 3, several simple networks will be presented that perform some elemen-
tary functions such as storing, recalling and recognizing neuron activation patterns.
The important on-center off-surround connection topology and its connection to the
viii
solution of the noise-saturation dilemma will be discussed. Two important issues
related to applications: the choice of signal functions and the determination of the
synaptic coupling coefficients will also be addressed.
A central subject of this book is the long-term behaviors of the network. In Chap-
ter 4, the connection between the convergence and the global attractor and the im-
portant property of content-addressable memory of many networks will be discussed.
The convergence theorem due to Cohen, Grossberg and Hopfield based on LaSalle's
Invariance Principle and its various extensions and modifications will be presented.
Other techniques for establishing the convergence of almost every trajectory of a given
network will also be provided, including the theory of monotone dynamical systems
and the combinatorial matrix theory.
A special feature of this book is its emphasis on the effect of signal delays on the
long-term performance of the networks under consideration. Such time lags exist due
to the finite propagation speeds of neural signals along axons and the finite speeds
of the neurotransmitters across synaptic gaps in a biological neural network and due
to the finite switching speeds of amplifiers (neurons) in artificial neural networks.
In the final chapter, various phenomena associated with the signal delays: delay-
induced instability, nonlinear periodic solutions, transient oscillations, phase-locked
oscillations and changes of the basins of attraction will be demonstrated.
This book grows from the lecture notes given by the author during a summer
graduate course in Neural Dynamics at York University in 1999. I am deeply indebted
to all the students form this class. I especially want to thank Yuming Chen who
typed the whole text, struggled with evolving versions of it and offered many critical
comments and suggestions.
Professor Hugh R. Wilson read several chapters of the manuscript and offered
many helpful comments which are greatly appreciated. I thank my editor, Manfred
Karbe, for his patience, understanding, encouragement and assistance.
It is also a pleasure to acknowledge the financial support from Natural Sciences
and Engineering Research Council of Canada, and from the Network of Centers of
Excellence: Mathematics for Information Technology and Complex Systems.
Toronto, February 2001 Jianhong Wu
Contents
Preface vii
1 The structure of neural networks 1
1.1 The structure of a single neuron 1
1.2 Transmission of neural signals 2
1.3 Neural circuits, CNS and ANN 7
2 Dynamic models of networks 9
2.1 Biological models 9
2.2 Signal functions 14
2.3 General models and network architectures 16
3 Simple networks 24
3.1 Outstars: pattern learning 24
3.2 Instars: pattern recognition 30
3.3 Lateral inhibition: noise-saturation dilemma 34
3.4 Recurrent ON-CTR OFF-SUR networks: signal enhancement
and noise suppression 36
3.5 Determining synaptic weights 50
Appendix. Grossberg's Learning Theorem 57
4 Content-addressable memory storage 61
4.1 Parallel memory storage by competitive networks 62
4.2 Convergence in networks with a nonsymmetric
interconnection matrix 70
4.3 Implementation of CAM: Hopfield networks 75
4.4 Generic convergence in monotone networks 79
5 Signal transmission delays 88
5.1 Neural networks with delay and basic theory 90
5.2 Global stability analysis 95
5.3 Delay-induced instability 99
5.4 Hopf bifurcation of periodic solutions 105
5.5 A network of two neurons: McCulloch-Pitts nonlinearity 115
5.6 Delay-induced transient oscillations 138
5.7 Effect of delay on the basin of attraction 146
χ Contents
5.8 Synchronized activities 154
5.9 Desynchronization, phase-locked oscillation and connecting orbits . . 165
Bibliography 171
Index 179