Silicon Implementation of Pulse Coded Neural Networks PDF
Preview Silicon Implementation of Pulse Coded Neural Networks
SILICON IMPLEMENTATION OF PULSE CODED NEURAL NETWORKS THE KLUWER INTERNATIONAL SERIES IN ENGINEERING AND COMPUTER SCIENCE SILICON IMPLEMENTATI ON OF PULSE CODED NEURAL NETWORKS edited by Mona E. Zaghloul The George Washington University Jack L. Meador Washington State University Robert W. Newcomb University of Maryland ~. " SPRINGER SCIENCE+BUSINESS MEDIA, LLC Library of Congress Cataloging-in-Publication Data Silicon implementation of pulse coded neural networks / edited by Mona E. Zaghloul, Jack L. Meador, Robert W. Newcomb. p. cm. -- (The Kluwer international series in engineering and computer science) Includes bibliographical references and index. ISBN 978-1-4613-6152-7 ISBN 978-1-4615-2680-3 (eBook) DOI 10.1007/978-1-4615-2680-3 1. Neural networks (Computer science) 2. Semiconductors. 1. Zaghloul, M. E. (Mona Elwakkad) II. Meador, Jack L. III. Newcomb, Robert W. IV. Series. QA76.87.S55 1994 006.3'3--dc20 93-48181 CIP Copyright @ 1994 by Springer Science+Business Media New York Originally published by Kluwer Academic Publishers in 1994 Softcover ceprint of the hardcover 1s t edition 1994 AH rights reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, mechanical, photo-copying, recording, or otherwise, without the prior written permission of the publisher, Springer Science+Business Media, LLC. Printed on acid-free paper. TABLE OF CONTENTS PREFACE vii 1. Some Historical Perspectives on Early Pulse Coded Neural Network Circuits R. W. Newcomb 1 2. Pulse Techniques in Neural VLSI: AReview A. F. Murray 9 3. Silicon Dendritic Trees J. G. Elias 39 4. Silicon Neurons for Phase and Frequency Detection and Pattern Generation M. DeYong and C. Fields 65 5. Pulse Coded Winner-Take-AII Networks J. L. Meador and P. D. Rylander 79 6. Realization ofBoolean Functions Using a Pulse Coded Neuron M. de Savigny andR. W. Newcomb 101 7. Design ofPulse Coded Neural Processing Element Using Modified Neural Type Cells G. Moon andM. E. Zaghloul 113 8. Low-Power Silicon Neurons, Axons and Synapses J. Lazzaro and J. Wawrzynek 153 9. Synchronous Pulse Density Modulation in Neural Network Implementation J. Tomberg 165 vi 10. CMOS Analog Neural Network Systems Based on Oscillatory Neurons B. Linares-Barranco, E. Sanchez-Sinencio, A. Rodriguez-Vazquez andJ. L. Huertas 199 11. ADigital Neural Network Architecture Using Random Pulse Trains G. R. Salam andR. M. Goodman 249 12. An Unsupervised Neural Processor 1. Donald andL. A. Akers 263 INDEX 291 Preface When confronted with the hows and whys of nature's computational engines, some prefer to focus upon neural function: addressing issues of neural system behavior and its relation to natural intelligence. Then there are those who preferthe study ofthe "mechanics" ofneural systems: the nuts and bolts of the "wetware": the neurons and synapses. Those who investigate pulse coded implementations ofartificial neural networks know whatit means to stand at the boundary which lies between these two worlds: notjust asking why natural neural systems behaveas they do, but also how they achieve their marvelous feats. The research results presented in this book not only address moreconventional abstractnotionsofneural-likeprocessing,butalsothemore specific details ofneural-likeprocessors. It has been established for some time that natural neural systems perform a great deal of information processing via electrochemical pulses. Accordingly, pulse coded neural network concepts are receiving increased attention in artificial neural network research. This increased interest is compounded by continuing advances in the field ofVLSIcircuitdesign. This is the first time in history in which it is practical to construct networks of neuron-like circuits of reasonable complexity that can be applied to real problems. We believe that the pioneering work in artificial neural systems presented in this book will lead to furtheradvances that will notonly be useful in some practical sense, but may also provide some additional insight into the operation oftheirnatural counterparts. The idea ofcreating abookdedicatedto pulsecodingin neural VLSI was first conceived at the International Conference on Circuits and Systems by Monazaghloul in the springof 1992. Aspecial session had been organizedby Jack Meador and Robert Newcomb for the conference which was the first of its kind and had attracted a number of researchers from both Europe and the US who are working in the area. A poll ofauthors presenting at that session showed unanimous support for the development of a book which would showcase recent progress in pulse coded implementations. This book represents the combined contributions of both those who were present at that first special session as well as several others who are currently conducting research in this fascinating area. This volume seeks to cover many of the relevant contemporary studies coming out of this newly emerging area. It is essentially a selection, reorganization,and expansion ofrecentpublications. In addition, Prof. Robert viii Newcomb provides a historical perspective on early pulse coded neural network circuits in Chapter 1. This is followed by a review of more recent pulsed neural VLSI techniques given by Alan Murray in Chapter 2. In that chapter, Dr. Murray reviews the variety of pulse coding strategies available and discusses pulse implementation techniques being investigated at the University ofEdinburgh. Silicondendritictrees which areintended toemulate the behavior of spatially extensive biological neurons are then discussed by John Elias in Chapter 3. A model for a typical spiking neuron is described by Mark DeYong and Chris Fields in Chapter 4 and is shown to be useful for phase and frequency detection, filtering, and pattern encoding. In Chapter 5, Jack Meadorand PaulHylander presentapulsecodedwinner-take-all network which encodes decision strength as a variable rate pulse train generated by the winning unit. The Neural Type Cell which is a special circuit that accepts analog inputs and produces pulse coded outputs is used in Chapter 6 by Marc De Savigny and Robert Newcomb to implement Boolean functions as well as by Gyu Moon and Mona Zaghloul in Chapter7 as part ofa unique processing element that can be used as a basic cell for artificial neural network implementation. Low power silicon implementation of Neurons, Axons and Synapses are introduced by John Lazzaro, and John Wawrzynek in Chapter 8. These new implementations improve upon the power dissipation characteristics of the self-resetting neuron and silicon axon described by Carver Mead in his popular 1989 book. Synchronous pulse density modulation techniques are presented in Chapter 9 by Jouni Tomberg. Both switched capacitor and digital implementations of synchronous pulse coded arithmetic are discussed in the context of established network architectures here. The transconductance mode technique is presented in Chapter 10 by Bernabe Linares-Barranco, Edgar Sanchez-Sinencio, Angel Rodriquez Vazquez,andJose Huertas to implementanequivalentcircuitforan oscillatory neuron based on the simplified operation ofthe living neuron. This oscillatory neuron is used to implement and test established neural architectures. In Chapter II a digital architecture that utilizes random bit sequence properties is described by Gamze Salam and Rodney Goodman. The extension of previously established stochastic pulse coded implementation techniques are presented and demonstrated. Finally in Chapter 12, James Donald and Lex Akers present experimental results obtained with two adaptive pulse coded signal processing chips for real-time control. These chips use pulse coded signals for inter-chip communication and analog weights for information storage. In all of this work, novel circuits are presented and innovative techniques for implementing conventional neural network architectures are introduced. ix By way of acknowledgment we would first like to thank all of the contributing authors for their prompt assistance with the details, some requiring their attention on very short notice. We would also like to express special thanks to PaulHylanderand IoanaPirvulescufor theirsuggestions and assistance with theearly drafts ofthe book. We would also like toexpress our gratitude to Holly Snyder for her assistance with the details of final draft preparation. Mona Zaghloul lackMeador RobertNewcomb 1 SOME HISTORICAL PERSPECTIVES ON EARLY PULSE CODED NEURAL NETWORK CIRCUITS Robert W. Newcomb Electrical Engineering Department University ofMaryland, College Park MD, 20742 PERSPECTIVES From the beginnings of mankind the means of brain activity must have fascinated man. And although Galvani had shown in the late 1700s that muscles were excited by electrical activity of the nerves [Galvani 1791, Brazier61] it wasnotknown through mostofthe 1800'swhatwas thebasisfor activity of the brain - indeed it is still unknown how a person thinks. In any event the publication by the Polish neurophysiologist Adolf Beck in the Centralblatt fur Physiologie [Beck 1890], concerning his measurements of electrical activity in thebrain [Beck 1888],causedconsiderablecontroversy as towhomwas the firstoneto achievesuchanaccomplishment. Afteralmostall sides were heard from, the controversy was settled by a further letter to the Centralblatt by Richard Caton calling attention to the measurements he had reported to the August 24, 1875, meeting of the British Medical Association and recorded in the report ofthe meeting [Caton 1875]. Among statements in Caton'soriginalreportis thefollowing: "Whenany partofthegrey matterisin a state of functional activity, its electric current usually exhibits negative variation" [Brazier 61] where by "negative variation" at the time was meant action potentials. Thus, we see that measurements were made on the pulse coded electricalactivity in the brain asearly as 1875.
The list of books you might like
