Neural Networks and Artificial Intelligence for Biomedical Engineering

314 Pages·1999·9.766 MB·English

by  Donna L. Hudson, Maurice E. Cohen

NEURAL NETWORKS AND
ARTIFICIAL INTELLIGENCE FOR
BIOMEDICAL ENGINEERING
IEEE PRESS SERIES IN BIOMEDICAL ENGINEERING
The focus of our series is to introduce current and emerging technologies to biomed-
ical and electrical engineering practitioners, researchers, and students. This series seeks
to foster interdisciplinary biomedical engineering education to satisfy the needs of the
industrial and academic areas. This requires an innovative approach that overcomes
the difficulties associated with the traditional textbook and edited collections.
Series Editor
Metin Akay
Dartmouth College
Advisory Board
Ingrid Daubechies Murat Kunt Richard Satava
Murray Eden Paul Lauterbur Malvin Teich
James Greenleaf Larry Mclntire Herbert Voigt
Simon Haykin Robert Plonsey Lotfi Zadeh
Editorial Board
Eric W.Abel Gabor Herman Peter Richardson
Peter Adlassing Helene Hoffman Richard Robb
Akram Aldroubi Donna Hudson Joseph Rosen
Erol Basar Yasemin Kahya Christian Roux
Katarzyna Blinowska Michael Khoo Janet Rutledge
Bernadette Bouchon-Meunier Yongmin Kim Wim L. C. Rutten
Tom Brotherton Andrew Laine Alan Sahakian
Sergio Cerutti Rosa Lancini Paul S. Schenker
Jean-Louis Coatrieux Swamy Laxminarayan G. W. Schmid-Schönbein
Maurice Cohen Richard Leahy Ernest Stokely
John Collier Zhi-Pei Liang Ahmed Tewfik
Steve Cowin Jennifer Linderman Nitish Thakor
Jerry Daniels Richard Magin Michael Unser
Andrew Daubenspeck Jaakko Malmivuo Eugene Veklerov
Yves Meyer AlWald
Jaques Duchene
Michael Neuman Bruce Wheeler
Patrick Flandrin
Tim Olson William Williams
Walter Greenleaf
Banu Onaral Andy Yagle
Daniel Hammer
Keith Paulsen Yuan-Ting Zhang
Dennis Healy
NEURAL NETWORKS AND
ARTIFICIAL INTELLIGENCE FOR
BIOMEDICAL ENGINEERING
Donna L. Hudson
UCSF Medical Education Program
University of California, San Francisco
Maurice E. Cohen
UCSF Medical Education Program
University of California, San Francisco
California State University, Fresno
Φ
IEEE Engineering in Medicine and Biology Society, Sponsor
IEEE
PRESS
IEEE Press Series in Biomedical Engineering
Metin Akay, Series Editor
The Institute of Electrical and Electronics Engineers, Inc., New York
This book and other books may be purchased at a discount from the
publisher when ordered in bulk quantities. Contact:
IEEE Press Marketing
Attn: Special Sales
Piscataway, NJ 08855-1331
Fax: [phone]
For more information about IEEE Press products, visit the
IEEE Press Home Page: http://www.ieee.org/press
© 2000 by the Institute of Electrical and Electronics Engineers, Inc.
3 Park Avenue, 17th Floor, New York, NY 10016-5997
All rights reserved. No part of this book may be reproduced in any form,
nor may it be stored in a retrieval system or transmitted in any form,
without written permission from the publisher.
Printed in the United States of America
10 9 8 7 6 5 4 3 21
ISBN 0-7803-3404-3
IEEE Order Number PC5675
Library of Congress Cataloging-in-Publication Data
Hudson, D. L. (Donna L.)
Neural networks and artificial intelligence for biomedical
engineering / Donna L. Hudson, Maurice E. Cohen.
p. cm. — (IEEE Press series in biomedical engineering)
Includes bibliographical references and index.
ISBN 0-7803-3404-3
1. Artificial intelligence—Medical Applications. 2. Neural
networks (Computer science) 3. Expert systems (Computer science)
4. Biomedical engineering—Computer simulation. I. Cohen, M. E.
(Maurice E.) II. Title. III. Series.
R859.7.A78H84 1999
610' .285'63—dc21 99-30757
CIP
To our parents
David and Elvera Hegquist Harder
and
Elias and Katie Hey Cohen
for instilling in us at an early age the importance of knowledge
IEEE Press
445 Hoes Lane, P.O. Box 1331
Piscataway, NJ 08855-1331
IEEE Press Editorial Board
Robert J. Herrick, Editor in Chief
J. B. Anderson S. Furui P. Laplante
P. M. Anderson A. H. Haddad M. Padgett
M. Eden S. Kartalopoulos W. D. Reeve
M. E. El-Hawary D. Kirk G. Zobrist
Kenneth Moore, Director of IEEE Press
Linda Matarazzo, Assistant Editor
Surendra Bhimani, Production Editor
IEEE Engineering in Medicine and Biology Society, Sponsor
EMB-S Liaison to IEEE Press, Metin Akay
Cover Design: William T. Donnelly, WT Design
Technical Reviewers
Dr. Kim Dremstrup Nielsen, Aalborg University, Denmark
Dr. Eric W. Abel, University of Dundee, United Kingdom
Dr. Tom Brotherton, Orincon Corporation, San Diego, CA
Professor Maria Gini, University of Minnesota
Dr. Ludmila Kuncheva, University of Wales, Bangor
Dr. James Anderson, Brown University, RI
Books of Related Interest from the IEEE Press . . .
PRINCIPLES OF MRLA Signal Processing Perspective
Zhi-Pei Liang and Paul C. Lauterbur
2000 Hardcover 416 pp IEEE Order No. PC5769 ISBN 0-7803-4723-4
PHYSIOLOGICAL CONTROL SYSTEMS: Analysis, Simulation, and Estimation
Michael Khoo
2000 Hardcover 336 pp IEEE Order No. PC5680 ISBN 0-7803-3408-6
TIME FREQUENCY AND WAVELETS IN BIOMEDICAL SIGNAL PROCESSING
Edited by Metin Akay
1998 Hardcover 768 pp IEEE Order No. PC5619 ISBN 0-7803-1447-7
BIOMEDICAL TELEMETRY: Sensing and Transmitting Biological Information from
Animals and Man, Second Edition
R. Stuart Mackay
1993 Softcover 560 pp IEEE Order No. PP3772 ISBN 0-7803-4718-8
Contents
PREFACE xxi
ACKNOWLEDGMENTS xxiii
Overview 1
O.l Early Biomedical Systems 1
O.l.l History 1
0.1.2 Medical Records 2
0.1.3 Drawbacks of Traditional Approaches 3
0.1.4 Numerical versus Symbolic Approaches 3
0.2 Medical and Biological Data 3
0.2.1 Binary data 4
0.2.2 Categorical data 4
0.2.3 Integer Data 4
0.2.4 Continuous Data 4
0.2.5 Fuzzy Data 4
0.2.6 Temporal Data 5
0.2.7 Time Series Data 6
0.2.8 Image Data 8
0.3 Organization of the Book 8
References 9
PARTI NEURAL NETWORKS
CHAPTER 1 Foundations of Neural Networks 13
1.1 Objectives of Neural Networks 13
1.1.1 Modeling Biomedical Systems 13
1.1.2 Establishment of Decision-Making Systems 14
vii
viii Contents
1.2 Biological Foundations of Neural Networks 14
1.2.1 Structure of the Neuron 14
1.2.2 Structure of the Central Nervous System 16
1.3 Early Neural Models 16
1.3.1 McCulloch and Pitts Neuron 16
1.3.2 Hebbian Learning 17
1.3.3 ADALINE 17
1.3.4 Rosenblatt Perceptron 17
1.3.5 Problems with Early Systems 18
1.4 Precursor to Current Models: Pattern Classification 19
1.4.1 Feature Extraction 19
1.4.2 Supervised Learning 21
1.4.3 Unsupervised Learning 21
1.4.4 Learning Algorithms 22
1.5 Resurgence of the Neural Network Approach 24
1.6 Basic Concepts 25
1.6.1 Artificial Neurons 25
1.6.2 Selection of Input Nodes 25
1.6.3 Network Structure 25
1.6.3.1 Feed-Forward Networks 25
1.6.3.2 Feed-Backward Networks 25
1.6.4 Learning Mechanism 26
1.6.5 Output 26
1.7 Summary 26
References 27
CHAPTER 2 Classes of Neural Networks 29
2.1 Basic Network Properties 29
2.1.1 Terminology 29
2.1.2 Structure of Networks 29
2.1.3 Computational Properties of Nodes 30
2.1.4 Algorithm Design 31
2.2 Classification Models 32
2.3 Association Models 32
2.3.1 Hopfield Nets 33
2.3.1.1 Theoretical Basis 33
2.3.2 Other Associative Memory Approaches 34
2.3.2.1 Theoretical Basis of Bidirectional
Associative Memory (BAM) 34
2.3.3 Hamming Net 36
2.3.3.1 Theoretical Basis 36
2.3.4 Applications of Association Models 37
2.4 Optimization Models 38
2.4.1 Hopfield Net 38
2.4.2 Boltzmann Machines 39
2.4.2.1 Theoretical Basis 39
2.4.3 Applications of Optimization Models 40
2.5 Self-Organization Models 40
2.6 Radial Basis Functions (RBFs) 41
2.6.1 Theoretical Basis 41
2.6.2 Applications of Radial Basis Functions 42
2.7 Summary 43
References 43
CHAPTER 3 Classification Networks and Learning 45
3.1 Network Structure 45
3.1.1 Layer Definition 45
3.1.2 Input Layer 45
3.1.3 Hidden Layer 45
3.1.4 Output Layer 45
3.2 Feature Selection 46
3.2.1 Types of Variables 46
3.2.2 Feature Vectors 46
3.2.3 Image Data 47
3.2.4 Time Series Data 47
3.2.4.1 Chaotic Analysis of Time Series 48
3.2.4.2 Graphical Measures of Chaos 49
3.2.4.3 Numerical Measures of Chaos 49
3.2.5 Issues of Dimensionality 50
3.3 Types of Learning 51
3.3.1 Supervised Learning 51
3.3.1.1 Selection of Training and Test Sets 51
3.3.1.2 Selection of Learning Algorithm 52
3.3.2 Unsupervised Learning 52
3.3.3 Causal Models 55
3.4 Interpretation of Output 55
3.5 Summary 55
References 56
CHAPTER 4 Supervised Learning 59
4.1 Decision Surfaces 59
4.2 Two-Category Separation, Linearly Separable Sets 60
4.2.1 Fisher's Linear Discriminant 60
4.2.2 Gradient Descent Procedures 61
4.2.3 Perceptron Algorithm 62
4.2.4 Relaxation Procedures 62
4.2.5 Potential Functions 63
4.3 Nonlinearly Separable Sets 64
4.3.1 Nonlinear Discriminant Functions 64
4.3.2 Hypernet, A Nonlinear Potential Function Algorithm 64
4.3.3 Categorization of Nonlinearly Separable Sets 65
4.3.3.1 Minimum Squared Error Procedures (MSEs) 65
4.3.3.2 Ho-Kashyap Procedure 66
4.4 Multiple Category Classification Problems 66
4.4.1 Extension of Fisher Discriminant 66
Contents
4.4.2 Kesler Construction 67
4.4.3 Backpropagation 68
4.5 Relationship to Neural Network Models 69
4.6 Comparison of Methods 70
4.6.1 Convergence and Stability 70
4.6.2 Training Time 70
4.6.3 Predictive Power 70
4.7 Applications 71
4.7.1 Single-Category Classification 71
4.7.2 Multicategory Classification 72
4.7.3 Reduction of Nodes 74
4.8 Summary 74
References 76
CHAPTER 5 Unsupervised Learning 79
5.1 Background 79
5.2 Clustering 79
5.2.1 Basic Isodata 79
5.2.2 Similarity Measures 80
5.2.3 Criterion Functions 80
5.2.3.1 Sum of Squared Error Criteria 80
5.2.3.2 Minimum Error Criteria 81
5.2.3.3 Scattering Criteria 81
5.2.3.4 Iterative Optimization 81
5.2.4 Hierarchical Clustering 82
5.2.5 Metrics 82
5.3 Kohonen Networks and Competitive Learning 83
5.4 Hebbian Learning 85
5.5 Adaptive Resonance Theory (ART) 86
5.6 Applications 88
5.6.1 Dimensionality Reduction 88
5.6.1.1 Minimization of Criteria Functions 88
5.6.1.2 Clustering and Dimensionality Reduction 89
5.6.2 Biomedical Applications 89
5.6.3 Diagnosis of CAD as a Clustering Problem 89
5.6.4 Other Biomedical Applications 90
5.7 Summary 91
References 92
CHAPTER 6 Design Issues 95
6.1 Introduction 95
6.1.1 Objective of the Model 95
6.1.2 Information Sources 95
6.2 Input Data Types 96
6.2.1 Extracting Information from the Medical Record 96
6.2.2 Using Information from Data Collection Sheets 97
6.2.2.1 Coding Multiple Responses 98
6.2.2.2 Ordering Categorical Data 99