ebook img

Intelligent Signal Processing PDF

595 Pages·2001·38.642 MB·English
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Intelligent Signal Processing

INTELLIGENT SIGNAL PROCESSING detidE yb nomiS nikyaH McMaster ytisrevinU Hamilton, Ontario, Canada Bart Kosko ytisrevinU of Southern ainrofilaC Los Angeles, ,AC USA A Selected Reprint Volume IEEE PRESS ehT Institute 6f Electrical and Electronics Engineers, Inc., New York This book and other books may be purchased at a discount from the publisher when ordered in bulk quantities. Contact: IEEE Press Marketing Attn: Special Sales 544 Hoes Lane P.O. Box 1331 Piscataway, NJ 08855-1331 Fax: 1÷ 732 189 9334 For more information about IEEE Press products, visit the IEEE Online Catalog and Store: http://www.ieee.org/store. © 2001 by the Institute of Electrical and Electronics Engineers, Inc. 3 Park Avenue, 17th Floor, New York, NY 10016-5997. All rights reserved. No part of this book may be reproduced in any form, nor may it be stored in a retrieval system or transmitted in any form, without written permission from the publisher. Printed in the United States of America. 01 9 8 7 6 5 4 3 2 1 ISBN 0-7803-6010-9 IEEE Order No. PC5860 Library of Congress Cataloging-in-Publication Data Intelligent signal processing / edited by Simon Haykin, Bart Kosko. p. cm. "A selected reprint volume." Includes index. ISBN 0-7803-6010-9 .1 Signal processingmDigital techniques. .2 Intelligent control systems. .3 Adaptive signal processing. I. Haykin, Simon ,.S 1931- II. Kosko, Bart. TK5102.9.I5455 2000 621.382'2mdc21 00-061369 This book is dedicated to Bernard Widrow for laying the foundations of adaptive filters Preface This expanded reprint volume is the first book devoted to ISP has emerged recently in signal processing in much the new field of intelligent signal processing (ISP). It grew the same way that intelligent control has emerged from out of the November 1998 ISP special issue of the IEEE standard linear control theory. Researchers have guessed Proceedings that the two of us coedited. This book contains less at equations to model a complex system's throughput new ISP material and a fuller treatment of the articles that and have instead let so-called "intelligent" or "model- appeared in the ISP special issue. free" techniques guess more for them. Adaptive neural networks have been'the most popular black box tools in ISP. Multilayer perceptrons and radial- basis function networks extend adaptive linear combiners TAHW Is ISP? to the nonlinear domain but require vastly more computa- tion. Other ISP techniques include fuzzy rule-based sys- ISP uses learning and other "smart" techniques to extract tems, genetic algorithms, and the symbolic expert systems as much information as possible from incoming signal and of artificial intelligence. Both neural and fuzzy systems can noise data. It makes few if any assumptions about the learn with supervised and unsupervised techniques. Both statistical structure of signals and their environment. ISP are (like polynomials) universal function approximators: seeks to let the data set tell its own story rather than They can uniformly approximate any continuous function to impose a story on the data in the form of a simple on a compact domain, but this may not be practical in many mathematical model. real-world cases. The property of universal approximation Classical signal processing has largely worked with math- justifies the term "model free" to describe neural and fuzzy ematical models that are linear, local, stationary, and systems even though equations describe their own Gaussian. These assumptions stem from the precomputer throughput structure. They are one-size-fits-all approxima- age. They have always favored closed-form tractability tors that can model any process if they have access to over real-world accuracy, and they are no less extreme enough training data. because they are so familiar. But ISP tools face new problems when we apply them But real systems are nonlinear except for a vanishingly to more real-world problems that are nonlinear, nonlocal, small set'of linear systems. Almost all bell-curve probabil- nonstationary, non-Gaussian, and of high dimension. Prac- ity densities have infinite variance and infinite higher order tical neural systems may require prohibitive computation moments. The set of bell-curve densities itself is a vanishin- to tune the values of their synaptic weights for large sets gly small set in the space of all probability densities. Real- of high-dimensional data. New signal data may require world systems are often highly nonlinear and can depend total retraining or may force the neural network's vast and on many partially correlated variables. The systems can unfathomable set of synapses to forget some of the signal have an erratic or impulsive statistical structure that varies structure it has learned. Blind fuzzy approximators need in time in equally erratic ways. Small changes in the signal a number of if-then rules that grows exponentially with or noise structure can lead to qualitative global changes the dimension of the training data. This volume explores in how the system filters noise or maintains stability. how the ISP tools can address these problems. xvii ecaferP ORGANIZATION OF THE VOLUME .9 Yann LeCun, L6on Bottou, Yoshua Bengio, and Pat- rick Haffner review ways that gradient-descent learn- The 51 chapters in this book give a representative sample ing can train a multilayer perceptron for handwritten of current research in ISP and each has helped extend the character recognition. ISP frontier. Each chapter passed through a full peer- .01 Shigeru Katagiri, Biing-Hwang Juang, and Chin-Hui review filter: Lee show how to use the new technique of generalized probabilistic gradients to solve problems in pattern rec- .1 Steve Mann describes a novel technique that lets one ognition. include human intelligence in the operation of a wear- .11 Lee A. Feldkamp, Timothy M. Feldkamp, and Danil able computer. V. Prokhorov present an adaptive classification scheme .2 Sanya Mitaim and Bart Kosko present the noise pro- that combines both supervised and unsupervised cessing technique of stochastic resonance in a signal learning. processing framework and then show how neural or .21 J. Scott Goldstein, J.R. Guescin and I.S. Reed describe fuzzy or other model-free systems can adaptively add an algebraic procedure based on reduced rank model- many types of noise to nonlinear dynamical systems ing as a basis for intelligent signal processing to improve their signal-to-noise ratios. .31 Simon Haykin and David J. Thomson discuss an adap- .3 Malik Magdon-Ismail, Alexander Nicholson, and tive procedure for the difficult task of detecting a non- Yaser .S Abu-Mostafa explore how additive noise af- stationary target signal in a nonstationary background fects information processing in problems of financial with unknown statistics. engineering. .41 Robert D. Dony and Simon Haykin describe an image .4 Partha Niyogi, Fredrico Girosi, and Tomaso Poggio segmentation system based on a mixture of principal show how prior knowledge and virtual sampling can components. expand the size of a data set that trains a generalized .51 Aapo Hyv~irinen, Patrik Hoyar and Erkki Oja discuss supervised learning system. how sparse coding can denoise images. .5 Kenneth Rose reviews how the search technique of deterministic annealing can optimize the design of un- These chapters show how adaptive systems can solve a supervised and supervised learning systems. wide range of difficult tasks in signal processing that arise .6 Jose C. Principe, Ludong Wang, and Mark A. Motter in highly diverse fields. They are a humble but important use the neural self-organizing map as a tool for the first step on the road to truly intelligent signal processing. local modeling of a nonlinear dynamical system. .7 Lee A. Feldkamp and Gintaras V. Puskorius describe how time-lagged recurrent neural networks can per- Simon Haykin form difficult tasks of nonlinear signal processing. McMaster University .8 Davide Mattera, Francesco Palmieri, and Simon Hay- Hamilton, Ontario, Canada kin describe a semiparametric form of support vector machine for nonlinear model estimation that uses prior Bart Kosko knowledge that comes from a rough parametric model University of Southern California of the system under study. Los Angeles, CA xviii List of Contributors retpahC 1 retpahC 4 Steve Mann Partha Niyogi University of Toronto Massachusetts Institute of Technology Department of Electrical and Computer Engineering Center for Biological and Computational Learning 01 King's College Road, S.F. 1002 Cambridge, MA 92120 Toronto, Ontario, S5M 3G4 CANADA Fredrico Girosi Massachusetts Institute of Technology retpahC 2 Center for Biological and Computational Learning Bart Kosko Cambridge, MA 92120 Signal and Image Processing Institute Tomaso Poggio Department of Electrical Engineering-Systems Massachusetts Institute of Technology University of Southern California Center for Biological and Computational Learning Los Angeles, California 4652-98009 Cambridge, MA 92120 Sanya Mitaim Signal and Image Processing Institute retpahC 5 Department of Electrical Engineering-Systems Kenneth Rose University of Southern California Department of Electrical and Computer Engineering Los Angeles, California 4652-98009 University of California Santa Barbara, _AC 60139 retpahC 3 retpahC 6 Malik Magdon-Ismail Jose .C Principe Department of Electrical Engineering Computational NeuroEngineering Laboratory California Institute of Technology University of Florida 39-631 Pasadena, CA 52119 Gainsville, FL 11623 Alexander Nicholson Uudong Wang Department of Electrical Engineering Computational NeuroEngineering Laboratory California Institute of Technology University of Florida 39-631 Pasadena, CA 52119 Gainsville, FL 11623 Yaser .S Abu-Mostafa Mark A. Motter Department of Electrical Engineering Computational NeuroEngineering Laboratory California Institute of Technology University of Florida 39-631 Pasadena, CA 52119 Gainsville, FL 11623 xix INTELLIGENT SIGNAL PROCESSING detidE yb nomiS nikyaH McMaster ytisrevinU Hamilton, Ontario, Canada Bart Kosko ytisrevinU of Southern ainrofilaC Los Angeles, ,AC USA A Selected Reprint Volume IEEE PRESS ehT Institute 6f Electrical and Electronics Engineers, Inc., New York This book and other books may be purchased at a discount from the publisher when ordered in bulk quantities. Contact: IEEE Press Marketing Attn: Special Sales 544 Hoes Lane P.O. Box 1331 Piscataway, NJ 08855-1331 Fax: 1÷ 732 189 9334 For more information about IEEE Press products, visit the IEEE Online Catalog and Store: http://www.ieee.org/store. © 2001 by the Institute of Electrical and Electronics Engineers, Inc. 3 Park Avenue, 17th Floor, New York, NY 10016-5997. All rights reserved. No part of this book may be reproduced in any form, nor may it be stored in a retrieval system or transmitted in any form, without written permission from the publisher. Printed in the United States of America. 01 9 8 7 6 5 4 3 2 1 ISBN 0-7803-6010-9 IEEE Order No. PC5860 Library of Congress Cataloging-in-Publication Data Intelligent signal processing / edited by Simon Haykin, Bart Kosko. p. cm. "A selected reprint volume." Includes index. ISBN 0-7803-6010-9 .1 Signal processingmDigital techniques. .2 Intelligent control systems. .3 Adaptive signal processing. I. Haykin, Simon ,.S 1931- II. Kosko, Bart. TK5102.9.I5455 2000 621.382'2mdc21 00-061369 This book is dedicated to Bernard Widrow for laying the foundations of adaptive filters Preface This expanded reprint volume is the first book devoted to ISP has emerged recently in signal processing in much the new field of intelligent signal processing (ISP). It grew the same way that intelligent control has emerged from out of the November 1998 ISP special issue of the IEEE standard linear control theory. Researchers have guessed Proceedings that the two of us coedited. This book contains less at equations to model a complex system's throughput new ISP material and a fuller treatment of the articles that and have instead let so-called "intelligent" or "model- appeared in the ISP special issue. free" techniques guess more for them. Adaptive neural networks have been'the most popular black box tools in ISP. Multilayer perceptrons and radial- basis function networks extend adaptive linear combiners TAHW Is ISP? to the nonlinear domain but require vastly more computa- tion. Other ISP techniques include fuzzy rule-based sys- ISP uses learning and other "smart" techniques to extract tems, genetic algorithms, and the symbolic expert systems as much information as possible from incoming signal and of artificial intelligence. Both neural and fuzzy systems can noise data. It makes few if any assumptions about the learn with supervised and unsupervised techniques. Both statistical structure of signals and their environment. ISP are (like polynomials) universal function approximators: seeks to let the data set tell its own story rather than They can uniformly approximate any continuous function to impose a story on the data in the form of a simple on a compact domain, but this may not be practical in many mathematical model. real-world cases. The property of universal approximation Classical signal processing has largely worked with math- justifies the term "model free" to describe neural and fuzzy ematical models that are linear, local, stationary, and systems even though equations describe their own Gaussian. These assumptions stem from the precomputer throughput structure. They are one-size-fits-all approxima- age. They have always favored closed-form tractability tors that can model any process if they have access to over real-world accuracy, and they are no less extreme enough training data. because they are so familiar. But ISP tools face new problems when we apply them But real systems are nonlinear except for a vanishingly to more real-world problems that are nonlinear, nonlocal, small set'of linear systems. Almost all bell-curve probabil- nonstationary, non-Gaussian, and of high dimension. Prac- ity densities have infinite variance and infinite higher order tical neural systems may require prohibitive computation moments. The set of bell-curve densities itself is a vanishin- to tune the values of their synaptic weights for large sets gly small set in the space of all probability densities. Real- of high-dimensional data. New signal data may require world systems are often highly nonlinear and can depend total retraining or may force the neural network's vast and on many partially correlated variables. The systems can unfathomable set of synapses to forget some of the signal have an erratic or impulsive statistical structure that varies structure it has learned. Blind fuzzy approximators need in time in equally erratic ways. Small changes in the signal a number of if-then rules that grows exponentially with or noise structure can lead to qualitative global changes the dimension of the training data. This volume explores in how the system filters noise or maintains stability. how the ISP tools can address these problems. xvii

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.