Adaptive Signal Processing: Next Generation Solutions (Adaptive and Learning Systems for Signal Processing, Communications and Control Series)

428 Pages·2010·5.21 MB·English

by  Tulay Adali

ADAPTIVE SIGNAL
PROCESSING
AdaptiveandLearningSystemsforSignalProcessing,
Communications,andControl
Editor:SimonHaykin
AdaliandHaykin/ADAPTIVESIGNALPROCESSING:NextGenerationSolutions
Beckerman/ADAPTIVECOOPERATIVESYSTEMS
Candy/BAYESIANSIGNALPROCESSING:CLASSICAL,MODERN,ANDPARTICLEFILTERING
METHODS
Candy/MODEL-BASEDSIGNALPROCESSING
ChenandGu/CONTROL-ORIENTEDSYSTEMIDENTIFICATION:AnH1Approach
Chen,Haykin,Eggermont,andBecker/CORRELATIVELEARNING:ABasisforBrainand
AdaptiveSystems
CherkasskyandMulier/LEARNINGFROMDATA:Concepts,Theory,andMethods
CostaandHaykin/MULTIPLE-INPUTMULTIPLE-OUTPUTCHANNELMODELS:Theoryand
Practice
DiamantarasandKung/PRINCIPALCOMPONENTNEURALNETWORKS:Theoryand
Applications
FarrellandPolycarpou/ADAPTIVEAPPROXIMATIONBASEDCONTROL:UnifyingNeural,
FuzzyandTraditionalAdaptiveApproximationApproaches
GiniandRangaswamy/KNOWLEDGE-BASEDRADARDETECTION:Trackingand
Classification
Ha¨nslerandSchmidt/ACOUSTICECHOANDNOISECONTROL:APracticalApproach
Haykin/UNSUPERVISEDADAPTIVEFILTERING:BlindSourceSeparation
Haykin/UNSUPERVISEDADAPTIVEFILTERING:BlindDeconvolution
HaykinandPuthussarypady/CHAOTICDYNAMICSOFSEACLUTTER
HaykinandWidrow/LEAST-MEAN-SQUAREADAPTIVEFILTERS
Hrycej/NEUROCONTROL:TowardsanIndustrialControlMethodology
Hyva¨rinen,Karhunen,andOja/INDEPENDENTCOMPONENTANALYSIS
Kristic´,Kanellakopoulos,andKokotovic´ /NONLINEARANDADAPTIVECONTROLDESIGN
Mann/INTELLIGENTIMAGEPROCESSING
NikiasandShao/SIGNALPROCESSINGWITHALPHA-STABLEDISTRIBUTIONSAND
APPLICATIONS
PassinoandBurgess/STABILITYANALYSISOFDISCRETEEVENTSYSTEMS
Sa´nchez-Pen˜aandSznaier/ROBUSTSYSTEMSTHEORYANDAPPLICATIONS
Sandberg,Lo,Fancourt,Principe,Katagiri,andHaykin/NONLINEARDYNAMICAL
SYSTEMS:FeedforwardNeuralNetworkPerspectives
SellathuraiandHaykin/SPACE-TIMELAYEREDINFORMATIONPROCESSINGFORWIRELESS
COMMUNICATIONS
Spooner,Maggiore,Ordo´n˜ez,andPassino/STABLEADAPTIVECONTROLANDESTIMATION
FORNONLINEARSYSTEMS:NeuralandFuzzyApproximatorTechniques
Tao/ADAPTIVECONTROLDESIGNANDANALYSIS
TaoandKokotovic´ /ADAPTIVECONTROLOFSYSTEMSWITHACTUATORANDSENSOR
NONLINEARITIES
TsoukalasandUhrig/FUZZYANDNEURALAPPROACHESINENGINEERING
VanHulle/FAITHFULREPRESENTATIONSANDTOPOGRAPHICMAPS:FromDistortion-to
Information-BasedSelf-Organization
Vapnik/STATISTICALLEARNINGTHEORY
Werbos/THEROOTSOFBACKPROPAGATION:FromOrderedDerivativestoNeural
NetworksandPoliticalForecasting
YeeandHaykin/REGULARIZEDRADIALBIASFUNCTIONNETWORKS:Theoryand
Applications
ADAPTIVE SIGNAL
PROCESSING
Next Generation Solutions
Edited by
Tu¨lay Adalı
Simon Haykin
Wiley Series in Adaptive and Learning Systems for
Signal Processing, Communication, and Control
Copyright#2010byJohnWiley&Sons,Inc.Allrightsreserved
PublishedbyJohnWiley&Sons,Inc.,Hoboken,NewJersey
PublishedsimultaneouslyinCanada
Coverphoto:Humuhumunukunukua¯pua’a,Hawaiianstatefish.
PhototakenApril2007inHonolulu,HawaiiduringICASSP2007wheretheideaforthebook
wasfirstconceived.Photocopyright#2010byTu¨layAdalı.
Nopartofthispublicationmaybereproduced,storedinaretrievalsystem,ortransmittedinanyformor
byanymeans,electronic,mechanical,photocopying,recording,scanning,orotherwise,exceptaspermitted
underSection107or108ofthe1976UnitedStatesCopyrightAct,withouteitherthepriorwritten
permissionofthePublisher,orauthorizationthroughpaymentoftheappropriateper-copyfeetothe
CopyrightClearanceCenter,Inc.,222RosewoodDrive,Danvers,MA01923,[phone],fax(978)
750-4470,oronthewebatwww.copyright.com.RequeststothePublisherforpermissionshouldbe
addressedtothePermissionsDepartment,JohnWiley&Sons,Inc.,111RiverStreet,Hoboken,NJ07030,
[phone],fax[phone],oronlineathttp://www.wiley.com/go/permission.
LimitofLiability/DisclaimerofWarranty:Whilethepublisherandauthorhaveusedtheirbesteffortsin
preparingthisbook,theymakenorepresentationsorwarrantieswithrespecttotheaccuracyorcompleteness
ofthecontentsofthisbookandspecificallydisclaimanyimpliedwarrantiesofmerchantabilityorfitnessfor
aparticularpurpose.Nowarrantymaybecreatedorextendedbysalesrepresentativesorwrittensales
materials.Theadviceandstrategiescontainedhereinmaynotbesuitableforyoursituation.Youshould
consultwithaprofessionalwhereappropriate.Neitherthepublishernorauthorshallbeliableforanylossof
profitoranyothercommercialdamages,includingbutnotlimitedtospecial,incidental,consequential,or
otherdamages.
Forgeneralinformationonourotherproductsandservicesorfortechnicalsupport,pleasecontactour
CustomerCareDepartmentwithintheUnitedStatesat(800)762-2974,outsidetheUnitedStatesat
[phone]orfax(317)572-4002.
Wileyalsopublishesitsbooksinavarietyofelectronicformats.Somecontentthatappearsinprintmay
notbeavailableinelectronicformats.FormoreinformationaboutWileyproducts,visitourwebsiteat
www.wiley.com.
LibraryofCongressCataloging-in-PublicationData:
Adaptivesignalprocessing:nextgenerationsolutions/[editedby]Tu¨layAdalı,SimonHaykin.
p. cm.
Includesbibliographicalreferences.
ISBN978-0-470-19517-8(cloth)
1. Adaptivesignalprocessing. I. Adalı,Tu¨lay II. Haykin,Simon,1931–
TK5102.9.A288 2010
621.38202—dc22 2009031378
PrintedintheUnitedStatesofAmerica
10 9 8 7 6 5 4 3 2 1
CONTENTS
Preface xi
Contributors xv
Chapter1 Complex-ValuedAdaptiveSignalProcessing 1
1.1 Introduction / 1
1.1.1 WhyComplex-ValuedSignalProcessing / 3
1.1.2 OutlineoftheChapter / 5
1.2 Preliminaries / 6
1.2.1 Notation / 6
1.2.2 EfficientComputationofDerivativesintheComplexDomain / 9
1.2.3 Complex-to-RealandComplex-to-ComplexMappings / 17
1.2.4 SeriesExpansions / 20
1.2.5 StatisticsofComplex-ValuedRandomVariablesand
RandomProcesses / 24
1.3 OptimizationintheComplexDomain / 31
1.3.1 BasicOptimizationApproachesinRN / 31
1.3.2 VectorOptimizationinCN / 34
1.3.3 MatrixOptimizationinCN / 37
1.3.4 Newton-VariantUpdates / 38
1.4 WidelyLinearAdaptiveFiltering / 40
1.4.1 LinearandWidelyLinearMean-SquareErrorFilter / 41
1.5 NonlinearAdaptiveFilteringwithMultilayerPerceptrons / 47
1.5.1 ChoiceofActivationFunctionfortheMLPFilter / 48
1.5.2 DerivationofBack-PropagationUpdates / 55
1.6 ComplexIndependentComponentAnalysis / 58
1.6.1 ComplexMaximumLikelihood / 59
1.6.2 ComplexMaximizationofNon-Gaussianity / 64
1.6.3 MutualInformationMinimization:ConnectionstoMLandMN / 66
1.6.4 DensityMatching / 67
1.6.5 NumericalExamples / 71
1.7 Summary / 74
v
vi CONTENTS
1.8 Acknowledgment / 76
1.9 Problems / 76
References / 79
Chapter2 RobustEstimationTechniquesforComplex-Valued
RandomVectors 87
2.1 Introduction / 87
2.1.1 SignalModel / 88
2.1.2 OutlineoftheChapter / 90
2.2 StatisticalCharacterizationofComplexRandomVectors / 91
2.2.1 ComplexRandomVariables / 91
2.2.2 ComplexRandomVectors / 93
2.3 ComplexEllipticallySymmetric(CES)Distributions / 95
2.3.1 Definition / 96
2.3.2 CircularCase / 98
2.3.3 TestingtheCircularityAssumption / 99
2.4 ToolstoCompareEstimators / 102
2.4.1 RobustnessandInfluenceFunction / 102
2.4.2 AsymptoticPerformanceofanEstimator / 106
2.5 ScatterandPseudo-ScatterMatrices / 107
2.5.1 BackgroundandMotivation / 107
2.5.2 Definition / 108
2.5.3 M-EstimatorsofScatter / 110
2.6 ArrayProcessingExamples / 114
2.6.1 Beamformers / 114
2.6.2 SubspaceMethods / 115
2.6.3 EstimatingtheNumberofSources / 118
2.6.4 SubspaceDOAEstimationforNoncircularSources / 120
2.7 MVDRBeamformersBasedonM-Estimators / 121
2.7.1 TheInfluenceFunctionStudy / 123
2.8 RobustICA / 128
2.8.1 TheClassofDOGMAEstimators / 129
2.8.2 TheClassofGUTEstimators / 132
2.8.3 CommunicationsExample / 134
2.9 Conclusion / 137
2.10 Problems / 137
References / 138
Chapter3 TurboEqualization 143
3.1 Introduction / 143
3.2 Context / 144
CONTENTS vii
3.3 CommunicationChain / 145
3.4 TurboDecoder:Overview / 147
3.4.1 BasicPropertiesofIterativeDecoding / 151
3.5 Forward-BackwardAlgorithm / 152
3.5.1 WithIntersymbolInterference / 160
3.6 SimplifiedAlgorithm:InterferenceCanceler / 163
3.7 CapacityAnalysis / 168
3.8 BlindTurboEqualization / 173
3.8.1 DifferentialEncoding / 179
3.9 Convergence / 182
3.9.1 BitErrorProbability / 187
3.9.2 OtherEncoderVariants / 190
3.9.3 EXITChartforInterferenceCanceler / 192
3.9.4 RelatedAnalyses / 194
3.10 MultichannelandMultiuserSettings / 195
3.10.1 Forward-BackwardEqualizer196
3.10.2 InterferenceCanceler197
3.10.3 MultiuserCase198
3.11 ConcludingRemarks / 199
3.12 Problems / 200
References / 206
Chapter4 SubspaceTrackingforSignalProcessing 211
4.1 Introduction / 211
4.2 LinearAlgebraReview / 213
4.2.1 EigenvalueValueDecomposition / 213
4.2.2 QRFactorization / 214
4.2.3 VariationalCharacterizationofEigenvalues/Eigenvectors
ofRealSymmetricMatrices / 215
4.2.4 StandardSubspaceIterativeComputationalTechniques / 216
4.2.5 CharacterizationofthePrincipalSubspaceofaCovariance
MatrixfromtheMinimizationofaMeanSquareError / 218
4.3 ObservationModelandProblemStatement / 219
4.3.1 ObservationModel / 219
4.3.2 StatementoftheProblem / 220
4.4 PreliminaryExample:Oja’sNeuron / 221
4.5 SubspaceTracking / 223
4.5.1 SubspacePower-BasedMethods / 224
4.5.2 ProjectionApproximation-BasedMethods / 230
4.5.3 AdditionalMethodologies / 232
4.6 EigenvectorsTracking / 233
4.6.1 RayleighQuotient-BasedMethods / 234
viii CONTENTS
4.6.2 EigenvectorPower-BasedMethods / 235
4.6.3 ProjectionApproximation-BasedMethods / 240
4.6.4 AdditionalMethodologies / 240
4.6.5 ParticularCaseofSecond-OrderStationaryData / 242
4.7 ConvergenceandPerformanceAnalysisIssues / 243
4.7.1 AShortReviewoftheODEMethod / 244
4.7.2 AShortReviewofaGeneralGaussianApproximation
Result / 246
4.7.3 ExamplesofConvergenceandPerformanceAnalysis / 248
4.8 IllustrativeExamples / 256
4.8.1 DirectionofArrivalTracking / 257
4.8.2 BlindChannelEstimationandEqualization / 258
4.9 ConcludingRemarks / 260
4.10 Problems / 260
References / 266
Chapter5 ParticleFiltering 271
5.1 Introduction / 272
5.2 MotivationforUseofParticleFiltering / 274
5.3 TheBasicIdea / 278
5.4 TheChoiceofProposalDistributionandResampling / 289
5.4.1 ChoiceofProposalDistribution / 290
5.4.2 Resampling / 291
5.5 SomeParticleFilteringMethods / 295
5.5.1 SIRParticleFiltering / 295
5.5.2 AuxiliaryParticleFiltering / 297
5.5.3 GaussianParticleFiltering / 301
5.5.4 ComparisonoftheMethods / 302
5.6 HandlingConstantParameters / 305
5.6.1 Kernel-BasedAuxiliaryParticleFilter / 306
5.6.2 Density-AssistedParticleFilter / 308
5.7 Rao–Blackwellization / 310
5.8 Prediction / 314
5.9 Smoothing / 316
5.10 ConvergenceIssues / 320
5.11 ComputationalIssuesandHardwareImplementation / 323
5.12 Acknowledgments / 324
5.13 Exercises / 325
References / 327