Table Of Content

EURASIP Journal on Audio, Speech, and Music Processing Adaptive Partial-Update and Sparse System Identification Guest Editors: Kutluyıl Dog˘ançay and Patrick A. Naylor Adaptive Partial-Update and Sparse System Identification EURASIP Journal on Audio, Speech, and Music Processing Adaptive Partial-Update and Sparse System Identification Guest Editors: Kutluyıl Dog˘ançay and Patrick A. Naylor Copyright©2007HindawiPublishingCorporation.Allrightsreserved. Thisisaspecialissuepublishedinvolume2007of“EURASIPJournalonAudio,Speech,andMusicProcessing.”Allarticlesareopen accessarticlesdistributedundertheCreativeCommonsAttributionLicense,whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalworkisproperlycited. Editor-in-Chief DouglasO’Shaughnessy,UniversityofQuebec,Canada Associate Editors JontB.Allen,USA HoracioFranco,USA ClimentNadeu,Spain XavierAmatriain,USA Qian-JieFu,USA ElmarNoth,Germany Ge´rardBailly,France JimGlass,USA HiroshiOkuno,Japan MartinBouchard,Canada StevenGreenberg,USA JoePicone,USA DouglasS.Brungart,USA R.CapobiancoGuido,Brazil GerhardRigoll,Germany GeoffreyChan,Canada R.Heusdens,TheNetherlands MarkSandler,UK DanChazan,Israel JamesKates,USA ThippurV.Sreenivas,India MarkClements,USA TatsuyaKawahara,Japan YannisStylianou,Greece C.D’alessandro,France YvesLaprie,France StephenVoran,USA RogerDannenberg,USA Lin-ShanLee,Taiwan DeliangWang,USA LiDeng,USA DominicMassaro,USA ThomasEriksson,Sweden BenMilner,USA Contents AdaptivePartial-UpdateandSparseSystemIdentification,KutluyılDog˘ançayandPatrickA.Naylor Volume2007,ArticleID12046,2pages Set-MembershipProportionateAffineProjectionAlgorithms,StefanWerner,Jose´ A.Apolina´rio,Jr., andPauloS.R.Diniz Volume2007,ArticleID34242,10pages Wavelet-BasedMPNLMSAdaptiveAlgorithmforNetworkEchoCancellation,HongyangDengand Milos˘Doroslovac˘ki Volume2007,ArticleID96101,5pages ALowDelayandFastConvergingImprovedProportionateAlgorithmforSparseSystemIdentification, AndyW.H.Khong,PatrickA.Naylor,andJacobBenesty Volume2007,ArticleID84376,8pages AnalysisofTransientandSteady-StateBehaviorofaMultichannelFiltered-xPartial-ErrorAffine ProjectionAlgorithm,AlbertoCariniandGiovanniL.Sicuranza Volume2007,ArticleID31314,15pages StepSizeBoundoftheSequentialPartialUpdateLMSAlgorithmwithPeriodicInputSignals, PedroRamos,RobertoTorrubia,AnaLo´pez,AnaSalinas,andEnriqueMasgrau Volume2007,ArticleID10231,15pages Detection-GuidedFastAffineProjectionChannelEstimatorforSpeechApplications,YanWuJennifer, JohnHomer,GeertRombouts,andMarcMoonen Volume2007,ArticleID71495,13pages EfficientMultichannelNLMSImplementationforAcousticEchoCancellation,FredricLindstrom, ChristianSchu¨ldt,andIngvarClaesson Volume2007,ArticleID78439,6pages Time-DomainConvolutiveBlindSourceSeparationEmployingSelective-TapAdaptiveAlgorithms, QiongfengPanandTyseerAboulnasr Volume2007,ArticleID92528,11pages UnderdeterminedBlindAudioSourceSeparationUsingModalDecomposition,Abdeldjalil A¨ıssa-El-Bey,KarimAbed-Meraim,andYvesGrenier Volume2007,ArticleID85438,15pages HindawiPublishingCorporation EURASIPJournalonAudio,Speech,andMusicProcessing Volume2007,ArticleID12046,2pages doi:10.1155/2007/12046 Editorial Adaptive Partial-Update and Sparse System Identification KutluyılDog˘ançay1andPatrickA.Naylor2 1SchoolofElectricalandInformationEngineering,UniversityofSouthAustralia,MawsonLakes,SouthAustralia5095,Australia 2DepartmentofElectricalandElectronicEngineering,ImperialCollegeLondon,ExhibitionRoad,LondonSW72AZ,UK Received1March2007;Accepted1March2007 Copyright©2007K.Dog˘ançayandP.A.Naylor. This is an open access article distributed under the Creative Commons AttributionLicense,whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalworkis properlycited. Systemidentificationisanimportanttaskinmanyapplica- tooperateonalongfilterandthecoefficientnoisefornear- tionareasincluding,forexample,telecommunications,con- zero-valued coefficients in the inactive regions is relatively trol engineering, sensing, and acoustics. It would be widely large.Toaddressthisproblem,theconceptofproportionate acceptedthatthescienceforidentificationofstationaryand updatingwasintroduced. dynamic systems is mature. However, several new applica- An important consideration for adaptive filters is the tionshaverecentlybecomeofheightenedinterestforwhich computationalcomplexitythatincreaseswiththenumberof system identification needs to be performed on high-order coefficients to be updated per sampling period. A straight- moving average systems that are either sparse in the time forwardapproachtocomplexityreductionistoupdateonly domain or need to be estimated using sparse computation a small number of filter coefficients at every iteration. This duetocomplexityconstraints.Inthisspecialissue,wehave approachistermedpartial-updateadaptivefiltering.Twokey brought together a collection of articles on recent work in questionsariseinthecontextofpartialupdating.Firstly,con- this field giving specific consideration to (a) algorithms for sideration must be given as to how to choose which coeffi- identification of sparse systems and (b) algorithms that ex- cientstoupdate.Secondly,theperformanceandcomplexity ploit sparseness in the coefficient update domain. The dis- of the partial update approach must be compared with the tinctionbetweenthesetwotypesofsparsenessisimportant, standardfullupdatealgorithmsinordertoassessthecost-to- aswehopewillbecomecleartothereaderinthemainbody benefitratioforthepartialupdateschemes.Usually,acom- ofthespecialissue. promisehastobemadebetweenaffordablecomplexityand Adrivingforcebehindthedevelopmentofalgorithmsfor desiredconvergencespeed. sparsesystemidentificationintelecommunicationshasbeen We have grouped the papers in this special issue into echo cancellation in packet switched telephone networks. fourareas.Thefirstareaissparsesystemidentification and Theincreasingpopularityofpacket-switchedtelephonyhas comprises three papers. In “Set-membership proportion- ledtoaneedfortheintegrationofolderanalogsystemswith, ate affine projection algorithms,” Stefan Werner et al. de- forexample,IPorATMnetworks.Networkgatewaysenable velopaffineprojectionalgorithmswithproportionateupdate theinterconnectionofsuchnetworksandprovideechocan- and set membership filtering. Proportionate updates facil- cellation. In such systems, the hybrid echo response is de- itate fast convergence for sparse systems, and set member- layedbyanunknownbulkdelayduetopropagationthrough shipfilteringreducestheupdatecomplexity.Thesecondpa- thenetwork.Theoveralleffectis,therefore,thatan“active” per in this area is “Wavelet-based MPNLMS adaptive algo- regionassociatedwiththetruehybridechoresponseoccurs rithm for network echo cancellation” by H. Deng and M. withanunknowndelaywithinanoverallresponse window Doroslovacˇki, which develops a wavelet-domain µ-law pro- that has to be sufficiently long to accommodate the worst portionateNLMSalgorithmforidentificationandcancelling casebulkdelay.Inthecontextofnetworkechocancellation of sparse telephone network echoes. This work exploits the thedirectapplicationofwell-knownalgorithms,suchasnor- whitening and good time-frequency localisation properties malizedleast-mean-square(NLMS),tosparsesystemidenti- of the wavelet transform to speed up the convergence for ficationgivesunsatisfactoryperformancewhentheechore- coloured input signals and to retain sparseness of echo re- sponseissparse.Thisisbecausetheadaptivealgorithmhas sponseinthewavelettransformdomain.In“Alowdelayand 2 EURASIPJournalonAudio,Speech,andMusicProcessing fastconvergingimprovedproportionatealgorithmforsparse has created the need for new algorithms for sparse adap- system identification,” Andy W. H. Khong et al. propose a tivefiltering—achallengethathasbeenwellmettodatefor multidelay filter (MDF) implementation for improved pro- the particular applications addressed. When sparseness ex- portionateNLMSforsparsesystemidentification,inheriting ists, or can be safely assumed, in input signals, this can be the beneficial properties of both; namely, fast convergence exploited to achieve both computational savings in partial and computational efficiency coupled with low bulk delay. update schemes and, in certain specific cases, performance Astheauthorsshow,theMDFimplementationisnontrivial improvements.Thereremainseveralopenresearchquestions andrequirestime-domaincoefficientupdating. inthiscontextandwelookforwardtoanongoingresearch The second area of papers is partial-update active noise effortinthescientificcommunityandopportunitiesforal- control. In the first paper in this area “Analysis of tran- gorithmdeploymentinreal-timeapplications. sient and steady-state behavior of a multichannel filtered- x partial-error affine projection algorithm,” A. Carini and ACKNOWLEDGMENTS S. L. Sicuranza apply partial-error complexity reduction to filtered-x affine projection algorithm for multichannel ac- This special issue has arisen as a result of the high levels of tive noise control, and provide a comprehensive analysis of interestshownataspecialsessiononthistopicatEUSIPCO thetransientandsteady-statebehaviouroftheadaptivealgo- 2005inAntalya,Turkey.Ithasbeenagreatprivilegetoactas rithmdrawingonenergyconservation.In“Stepsizebound guesteditorsforthisspecialissueandweextendourgrateful of the sequential partial update LMS algorithm with peri- thankstoalltheauthorsandthepublisher. odic input signals” Pedro Ramos et al. show that for pe- riodic input signals the sequential partial update LMS and KutluyılDog˘an¸cay filtered-xLMSalgorithmscanachievethesameconvergence PatrickA.Naylor performanceastheirfull-updatecounterpartsbyincreasing the step-size appropriately. This essentially avoids any con- vergencepenaltyassociatedwithsequentialupdating. The third area focuses on general partial update algorithms. In the first paper in this area, “Detection guided fast affine projection channel estimator for speech applications,” Yan Wu Jennifer et al. consider detection guided identification of active taps in a long acoustic echo channel in order to shorten the actual channel and integrate it intothefastaffineprojectionalgorithmtoattainfastercon- vergence. The proposed algorithm is well suited for highly correlatedinputsignalssuchasspeechsignals.In“Efficient multichannelNLMSimplementationforacousticechocan- cellation,” Fredric Lindstrom et al. propose a multichannel acoustic echo cancellation algorithm based on normalized least-mean-squarewithpartialupdatesfavouringfilterswith largestmisadjustment. The final area is devoted to blind source separation. In “Timedomainconvolutiveblindsourceseparationemploy- ingselective-tapadaptivealgorithms,”Q.PanandT.Aboul- nasrproposetime-domainconvolutiveblindsourcesepara- tionalgorithmsemployingM-maxandexclusivemaximum selective-tap techniques. The resulting algorithms have reduced complexity and improved convergence performance thanks to partial updating and reduced interchannel co- herence. In the final paper “Underdetermined blind audio source separation using modal decomposition,” Abdeljalil A¨ıssa-El-Bey et al. present a novel blind source separation algorithm for audio signals using modal decomposition. In additiontoinstantaneousmixing,theauthorsconsidercon- volutive mixing and exploit the sparseness of audio signals toidentifythechannelresponsesbeforeapplyingmodalde- composition. In summary, we can say that sparseness in the context of adaptive filtering presents both challenges and opportu- nities.Standardadaptivealgorithmssufferadegradationin performancewhenthesystemtobeidentifiedissparse.This HindawiPublishingCorporation EURASIPJournalonAudio,Speech,andMusicProcessing Volume2007,ArticleID34242,10pages doi:10.1155/2007/34242 Research Article Set-Membership Proportionate Affine Projection Algorithms StefanWerner,1Jose´ A.Apolina´rio,Jr.,2andPauloS.R.Diniz3 1SignalProcessingLaboratory,HelsinkiUniversityofTechnology,Otakaari5A,02150Espoo,Finland 2DepartmentofElectricalEngineering,InstitutoMilitardeEngenharia,2229-270RiodeJaneiro,Brazil 3SignalProcessingLaboratory,COPPE/Poli/UniversidadeFederaldoRiodeJaneiro,21945-970RiodeJaneiro,Brazil Received30June2006;Revised15November2006;Accepted15November2006 RecommendedbyKutluyilDogancay Proportionateadaptivefilterscanimprovetheconvergencespeedfortheidentificationofsparsesystemsascomparedtotheir conventionalcounterparts.Inthispaper,theideaofproportionateadaptationiscombinedwiththeframeworkofset-membership filtering(SMF)inanattempttoderivenovelcomputationallyefficientalgorithms.Theresultingalgorithmsattainanattractive fasterconvergeforbothsituationsofsparseanddispersivechannelswhiledecreasingtheaveragecomputationalcomplexitydueto thedatadiscerningfeatureoftheSMFapproach.Inaddition,weproposearulethatallowsustoautomaticallyadjustthenumber ofpastdatapairsemployedintheupdate.Thisleadstoaset-membershipproportionateaffineprojectionalgorithm(SM-PAPA) havingavariabledata-reusefactorallowingasignificantreductionintheoverallcomplexitywhencomparedwithafixeddata- reusefactor.Reduced-complexityimplementationsoftheproposedalgorithmsarealsoconsideredthatreducethedimensionsof thematrixinversionsinvolvedintheupdate.Simulationsshowgoodresultsintermsofreducednumberofupdates,speedof convergence,andfinalmean-squarederror. Copyright©2007StefanWerneretal.ThisisanopenaccessarticledistributedundertheCreativeCommonsAttributionLicense, whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalworkisproperlycited. 1. INTRODUCTION PNLMS algorithm after the initial transient can be circum- ventedbyswitchingtotheNLMSalgorithm[11]. Frequently used adaptive filtering algorithms like the least Another problem related to the conventional PNLMS mean square (LMS) and the normalized LMS (NLMS) al- algorithm is the poor performance in dispersive or semi- gorithmssharethefeaturesoflowcomputationalcomplex- dispersive channels [3]. Refinements of the PNLMS have ity and proven robustness. The LMS and the NLMS algo- beenproposed[3,4]toimproveperformanceinadispersive rithms have in common that the adaptive filter is updated medium and to combat the slowdown after the initial in the direction of the input vector without favoring any adaptation.ThePNLMS++algorithmin[3]approachesthe particulardirection.Inotherwords,theyarewellsuitedfor problem by alternating the NLMS update with a PNLMS dispersive-type systems where the energy is uniformly dis- update. The improved PNLMS (IPNLMS) algorithm [4] tributedamongthecoefficientsintheimpulseresponse.On combines the NLMS and PNLMS algorithms into one the other hand, if the system to be identified is sparse, that single updating expression. The main idea of the IPNLMS is,theimpulseresponseischaracterizedbyafewdominant algorithmwastoestablisharuleforautomaticallyswitching coefficients(see[1]foradefinitionofameasureofsparsity), from one algorithm to the other. It was further shown in using different step sizes for each adaptive filter coefficient [6]thattheIPNLMSalgorithmisagoodapproximationof canimprovetheinitialconvergenceoftheNLMSalgorithm. the exponentiated gradient algorithm [1, 12]. Extension of Thisbasicconceptisexploredinproportionateadaptivefilters the proportionate adaptation concept to affine projection [2–10],whichincorporatestheimportanceoftheindividual (AP)typealgorithms,proportionateaffineprojection(PAP) components by assigning weights proportional to the mag- algorithms,canbefoundin[13,14]. nitudeofthecoefficients. Using the PNLMS algorithm instead of the NLMS al- The conventional proportionate NLMS (PNLMS) algo- gorithm leads to 50% increase in the computational com- rithm[2]experiencesfastinitialadaptationforthedominant plexity. An efficient approach to reduce computations is to coefficientsfollowedbyaslowersecondtransientforthere- employset-membershipfiltering(SMF)techniques[15,16], mainingcoefficients.Therefore,theslowconvergenceofthe where the filter is designedsuchthattheoutputestimation 2 EURASIPJournalonAudio,Speech,andMusicProcessing errorisupperboundedbyapredeterminedthreshold.1 Set- smaller than a deterministic threshold γ, where x(k) ∈ CN membership adaptive filters (SMAF) feature data-selective andd(k) ∈ Cdenotetheinputvectorandthedesiredout- (sparse in time) updating, and a time-varying data- putsignal,respectively.Asaresultoftheboundederrorcon- dependent step size that provides fast convergence as well straint,therewillexistasetoffiltersratherthanasingleesti- as low steady-state error. SMAFs with low computational mate. complexityperupdatearetheset-membershipNLMS(SM- Let S denote the set of all possible input-desired data NLMS)[15],theset-membershipbinormalizeddata-reusing pairs (x,d) of interest. Let Θ denote the set of all possible (SM-BNDRLMS) [16], and the set-membership affine pro- vectorswthatresultinanoutputerrorboundedbyγwhen- jection(SM-AP)[17]algorithms.Inthefollowing,wecom- ever(x,d) ∈ S.ThesetΘreferredtoasthefeasibilityset is binetheframeworksofproportionateadaptationandSMF. givenby Aset-membershipproportionateNLMS(SM-PNLMS)algo- (cid:2) (cid:3) (cid:4) (cid:4) (cid:5) rithmisproposedasaviablealternativetotheSM-NLMSal- Θ= w∈CN :(cid:4)d−wHx(cid:4)≤γ . (1) gorithm[15]foroperationinsparsescenarios.Followingthe (x,d)∈S ideas of the IPNLMS algorithm, an efficient weight-scaling Adaptive SMF algorithms seek solutions that belong to the assignment is proposed that utilizes the information pro- exact membership set ψ(k) constructed by input-signal and videdbythedata-dependentstepsize.Thereafter,wepropose desired-signalpairs, amoregeneralalgorithm,theset-membershipproportionate affine projection algorithm (SM-PAPA) that generalizes the (cid:2)k ψ(k)= H(i), (2) ideasoftheSM-PNLMStoreuseconstraintsetsfromafixed i=1 numberofpastinputanddesiredsignalpairsinthesameway as the SM-AP algorithm [17]. The resulting algorithm can whereH(k)isreferredtoastheconstraintset containingall be seen as a set-membership version of the PAP algorithm vectors w for which the associated output error at time in- [13,14]withanoptimized stepsize. AswiththePAPalgo- stantkisupperboundedinmagnitudebyγ: rithm,afasterconvergenceoftheSM-PAPAalgorithmmay H(k)=(cid:3)w∈CN :(cid:4)(cid:4)d(k)−wHx(k)(cid:4)(cid:4)≤γ(cid:5). (3) comeattheexpenseofaslightincreaseinthecomputational complexityperupdatethatisdirectlylinkedtotheamount ItcanbeseenthatthefeasibilitysetΘisasubsetoftheexact of reuses employed, or data-reuse factor. To lower the over- membershipsetψ atanygiventimeinstant.Thefeasibilityset k allcomplexity,weproposetouseatime-varyingdata-reuse isalsothelimitingsetoftheexactmembershipset,thatis,the factor.Theintroductionofthevariabledata-reusefactorre- twosetswillbeequalifthetrainingsignaltraversesallsignal sultsinanalgorithmthatclosetoconvergencetakestheform pairs belonging to S. The idea of set-membership adaptive ofthesimpleSM-PNLMSalgorithm.Thereafter,weconsider filters(SMAF)istofindadaptivelyanestimatethatbelongs anefficientimplementationofthenewSM-PAPAalgorithm tothefeasibilitysetortooneofitsmembers.Sinceψ(k)in thatreducesthedimensionsofthematricesinvolvedinthe (2)isnoteasilycomputed,oneapproachistoapplyoneof update. themanyoptimalboundingellipsoid(OBE)algorithms[18, Thepaperis organized asfollows.Section2 reviewsthe 20–24],whichtriestoapproximatetheexactmembershipset conceptofSMFwhiletheSM-PNLMSalgorithmisproposed ψ(k) by tightly outer bounding it with ellipsoids. Adaptive in Section3. Section4 derives the general SM-PAPA algo- approachesleadingtoalgorithmswithlowpeakcomplexity, rithmwherebothcasesoffixedandtime-varyingdata-reuse O(N), compute a point estimate through projections using factor are studied. Section5 provides the details of an SM- informationprovidedbypastconstraintsets[15–17,25–27]. PAPAimplementationusingreducedmatrixdimensions.In In this paper, we are interested in algorithms derived from Section6,theperformancesoftheproposed algorithmsare thelatterapproach. evaluated through simulations which are followed by con- clusions. 3. THESET-MEMBERSHIPPROPORTIONATE NLMSALGORITHM 2. SET-MEMBERSHIPFILTERING In this section, the idea of proportionate adaptation is ap- This section reviews the basic concepts of set-membership plied to SMF in order to derive a data-selective algorithm, filtering(SMF).Foramoredetailedintroductiontothecon- the set-membership proportionate normalized LMS (SM- ceptofSMF,thereaderisreferredto[18].Set-membership PNLMS),suitableforsparseenvironments. filteringisaframeworkapplicabletofilteringproblemsthat arelinearinparameters.2Aspecificationonthefilterparam- 3.1. Algorithmderivation etersw ∈ CN isachievedbyconstrainingthemagnitudeof the output estimation error, e(k) = d(k)−wHx(k), to be TheSM-PNLMSalgorithmusestheinformationprovidedby theconstraintsetH(k)andthecoefficientupdatingtosolve theoptimizationproblememployingthecriterion 1Forotherreduced-complexitysolutions,see,forexample,[11]wherethe (cid:6) (cid:6) conceptofpartialupdatingisapplied. w(k+1)=argmin(cid:6)w−w(k)(cid:6)2 subjectto:w∈H(k), 2ThisincludesnonlinearproblemslikeVolterramodeling,see,forexam- w G−1(k) (4) ple,[19].