Table Of ContentEURASIP Journal on Audio, Speech, and Music Processing
Adaptive Partial-Update and
Sparse System Identification
Guest Editors: Kutluyıl Dog˘ançay and Patrick A. Naylor
Adaptive Partial-Update and Sparse
System Identification
EURASIP Journal on
Audio, Speech, and Music Processing
Adaptive Partial-Update and Sparse
System Identification
Guest Editors: Kutluyıl Dog˘anc¸ay and Patrick A. Naylor
Copyright©2007HindawiPublishingCorporation.Allrightsreserved.
Thisisaspecialissuepublishedinvolume2007of“EURASIPJournalonAudio,Speech,andMusicProcessing.”Allarticlesareopen
accessarticlesdistributedundertheCreativeCommonsAttributionLicense,whichpermitsunrestricteduse,distribution,andrepro-
ductioninanymedium,providedtheoriginalworkisproperlycited.
Editor-in-Chief
DouglasO’Shaughnessy,UniversityofQuebec,Canada
Associate Editors
JontB.Allen,USA HoracioFranco,USA ClimentNadeu,Spain
XavierAmatriain,USA Qian-JieFu,USA ElmarNoth,Germany
Ge´rardBailly,France JimGlass,USA HiroshiOkuno,Japan
MartinBouchard,Canada StevenGreenberg,USA JoePicone,USA
DouglasS.Brungart,USA R.CapobiancoGuido,Brazil GerhardRigoll,Germany
GeoffreyChan,Canada R.Heusdens,TheNetherlands MarkSandler,UK
DanChazan,Israel JamesKates,USA ThippurV.Sreenivas,India
MarkClements,USA TatsuyaKawahara,Japan YannisStylianou,Greece
C.D’alessandro,France YvesLaprie,France StephenVoran,USA
RogerDannenberg,USA Lin-ShanLee,Taiwan DeliangWang,USA
LiDeng,USA DominicMassaro,USA
ThomasEriksson,Sweden BenMilner,USA
Contents
AdaptivePartial-UpdateandSparseSystemIdentification,KutluyılDog˘anc¸ayandPatrickA.Naylor
Volume2007,ArticleID12046,2pages
Set-MembershipProportionateAffineProjectionAlgorithms,StefanWerner,Jose´ A.Apolina´rio,Jr.,
andPauloS.R.Diniz
Volume2007,ArticleID34242,10pages
Wavelet-BasedMPNLMSAdaptiveAlgorithmforNetworkEchoCancellation,HongyangDengand
Milos˘Doroslovac˘ki
Volume2007,ArticleID96101,5pages
ALowDelayandFastConvergingImprovedProportionateAlgorithmforSparseSystemIdentification,
AndyW.H.Khong,PatrickA.Naylor,andJacobBenesty
Volume2007,ArticleID84376,8pages
AnalysisofTransientandSteady-StateBehaviorofaMultichannelFiltered-xPartial-ErrorAffine
ProjectionAlgorithm,AlbertoCariniandGiovanniL.Sicuranza
Volume2007,ArticleID31314,15pages
StepSizeBoundoftheSequentialPartialUpdateLMSAlgorithmwithPeriodicInputSignals,
PedroRamos,RobertoTorrubia,AnaLo´pez,AnaSalinas,andEnriqueMasgrau
Volume2007,ArticleID10231,15pages
Detection-GuidedFastAffineProjectionChannelEstimatorforSpeechApplications,YanWuJennifer,
JohnHomer,GeertRombouts,andMarcMoonen
Volume2007,ArticleID71495,13pages
EfficientMultichannelNLMSImplementationforAcousticEchoCancellation,FredricLindstrom,
ChristianSchu¨ldt,andIngvarClaesson
Volume2007,ArticleID78439,6pages
Time-DomainConvolutiveBlindSourceSeparationEmployingSelective-TapAdaptiveAlgorithms,
QiongfengPanandTyseerAboulnasr
Volume2007,ArticleID92528,11pages
UnderdeterminedBlindAudioSourceSeparationUsingModalDecomposition,Abdeldjalil
A¨ıssa-El-Bey,KarimAbed-Meraim,andYvesGrenier
Volume2007,ArticleID85438,15pages
HindawiPublishingCorporation
EURASIPJournalonAudio,Speech,andMusicProcessing
Volume2007,ArticleID12046,2pages
doi:10.1155/2007/12046
Editorial
Adaptive Partial-Update and Sparse System Identification
KutluyılDog˘anc¸ay1andPatrickA.Naylor2
1SchoolofElectricalandInformationEngineering,UniversityofSouthAustralia,MawsonLakes,SouthAustralia5095,Australia
2DepartmentofElectricalandElectronicEngineering,ImperialCollegeLondon,ExhibitionRoad,LondonSW72AZ,UK
Received1March2007;Accepted1March2007
Copyright©2007K.Dog˘anc¸ayandP.A.Naylor. This is an open access article distributed under the Creative Commons
AttributionLicense,whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalworkis
properlycited.
Systemidentificationisanimportanttaskinmanyapplica- tooperateonalongfilterandthecoefficientnoisefornear-
tionareasincluding,forexample,telecommunications,con- zero-valued coefficients in the inactive regions is relatively
trol engineering, sensing, and acoustics. It would be widely large.Toaddressthisproblem,theconceptofproportionate
acceptedthatthescienceforidentificationofstationaryand updatingwasintroduced.
dynamic systems is mature. However, several new applica- An important consideration for adaptive filters is the
tionshaverecentlybecomeofheightenedinterestforwhich computationalcomplexitythatincreaseswiththenumberof
system identification needs to be performed on high-order coefficients to be updated per sampling period. A straight-
moving average systems that are either sparse in the time forwardapproachtocomplexityreductionistoupdateonly
domain or need to be estimated using sparse computation a small number of filter coefficients at every iteration. This
duetocomplexityconstraints.Inthisspecialissue,wehave approachistermedpartial-updateadaptivefiltering.Twokey
brought together a collection of articles on recent work in questionsariseinthecontextofpartialupdating.Firstly,con-
this field giving specific consideration to (a) algorithms for sideration must be given as to how to choose which coeffi-
identification of sparse systems and (b) algorithms that ex- cientstoupdate.Secondly,theperformanceandcomplexity
ploit sparseness in the coefficient update domain. The dis- of the partial update approach must be compared with the
tinctionbetweenthesetwotypesofsparsenessisimportant, standardfullupdatealgorithmsinordertoassessthecost-to-
aswehopewillbecomecleartothereaderinthemainbody benefitratioforthepartialupdateschemes.Usually,acom-
ofthespecialissue. promisehastobemadebetweenaffordablecomplexityand
Adrivingforcebehindthedevelopmentofalgorithmsfor desiredconvergencespeed.
sparsesystemidentificationintelecommunicationshasbeen We have grouped the papers in this special issue into
echo cancellation in packet switched telephone networks. fourareas.Thefirstareaissparsesystemidentification and
Theincreasingpopularityofpacket-switchedtelephonyhas comprises three papers. In “Set-membership proportion-
ledtoaneedfortheintegrationofolderanalogsystemswith, ate affine projection algorithms,” Stefan Werner et al. de-
forexample,IPorATMnetworks.Networkgatewaysenable velopaffineprojectionalgorithmswithproportionateupdate
theinterconnectionofsuchnetworksandprovideechocan- and set membership filtering. Proportionate updates facil-
cellation. In such systems, the hybrid echo response is de- itate fast convergence for sparse systems, and set member-
layedbyanunknownbulkdelayduetopropagationthrough shipfilteringreducestheupdatecomplexity.Thesecondpa-
thenetwork.Theoveralleffectis,therefore,thatan“active” per in this area is “Wavelet-based MPNLMS adaptive algo-
regionassociatedwiththetruehybridechoresponseoccurs rithm for network echo cancellation” by H. Deng and M.
withanunknowndelaywithinanoverallresponse window Doroslovacˇki, which develops a wavelet-domain µ-law pro-
that has to be sufficiently long to accommodate the worst portionateNLMSalgorithmforidentificationandcancelling
casebulkdelay.Inthecontextofnetworkechocancellation of sparse telephone network echoes. This work exploits the
thedirectapplicationofwell-knownalgorithms,suchasnor- whitening and good time-frequency localisation properties
malizedleast-mean-square(NLMS),tosparsesystemidenti- of the wavelet transform to speed up the convergence for
ficationgivesunsatisfactoryperformancewhentheechore- coloured input signals and to retain sparseness of echo re-
sponseissparse.Thisisbecausetheadaptivealgorithmhas sponseinthewavelettransformdomain.In“Alowdelayand
2 EURASIPJournalonAudio,Speech,andMusicProcessing
fastconvergingimprovedproportionatealgorithmforsparse has created the need for new algorithms for sparse adap-
system identification,” Andy W. H. Khong et al. propose a tivefiltering—achallengethathasbeenwellmettodatefor
multidelay filter (MDF) implementation for improved pro- the particular applications addressed. When sparseness ex-
portionateNLMSforsparsesystemidentification,inheriting ists, or can be safely assumed, in input signals, this can be
the beneficial properties of both; namely, fast convergence exploited to achieve both computational savings in partial
and computational efficiency coupled with low bulk delay. update schemes and, in certain specific cases, performance
Astheauthorsshow,theMDFimplementationisnontrivial improvements.Thereremainseveralopenresearchquestions
andrequirestime-domaincoefficientupdating. inthiscontextandwelookforwardtoanongoingresearch
The second area of papers is partial-update active noise effortinthescientificcommunityandopportunitiesforal-
control. In the first paper in this area “Analysis of tran- gorithmdeploymentinreal-timeapplications.
sient and steady-state behavior of a multichannel filtered-
x partial-error affine projection algorithm,” A. Carini and ACKNOWLEDGMENTS
S. L. Sicuranza apply partial-error complexity reduction to
filtered-x affine projection algorithm for multichannel ac- This special issue has arisen as a result of the high levels of
tive noise control, and provide a comprehensive analysis of interestshownataspecialsessiononthistopicatEUSIPCO
thetransientandsteady-statebehaviouroftheadaptivealgo- 2005inAntalya,Turkey.Ithasbeenagreatprivilegetoactas
rithmdrawingonenergyconservation.In“Stepsizebound guesteditorsforthisspecialissueandweextendourgrateful
of the sequential partial update LMS algorithm with peri- thankstoalltheauthorsandthepublisher.
odic input signals” Pedro Ramos et al. show that for pe-
riodic input signals the sequential partial update LMS and KutluyılDog˘an¸cay
filtered-xLMSalgorithmscanachievethesameconvergence PatrickA.Naylor
performanceastheirfull-updatecounterpartsbyincreasing
the step-size appropriately. This essentially avoids any con-
vergencepenaltyassociatedwithsequentialupdating.
The third area focuses on general partial update algo-
rithms. In the first paper in this area, “Detection guided
fast affine projection channel estimator for speech appli-
cations,” Yan Wu Jennifer et al. consider detection guided
identification of active taps in a long acoustic echo chan-
nel in order to shorten the actual channel and integrate it
intothefastaffineprojectionalgorithmtoattainfastercon-
vergence. The proposed algorithm is well suited for highly
correlatedinputsignalssuchasspeechsignals.In“Efficient
multichannelNLMSimplementationforacousticechocan-
cellation,” Fredric Lindstrom et al. propose a multichannel
acoustic echo cancellation algorithm based on normalized
least-mean-squarewithpartialupdatesfavouringfilterswith
largestmisadjustment.
The final area is devoted to blind source separation. In
“Timedomainconvolutiveblindsourceseparationemploy-
ingselective-tapadaptivealgorithms,”Q.PanandT.Aboul-
nasrproposetime-domainconvolutiveblindsourcesepara-
tionalgorithmsemployingM-maxandexclusivemaximum
selective-tap techniques. The resulting algorithms have re-
duced complexity and improved convergence performance
thanks to partial updating and reduced interchannel co-
herence. In the final paper “Underdetermined blind audio
source separation using modal decomposition,” Abdeljalil
A¨ıssa-El-Bey et al. present a novel blind source separation
algorithm for audio signals using modal decomposition. In
additiontoinstantaneousmixing,theauthorsconsidercon-
volutive mixing and exploit the sparseness of audio signals
toidentifythechannelresponsesbeforeapplyingmodalde-
composition.
In summary, we can say that sparseness in the context
of adaptive filtering presents both challenges and opportu-
nities.Standardadaptivealgorithmssufferadegradationin
performancewhenthesystemtobeidentifiedissparse.This
HindawiPublishingCorporation
EURASIPJournalonAudio,Speech,andMusicProcessing
Volume2007,ArticleID34242,10pages
doi:10.1155/2007/34242
Research Article
Set-Membership Proportionate Affine Projection Algorithms
StefanWerner,1Jose´ A.Apolina´rio,Jr.,2andPauloS.R.Diniz3
1SignalProcessingLaboratory,HelsinkiUniversityofTechnology,Otakaari5A,02150Espoo,Finland
2DepartmentofElectricalEngineering,InstitutoMilitardeEngenharia,2229-270RiodeJaneiro,Brazil
3SignalProcessingLaboratory,COPPE/Poli/UniversidadeFederaldoRiodeJaneiro,21945-970RiodeJaneiro,Brazil
Received30June2006;Revised15November2006;Accepted15November2006
RecommendedbyKutluyilDogancay
Proportionateadaptivefilterscanimprovetheconvergencespeedfortheidentificationofsparsesystemsascomparedtotheir
conventionalcounterparts.Inthispaper,theideaofproportionateadaptationiscombinedwiththeframeworkofset-membership
filtering(SMF)inanattempttoderivenovelcomputationallyefficientalgorithms.Theresultingalgorithmsattainanattractive
fasterconvergeforbothsituationsofsparseanddispersivechannelswhiledecreasingtheaveragecomputationalcomplexitydueto
thedatadiscerningfeatureoftheSMFapproach.Inaddition,weproposearulethatallowsustoautomaticallyadjustthenumber
ofpastdatapairsemployedintheupdate.Thisleadstoaset-membershipproportionateaffineprojectionalgorithm(SM-PAPA)
havingavariabledata-reusefactorallowingasignificantreductionintheoverallcomplexitywhencomparedwithafixeddata-
reusefactor.Reduced-complexityimplementationsoftheproposedalgorithmsarealsoconsideredthatreducethedimensionsof
thematrixinversionsinvolvedintheupdate.Simulationsshowgoodresultsintermsofreducednumberofupdates,speedof
convergence,andfinalmean-squarederror.
Copyright©2007StefanWerneretal.ThisisanopenaccessarticledistributedundertheCreativeCommonsAttributionLicense,
whichpermitsunrestricteduse,distribution,andreproductioninanymedium,providedtheoriginalworkisproperlycited.
1. INTRODUCTION PNLMS algorithm after the initial transient can be circum-
ventedbyswitchingtotheNLMSalgorithm[11].
Frequently used adaptive filtering algorithms like the least Another problem related to the conventional PNLMS
mean square (LMS) and the normalized LMS (NLMS) al- algorithm is the poor performance in dispersive or semi-
gorithmssharethefeaturesoflowcomputationalcomplex- dispersive channels [3]. Refinements of the PNLMS have
ity and proven robustness. The LMS and the NLMS algo- beenproposed[3,4]toimproveperformanceinadispersive
rithms have in common that the adaptive filter is updated medium and to combat the slowdown after the initial
in the direction of the input vector without favoring any adaptation.ThePNLMS++algorithmin[3]approachesthe
particulardirection.Inotherwords,theyarewellsuitedfor problem by alternating the NLMS update with a PNLMS
dispersive-type systems where the energy is uniformly dis- update. The improved PNLMS (IPNLMS) algorithm [4]
tributedamongthecoefficientsintheimpulseresponse.On combines the NLMS and PNLMS algorithms into one
the other hand, if the system to be identified is sparse, that single updating expression. The main idea of the IPNLMS
is,theimpulseresponseischaracterizedbyafewdominant algorithmwastoestablisharuleforautomaticallyswitching
coefficients(see[1]foradefinitionofameasureofsparsity), from one algorithm to the other. It was further shown in
using different step sizes for each adaptive filter coefficient [6]thattheIPNLMSalgorithmisagoodapproximationof
canimprovetheinitialconvergenceoftheNLMSalgorithm. the exponentiated gradient algorithm [1, 12]. Extension of
Thisbasicconceptisexploredinproportionateadaptivefilters the proportionate adaptation concept to affine projection
[2–10],whichincorporatestheimportanceoftheindividual (AP)typealgorithms,proportionateaffineprojection(PAP)
components by assigning weights proportional to the mag- algorithms,canbefoundin[13,14].
nitudeofthecoefficients. Using the PNLMS algorithm instead of the NLMS al-
The conventional proportionate NLMS (PNLMS) algo- gorithm leads to 50% increase in the computational com-
rithm[2]experiencesfastinitialadaptationforthedominant plexity. An efficient approach to reduce computations is to
coefficientsfollowedbyaslowersecondtransientforthere- employset-membershipfiltering(SMF)techniques[15,16],
mainingcoefficients.Therefore,theslowconvergenceofthe where the filter is designedsuchthattheoutputestimation
2 EURASIPJournalonAudio,Speech,andMusicProcessing
errorisupperboundedbyapredeterminedthreshold.1 Set- smaller than a deterministic threshold γ, where x(k) ∈ CN
membership adaptive filters (SMAF) feature data-selective andd(k) ∈ Cdenotetheinputvectorandthedesiredout-
(sparse in time) updating, and a time-varying data- putsignal,respectively.Asaresultoftheboundederrorcon-
dependent step size that provides fast convergence as well straint,therewillexistasetoffiltersratherthanasingleesti-
as low steady-state error. SMAFs with low computational mate.
complexityperupdatearetheset-membershipNLMS(SM- Let S denote the set of all possible input-desired data
NLMS)[15],theset-membershipbinormalizeddata-reusing pairs (x,d) of interest. Let Θ denote the set of all possible
(SM-BNDRLMS) [16], and the set-membership affine pro- vectorswthatresultinanoutputerrorboundedbyγwhen-
jection(SM-AP)[17]algorithms.Inthefollowing,wecom- ever(x,d) ∈ S.ThesetΘreferredtoasthefeasibilityset is
binetheframeworksofproportionateadaptationandSMF. givenby
Aset-membershipproportionateNLMS(SM-PNLMS)algo- (cid:2) (cid:3) (cid:4) (cid:4) (cid:5)
rithmisproposedasaviablealternativetotheSM-NLMSal- Θ= w∈CN :(cid:4)d−wHx(cid:4)≤γ . (1)
gorithm[15]foroperationinsparsescenarios.Followingthe (x,d)∈S
ideas of the IPNLMS algorithm, an efficient weight-scaling Adaptive SMF algorithms seek solutions that belong to the
assignment is proposed that utilizes the information pro- exact membership set ψ(k) constructed by input-signal and
videdbythedata-dependentstepsize.Thereafter,wepropose desired-signalpairs,
amoregeneralalgorithm,theset-membershipproportionate
affine projection algorithm (SM-PAPA) that generalizes the (cid:2)k
ψ(k)= H(i), (2)
ideasoftheSM-PNLMStoreuseconstraintsetsfromafixed
i=1
numberofpastinputanddesiredsignalpairsinthesameway
as the SM-AP algorithm [17]. The resulting algorithm can whereH(k)isreferredtoastheconstraintset containingall
be seen as a set-membership version of the PAP algorithm vectors w for which the associated output error at time in-
[13,14]withanoptimized stepsize. AswiththePAPalgo- stantkisupperboundedinmagnitudebyγ:
rithm,afasterconvergenceoftheSM-PAPAalgorithmmay H(k)=(cid:3)w∈CN :(cid:4)(cid:4)d(k)−wHx(k)(cid:4)(cid:4)≤γ(cid:5). (3)
comeattheexpenseofaslightincreaseinthecomputational
complexityperupdatethatisdirectlylinkedtotheamount ItcanbeseenthatthefeasibilitysetΘisasubsetoftheexact
of reuses employed, or data-reuse factor. To lower the over- membershipsetψ atanygiventimeinstant.Thefeasibilityset
k
allcomplexity,weproposetouseatime-varyingdata-reuse isalsothelimitingsetoftheexactmembershipset,thatis,the
factor.Theintroductionofthevariabledata-reusefactorre- twosetswillbeequalifthetrainingsignaltraversesallsignal
sultsinanalgorithmthatclosetoconvergencetakestheform pairs belonging to S. The idea of set-membership adaptive
ofthesimpleSM-PNLMSalgorithm.Thereafter,weconsider filters(SMAF)istofindadaptivelyanestimatethatbelongs
anefficientimplementationofthenewSM-PAPAalgorithm tothefeasibilitysetortooneofitsmembers.Sinceψ(k)in
thatreducesthedimensionsofthematricesinvolvedinthe (2)isnoteasilycomputed,oneapproachistoapplyoneof
update. themanyoptimalboundingellipsoid(OBE)algorithms[18,
Thepaperis organized asfollows.Section2 reviewsthe 20–24],whichtriestoapproximatetheexactmembershipset
conceptofSMFwhiletheSM-PNLMSalgorithmisproposed ψ(k) by tightly outer bounding it with ellipsoids. Adaptive
in Section3. Section4 derives the general SM-PAPA algo- approachesleadingtoalgorithmswithlowpeakcomplexity,
rithmwherebothcasesoffixedandtime-varyingdata-reuse O(N), compute a point estimate through projections using
factor are studied. Section5 provides the details of an SM- informationprovidedbypastconstraintsets[15–17,25–27].
PAPAimplementationusingreducedmatrixdimensions.In In this paper, we are interested in algorithms derived from
Section6,theperformancesoftheproposed algorithmsare thelatterapproach.
evaluated through simulations which are followed by con-
clusions.
3. THESET-MEMBERSHIPPROPORTIONATE
NLMSALGORITHM
2. SET-MEMBERSHIPFILTERING
In this section, the idea of proportionate adaptation is ap-
This section reviews the basic concepts of set-membership plied to SMF in order to derive a data-selective algorithm,
filtering(SMF).Foramoredetailedintroductiontothecon- the set-membership proportionate normalized LMS (SM-
ceptofSMF,thereaderisreferredto[18].Set-membership PNLMS),suitableforsparseenvironments.
filteringisaframeworkapplicabletofilteringproblemsthat
arelinearinparameters.2Aspecificationonthefilterparam-
3.1. Algorithmderivation
etersw ∈ CN isachievedbyconstrainingthemagnitudeof
the output estimation error, e(k) = d(k)−wHx(k), to be TheSM-PNLMSalgorithmusestheinformationprovidedby
theconstraintsetH(k)andthecoefficientupdatingtosolve
theoptimizationproblememployingthecriterion
1Forotherreduced-complexitysolutions,see,forexample,[11]wherethe (cid:6) (cid:6)
conceptofpartialupdatingisapplied. w(k+1)=argmin(cid:6)w−w(k)(cid:6)2 subjectto:w∈H(k),
2ThisincludesnonlinearproblemslikeVolterramodeling,see,forexam- w G−1(k)
(4)
ple,[19].
