Optimal Design of the Adaptive Normalized Matched Filter Detector

0.27 MB·English

by  Abla Kammoun

1
Optimal Design of the Adaptive Normalized
Matched Filter Detector
Abla Kammoun, Romain Couillet, Fre´de´ric Pascal, Mohamed-Slim Alouini
Abstract—This article addresses improvements on the design latterscenarioisoftenmodeledbyassumingthatobservations
of the adaptive normalized matched filter (ANMF) for radar are drawn from complex elliptical symmetric distributions
detection. It is well-acknowledged that the estimation of the
(CES), originally introduced by Kelker [7]. The inherent
5 noise-cluttercovariancematrixisafundamentalstepinadaptive
1 radardetection.Inthispaper,weconsiderregularizedestimation nature of these distributions to produce atypical observations
0 methods which force by construction the eigenvalues of the makes the task of estimating the covariance matrix much
2 scatter estimates to be greater than a positive regularization more challenging. To tackle this issue, a class of covariance
y parameterρ.Thismakesthemmoresuitableforhighdimensional estimators coined robust estimators of scatter matrices have
problemswithalimitednumberofsecondarydatasamplesthan
a been proposed by Huber, Hampel, and Maronna [8]–[10],
M traditional sample covariance estimates. While an increase of
and extended more recently by Ollila to the complex case
ρ seems to improve the conditioning of the estimate, it might
howevercauseittosignificantlydeviatefromthetruecovariance [3], [4], [11]. Similar to the Gaussian case, the regularization
4
matrix. The setting of the optimal regularization parameter is a technique has been applied to the robust Tyler estimator [12],
1
difficultquestionfor whichno convincinganswers have thusfar yieldingtheso-calledregularizedTylerestimator(RTE).While

beenprovided.Thisconstitutesthemajormotivationbehindour
] conventionalrobust covariance methods are undefined for too
T work. More specifically, we consider the design of the ANMF
few samples (number of samples less than their dimensions),
detector for two kinds of regularized estimators, namely the
I
. regularizedsamplecovariancematrix(RSCM),appropriatewhen the existence of the RTE as well as the convergence of the
s
the clutter follows a Gaussian distribution and the regularized associated recursive algorithm are two major findings which
c
[ Tyler estimator (RTE) for non-Gaussian spherically invariant have recently been established in several works [13]–[16].
distributed clutters. The rationale behind this choice is that the
Unlike the RSCM, the RTE, as a derivative of the robust
2 RTE is efficient in mitigating the degradation caused by the
Tyler’s estimator, is resilient to the presence of outliers,
v presence of impulsive noises while inducing little loss when the
7 noise is Gaussian. thereby making it more suitable to radar applications, for
2 Based on recent random matrix theory results studying the which experimental evidence rules out Gaussian models for
0 asymptotic fluctuations of the statistics of the ANMF detector the clutter [17]–[20].
6 whenthenumberof samplesand theirdimensiongrow together
Asfarasregularizedestimationmethodsareconcerned,itis
0 toinfinity,weproposea designfor theregularization parameter
essentialtodetermineacleverwayofsettingtheregularization
. that maximizes the detection probability under constant false
1 alarm rates. Simulation results which support the efficiency of parameter. This question has essentially been investigated in
0
theproposedmethodareprovidedinordertoillustratethegain [21], [22] for the RSCM and in [15], [23] for the RTE.
5
of the proposed optimal design over conventional settings of the Althoughyielding differentexpressions, these works have the
1
regularization parameter.
: common denominator of being merely based on a distance
v
IndexTerms—RegularizedTyler’sestimator,AdaptiveNormal- minimization between the RTE or the SCM and the true
i
X ized Mached Filter, robust detection, Random Matrix Theory, covariance matrix. It is thus not clear whether these choices
Optimal design.
r will allow for good performances when applied to detection
a
problems.We considerin thisworkthedesignofthe adaptive
I. INTRODUCTION normalizedmatchedfilter (ANMF)forradardetection.Firstly
The estimation of scatter matrices is of fundamental im- introduced by [24] and analyzed in [25]–[27], this scheme
portance for space-time adaptive processing (STAP) which was shown to enjoy the interesting features of constant false
underlies the design of radar systems [1], [2]. In radar detec- alarmpropertywithrespecttotheclutterpowerandcovariance
tion for instance, it is well-acknowledged that a sufficiently matrix.Thisdetectorisobtainedbyreplacinginthestatisticof
accurate scatter matrix is key to enhancing the detection thenormalizedmatchedfilter(NMF)thecovariancematrixby
performance (see [3], [4] and references therein). In order to a given estimate [26], which is computedbased on secondary
support a possible deficiency in samples (number of samples data observations, i.e., n signal free independent and identi-
less than their dimensions), regularized covariance matrix cally distributed (i.i.d.) observations. Of interest in this work
estimation methods have been proposed [5]. One regularized are the cases where the RSCM or the RTE are used in place
estimation method is given by the use of the regularized of the unknown covariance matrix. We will consider first the
samplecovariancematrix(RSCM).TheRSCMfundamentally scenario where the detector operates overGaussian correlated
originates from the diagonal loading approach which can be clutters and thus uses the RSCM as a replacement for the
traced back to the works of Abramovich and Carlson [5], unknown covariance matrix, a scheme which will be referred
[6]. As a derivative of the sample covariance matrix (SCM), to as ANMF-RSCM. In order to come up with an appropriate
the RSCM inherits its main major limitation of exhibiting design for the ANMF detector, it is essential to characterize
poor performances when observations contain outliers. This the behaviour of its corresponding false alarm and detection
2
probabilities. where y CN represents the vector received by an N-
∈
Under the assumption of fixed dimensions, such a task dimensionalarrayofsensors,xstandsforthenoiseclutterand
seems to be out of reach. This has led us to consider in [28] αisacomplexscalarmodelingtheunknowntargetamplitude.
the regime wherein the number of secondary data samples The signal detection problem is phrased as the following
and their dimensions grow simultaneously to infinity, thereby binary hypothesis test:
allowing for the use of advanced tools from random matrix
H : y=αp+x
theory.Theasymptoticfalse alarmprobabilityfortheANMF- 1 (1)
H : y=x.
RTE was in particular derived in [28] (yet only as a mere (cid:26) 0
applicationexampleofthemainmathematicalresult.).Inorder Several models for the clutter x have been proposed. Among
to allow for an optimal choice of the regularization parame- them, we distinguish the class of CES random variates
ter, these results need to be augmented with an asymptotic whichencompassmostof thecommonlyencounteredrandom
analysisof thedetectionprobability.Thisconstitutesthemain models, including the standard Gaussian distribution, the K-
contribution of our work. In particular, we extend the results distribution, the Weibull distribution and many others [3]. A
of [28] to the ANMF-RSCM detector by showing that its CES distributed random variable is given by:
corresponding statistic flucutates as a Rayleigh distribution
1
when no target is present and, additionally, establish that it x=√τC2w˜
N
behaves like a Rice distributed random variable otherwise.
whereτ isapositivescalarrandomvariablecalledthetexture,
Basedontheasymptoticcharacterizationofthesedistributions,
C is the covariance matrix1 and w˜ is an N-dimensional
we propose an optimal setting of the regularizationparameter N
vector independent of τ, zero-mean unitarily invariant with
thatmaximizestheasymptoticprobabilityofdetectionforany
1
norm w˜ = √N. The quantity C2w˜ is referred to as
given false alarm probability. k k N
speckle. The design of an appropriate statistic to the above
In a second part, we consider the case where the clutter
hypothesis test depends on the amount of knowledge that is
is drawn from heavy tailed distributions. It is thus natural to
availableto thedetector.IftheclutterisGaussianwithknown
assume that the detector uses the RTE, since the RSCM is
covariance matrix C while α is unknown, the Generalized
vulnerable to the presence of outliers and may provide poor N
Likelihood Ratio (GLRT) for the detection problem in (1)
performances. This scheme will be coined ANMF-RTE. By
results in the following test statistic:
exploiting recent results on the asymptotic behavior of the
RTEestimator[28],[29],weprovethat,uptoacertainchange y C 1p
of variable, the ANMF-RTE is asymptotically equivalent to TN = ∗ −N
the ANMF-RSCM when operating over Gaussian clutters. y∗(cid:12)(cid:12)C−N1yp∗C(cid:12)(cid:12)−N1p
This argues in favor of the role of the RTE to retrieve the q
which corresponds to the square-root statistic of the ANMF
Gaussianperformanceswhileoperatingoverheavydistributed
detector. The statistic T has been derived independently
clutters. Finally, we prove through simulations the superiority N
by several works, thereby leading the corresponding detector
of our design to some of the adhoc recent settings of the
to have many alternative names: the constant false alarm
regularization parameter that have recently been proposed.
(CFAR) matched subspace detector (MSD) [30], the nor-
The gain of our design likely owes to the high accuracy
malized matched filter (NMF) [31], or the Linear Quadratic
of the derived asymptotic results in predicting the detection
GLRT(LQ-GLRT)[32].Iftheclutterisellipticallydistributed,
probability of the ANMF schemes.
optimal detection procedures based on the GLRT principle
The remainder of the paper is organized as follows. In the
lead to statistics that depend on the distribution of the texture
first section, we introduce the considered problem. Then, we
τ. Nevertheless, a complete knowledge of the statistics of the
propose an optimal design approach for the ANMF-RTE and
signal and noise cannot be acquired in practice. A reasonable
theANMF-RSCM.Finally,weillustrateusingsimulationsthe
hypothesis, largely used in radar detection, is to assume that
gainoftheproposeddesignmethodoverconventionalsettings
onlyp isknownwhile α andthestatistics ofthe noiseare ig-
of the regularization parameter.
nored.Tohandlethiscase,theuseofT whoseoptimalityhas
Notations: Throughout this paper, we depict vectors in N
only been shown in the Gaussian setting has been advocated
lowercase boldface letters and matrices in uppercase boldface
as a good detection technique. Such a choice has particularly
letters. The notation (.) stands for the transpose conjugate
∗ been motivated by the result of [24] showing the asymptotic
while tr(.) and (.) 1 are the trace and inverse operators. The
− optimality of T , when N becomes increasingly large, under
notation . standsfortheEuclideannormforvectorsandfor N
kk a.s. the setting of compound-Gaussian distributed clutters. From
spectral norm for matrices. The arrow designates almost
sure convergence. The statement X ,−→Y defines the new the expression of TN, it can be seen that the detector is only
requiredto know C up to a scale factor,which is muchless
N
notation X as being equal to Y.
restrictivethantherequirementofoptimaldetectionstrategies.
Since the covariance matrix C is unknown in practice,
N
II. PROBLEM STATEMENT a popular approach consists in replacing in T the unknown
N
We consider the problem of detecting a complex signal covariancematrixC byanestimatebuiltonsignalfreei.i.d.
N
vector p corrupted by an additive noise as:
1Notethatwhenthesecondorderstatisticsexist,thescattermatrixisequal
y=αp+x tothecovariance matrix(uptoaconstant).
3
observations x , ,x , termed secondary data. The result- of how should the regularization parameter ρ be set naturally
1 n
···
ing detector is called the adaptive normalized matched filter arises. Recent previous works dealing with this issue propose
(ANMF). Several concurrent estimators of C can be used. tosetρ insucha wayasto minimizea certainmean-squared-
N
The most popular one is the traditional sample covariance error between Cˆ and C [?], [15]. While easy-to-compute
N N
matrix (SCM) given by: estimates of these values of ρ were provided, one of the
n major criticism to these choices is that they are performed
1
RN = n xix∗i regardlessoftheapplicationunderconsideration.Inparticular,
i=1 a more relevantchoice to the applicationunder study consists
X
which corresponds tob the Maximum-Likelihood estimator in selecting the values of ρ that maximize the probability of
(MLE)iftheclutterisGaussiandistributed.However,insome detection while keeping fixed the false alarm probabilities.
scenarios where the available number of observations n is Thesevalueswillbeconsideredasoptimalinregardsofradar
of the same order or smaller than N, the SCM, being ill- detection applications.
conditioned, will not lead to accurate detection results2. A To this end, one needs to characterize the distribution of
practical approach that has received considerable attention is TNRSCM(ρ) and TNRTE(ρ) given by:
to regularize the SCM, thereby yielding the regularized SCM
(RSCM) given by: b TRSCMb(ρ)= y∗R−N1(ρ)p (4)
N (cid:12) (cid:12)
RN(ρ)=(1 ρ)RN+ρIN, (2) y∗R−N(cid:12)(cid:12)1(ρb)y p∗R(cid:12)(cid:12)−N1(ρ)p
−
iwmhpeorertatnhceeptaoratmhebetesarmρpl∈e [c0o,v1a]risaebnrcveesmtaotrigxivRe Nmodreepoerndliensgs bTNRTE(ρ)= q b (cid:12)y∗C−Nq1(ρ)pb(cid:12) (5)
on the available numberof samples. The ANMF thatuses the y∗C−N(cid:12)(cid:12)1(ρb)y p∗C(cid:12)(cid:12)−N1(ρ)p
RSCM as a plug-in estimator of CN will bebreferred to as underhypobthesesH anqdH .ForfixedqN andn,thisisnotan
ANMF-RSCM. 0 1b b
easy task and in our opinion would not lead, if ever feasible,
While this regularization artifice has revealed efficient in
toeasy-to-computeexpressionsforthe optimalvaluesofρ. In
handlingthescarcityoftheavailablesamples,ithastheserious
this paper,we relax this restrictive assumptionby considering
drawbackoffundamentallyrelyingonthe SCM. In effect,the
the case where N and n go to infinity with N c (0, ).
SCM, even thoughsuitable forGaussian settings, is knownto n → ∈ ∞
This in particularenables leveragingthe recentresults of [28]
be vulnerable to outliers and thus leads to highly inefficient
that will be reviewed in Section IV-A.
estimatorswhenthesamplesaredrawnfromheavytailednon-
Gaussian distributions. A standard alternative to conventional
III. OPTIMAL DESIGN OFTHEANMF-RSCM DETECTOR:
sample covariance estimates is constituted by the class of
GAUSSIAN CLUTTERCASE
robust-scatterestimators,knownfortheirresiliencetoatypical
Inthissection,weconsiderthecaseofaGaussianclutter.In
observations. The robust estimator that will be considered in
this work was defined in [14] as the unique solution Cˆ (ρ) otherwords,weassumethatallthesecondarydatax1, ,xn
N ···
are drawn from Gaussian distribution with zero-meanand co-
to:
CˆN(ρ)=(1−ρ)n1 Xi=n1 N1x∗iCxˆi−Nx1∗i(ρ)xi+ρIN. (3) cvwoaarrrirdaensspcaeonnCodpNinti.gmFsaotlrastρiest∈ttiicn(g0T,NoR1fS],tChwMeer.deIgenufilnoaerrditzeharetitRoonSpCcaoMveeffiatshcieiennw(t2aρ)y,awtnode-
with ρ max 0,1 n ,1 . This estimator corresponds need to characterize thebasymptotic false alarm and detection
to a hyb∈rid robust-shri−nkNage estimator reminding Tyler’s M- probabilitiesundertheassumptionsthatc , N c.Thatis,
(cid:0) (cid:0) (cid:1) (cid:3) N n →
estimator of scale [12] and Ledoit-Wolf’sshrinkage estimator providedH orH istheactualscenario,(y=xory=αp+
0 1
[21].WewillthusrefertoitastheRegularized-TylerEstimator x), we shall evaluate the probabilities P TRSCM >ΓH
(RTE). Besides its robustness, the RTE has many interesting N | 0
features. First, it is well-suited to situationswhere cN , Nn is and P TNRSCM >Γ|H1 for Γ > 0. Befhorbe going furtheri,
largewhilestandardrobustscatterestimatesareill-conditioned we neehd to stress that siome extra assumptions on the order
or even undefined if N > n. By varying the regularization of magnbitude of α and Γ with respect to N should be made
parameter ρ, one can move from the unbiased Tyler-estimator to avoid getting trivial results. Indeed, it appears that under
[34] (ρ = 0) to the identity matrix (ρ = 1) which represents H0, the randomquantities √1Ny∗R−N1(ρ) pp , N1y∗R−N1(ρ)y,
a crude guess for the unknown covariance CN. Its relation and p∗R−N1(ρ) pp2 are standard objectskikn random matrix
to the Tyler’s estimator has recently been reported in [15] by theory, which ckonkverge almost subrely to their meabns when
viewing it as the solution of a penalized M-estimation cost both Nband n grow to infinity with the same pace [35]. As
function. We will denote by ANMF-RTE the ANMF detector a result, since √1Ny∗R−N1(ρ)p −a.→s. 0, TNRSCM −a.→s. 0 for
that uses the RTE instead of the unknown covariance matrix. all Γ > 0, which does not allow to infer much information
Upon replacingin TN the unknowncovariancematrix by a about the false alarm pbrobability. It turnsbout that the proper
regularized estimate, be it the SCM or the RTE, the question scaling of Γ should be Γ = N−21r for some fixed r > 0,
an assumption already considered in [28]. Similarly, one can
2Traditionally, it is assumed that 2N observations are required to ensure
see that under H , the presence of a signal component in
goodperformancesofthesub-optimalfiltering,i.e.,a3dBlossoftheoutput 1
SNRcompared tooptimal filtering[33]. y causes TRSCM to converge almost surely to some positive
N
b
4
constantifαdoesnotvarywithN.Therefore,forΓ=N−12r, where:
P TRSCM >ΓH 1. In order to avoid this trivial
stahtemNent, we s|ha1llias→sume that α = N−12a for some fixed σN2,SCM(ρ), 21p Q p(ρ∗C)pN1Qt2NrC(ρ)pQ (ρ)
a >b 0 with p = N. In practice, this means that the ∗ N N N N
k k 1
dimension of the array is sufficiently large to enable working
×1 c(1 ρ)2m ( ρ)2 1 trC2 Q2 (ρ)
in low-SNR regimes. − − N − N N N
Prior to introducing the results about the false alarm and
detection probabilities, we shall introduce the following as- and QN(ρ),(IN+(1−ρ)mN(−ρ)CN)−1.
sumptions and notations: The uniformity over ρ of the convergence result in The-
1 orem 1 is essential in the sequel. It obviously implies the
Assumption A-1. For i 1, ,n , x =C2w with:
∈{ ··· } i N i pointwiseconvergenceforeachρ>0but,moreimportantly,it
w , ,w are N 1 independent standard Gaussian
1 n willallowustohandletheconvergenceofthefalsealarmprob-
• ··· ×
random vectors with zero-mean and covariance I ,
N ability when random values of the regularization parameter
C CN N is such that limsup C < and
N × N are considered. This feature becomes all the more interesting
• ∈ k k ∞
1 trC =1,
N N knowing that the detector is required to set the regularization
liminf 1p C p>0.
• N N ∗ N parameterbasedonrandomreceivedsecondarydata.Notethat,
Notethatthenormalization 1 trC =1isnotarestricting for technical issues, a set of the form [0,κ), where κ > 0 is
N N as small as desired but fixed, has to be discarded from the
constraint since the statistics under study are invariant to the
uniform convergence region.
scaling of C . The last item in Assumption 1 is required
N
for technical purposes in order to ensure that the considered The result of Theorem 1 provides an analytical expression
statistic exhibits fluctuations under H and H . In practice, for the false alarm probability. Since this expression depends
0 1
thisassumptionimpliesthatthesteeringvectordoesnotlie in on the unknown covariance matrix, it is of practical interest
the null space of the covariance matrix C . to provide a consistent estimate for it:
N
Denote for z C R by m (z) the unique complex
∈ \ + N Proposition 2. For ρ (0,1), define
solution to: ∈
m (z)=( z+c (1 ρ) 1 ρp∗Rb−N2(ρ)p
N ×N1−trCNN(IN−+(1−ρ)mN(z)CN)−1 −1 σˆN2,SCM(ρ)= 21 1−cN+cNNρtr−R−N1(pρ∗)Rb−N1p1−Nρ trR−N1(ρ)
(cid:19) (cid:16) (cid:17)(cid:16) (cid:17)
that satisfies (z) (mN(z)) 0 or unique positive if z <0. and let σˆ2 (1) = lim σˆ2b (ρ) = p∗RbNpb. Then,
Theexistenceℑandℑuniqueness≥ofmN(z)followsfromstandard we have, fNo,rSCanMy κ>0, ρ↑1 N,SCM trRbN
results of random matrix theory [36]. It is a deterministic
quantity, which can be computed easily for each z using sup σˆ2 (ρ) σ2 (ρ) a.s. 0.
fixed-point iterations. In our case, it helps characterize the ρ SCM N,SCM − N,SCM −→
asymptotic behavior of the empirical spectral measure of the ∈Rκ (cid:12) (cid:12)
(cid:12) (cid:12)
random matrix (1 ρ)1 n x x 3.
Define also for−κ>n0, i=SC1Mias∗i: The proof of Proposition 2 follows along the same lines as
PRκ that of Proposition 1 in [28] and is therefore omitted.
SCM ,[κ,1].
Rκ We will now derive the asymptotic equivalent for
With these notations at hand, we are now ready to analyze P TNRSCM(ρ)> √rN|H1 , where under H1 the received vec-
the asymptotic behaviour of the false alarm and detection tohr y is supposed to be giiven by:
probabilities. The proof for the following Theorem will not b
a
be provided since, as we shall see in Section IV, it follows H : y= p+x
1
√N
directly by applying the same approach used in [28].
Theorem 1 (False alarm probability). As N,n with with x distributed as the xi’s in Assumption1. The following
c c (0, ), → ∞ results constitute the major contribution of the present work.
N
→ ∈ ∞ They will lead in conjunction with those of Theorem 8 and
ρ∈sRuSκpCM(cid:12)(cid:12)P(cid:20)TNRSCM(ρ)> √rN|H0(cid:21)−e−2σN2,SrC2M(ρ)(cid:12)(cid:12)−a.→s. 0 PTrhoepoorseimtio3n(2Dteotetchteioonpptirmobaalbdielistiyg)n. Aosf Nthe,nANMF-wRiSthCcMN.
m(rtR1ˆhia−.n(neeN3td.ρL,−So(etPmzmˆizte))Nlnijν−ˆ==emN(s11zaRµt)(cid:12)(cid:12)(cid:12)txrrN(a=iitxnx(−msd∗ifzotbN.1N)1r)−n)m−(DρPzw1e)oPνˆnhfNiNoai=ca.nitesh1(=c.dδ1eta0bλr)pxt.yiap)=ii.rxnobmˆ∗ixeNd1iNemttwPhe(arziettmeNi)h=sienm1iiλitnssp1λtii,itcr1hS−·icem·tzai·ele.al,altTmjλssepuhNsoeresecnttt,rµrtasahqNulneursa,emfnoe(tiesrii.gmeateyens,un(cid:12)(cid:12)(cid:12)smmvergeaiNNlvmˆueo(e(nNfzsz))(tbhzo=iye)sf c, we have ρf−o∈rsQRuaSκ1pCn(cid:18)Myg(cid:12)(cid:12)(cid:12)(cid:12)κSPC>(cid:20)MTb0(NpRS),CσMN(ρ,S)Cr>M(√ρr)N(cid:19)|(cid:12)H−a→1.→s(cid:21).∞0. →
− −→ (cid:12)
(cid:12)
(cid:12)
5
where Q is the Marcum Q-function4 while σ is given matrix C which is unknown to the detector. Acquiring a
1 N,SCM N
in Theorem 1 and g (p) is given by: consistent estimate of f (ρ) based on the available R is
SCM SCM N
thus mandatory.This is the goal of the followingProposition.
1 c(1 ρ)2m( ρ)2 1 trC2 Q2 (ρ)
gSCM(p)= q − − p C−Q2N(ρ)p N N Proposition 4. For ρ∈(0,1), define fˆSCM(ρ) as: b
∗ N N 2 2
×rN2 a|p∗QpN(ρ)p|. fˆSCM(ρ)= (cid:16)p∗Rb−N1(ρp)∗pR(cid:17)−N(11(ρ−)pρ)−(cid:16)ρ1p−∗cR+−NN2c(ρρ)tprRb−N1(ρ)(cid:17)
andletfˆ (1),lim fˆ (ρ)= N .Then,wehave:
Proof: See Appendix A. SCM ρ↑1bSCM p∗RbbNp
According to Theorem 1 and Theorem 3, TRSCM(ρ) be-
N sup fˆ (ρ) f (ρ) a.s. 0,
haves differently depending on whether a signal is present SCM − SCM −→
ρ SCM
or not. In particular, under H , √NTRSCbM(ρ) behaves ∈Rκ (cid:12) (cid:12)
0 N (cid:12) (cid:12)
like a Rayleigh distributed random variate with parameter where we recall that(cid:12) SCM =[κ,1]. (cid:12)
Rκ
σN,SCM(ρ) while it becomeswell-approxibmatedunder H1 by Proof: See Appendix B.
a Rice distributed random variable with parameters g (p)
SCM Since the results in Proposition 4 and Theorem 3 are
andσ (ρ).Itisworthnoticingthatinthetheoryofradar
N,SCM uniform in ρ, we have the following corollary:
detection,gettingafalsealarmandadetectionprobabilitydis-
tributed as Rayleigh and Rice random variables is among the Corollary 5. Let fˆSCM(ρ) be defined as in Proposition 4.
simplestcasesthatonecaneverencounter,holdingonly,tothe Define ρˆ∗N as any value satisfying:
best of the authors’ knoweldege,if white Gaussian noises are
ρˆ argmax fˆ (ρ) .
considered [?, p.188]. We believe that the striking simplicity ∗N ∈ SCM
ρ SCM
of the obtained results inheres in the double averaging effect ∈Rκ n o
that is a consequence of the considered asymptotic regime. Then, for every r >0,
This is to be compared to the quite intricate expressions for
thefalsealarmprobabilityobtainedundertheclassicalregime P √NTN(ρˆ∗N)>r|H1
of n tending to infinity while N is fixed [40]. (cid:16)max P √NT (ρ(cid:17))>rH a.s. 0.
N 1
Wewillnowdiscussthechoiceoftheregularizationparam- −ρ SCM | −→
∈Rκ n (cid:16) (cid:17)o
eter ρ and the threshold r. In accordance with the theory of
radardetection,weaimatsettingρandrinsuchawaytokeep
Proof: The proof is similar to that of Corollary 1 of [28]
the asymptotic false alarm probability equal to a fixed value
and is thus omitted.
η while maximizing the asymptotic probability of detection.
From Corollary 5, the following design procedure leads to
From Theorem 1, one can easily see that the values of r and
optimal performance detection results:
ρ that provide an asymptotic false alarm probability equal to
First, setting the regularization parameter to one of the
η should satisfy: • values maximizing fˆ (ρ):
r SCM
= 2logη.
From these choiceσsN,,wSCeMh(aρv)e topta−ke those values that max- ρˆ∗N ∈aρ∈rgRmSκCaMxnfˆSCM(ρ)o (6)
imize the asymptotic detection which is given, according to Second, selecting the threshold rˆas:
•
Theorem 3, by:
r rˆ=σˆN,SCM(ρˆ∗N) −2logη (7)
Q1(cid:18)gSCM(p),σN,SCM(ρ)(cid:19). IV. OPTIMAL DESIGN OFTHpEANMF-RTE:
The second argument of Q1 should be kept fixed in order NON-GAUSSIAN CLUTTER
to ensure the required asymptotic false alarm probability. As
This section discusses the design of the ANMF-RTE de-
the Marcum-Q function increases with respect to the first
tector in the case where the clutter is non-Gaussian. In
argument, the optimization of the detection probability boils
particular, we assume that the secondary observations satisfy
down to considering the following values of ρ:
the following assumptions:
ρ argmax f (ρ) 1
∈ { SCM } Assumption A-2. For i ∈ {1,··· ,n}, xi = √τiCN2wi =
where: √τizi where
1
fSCM(ρ), 2a2gS2CM(p) • w1,··· ,wn are N×1 independent unitarly invariant
complex zero-mean random vectors with w 2 =N,
i
However,theoptimizationoffSCM(ρ)isnotpossibleinprac- CN CN×N is such that limsup CNk k< and
tice, since the expression of fSCM(ρ) features the covariance • N1 trC∈N =1. k k ∞
τ >0 are independent of w .
ord4erQm1(oad,ifib)ed=BRebs+se∞lfxunecxtpio(cid:16)n−ofxt2h+2eafi2r(cid:17)stIk0i(nadx.)dx where I0 isthe zero-th •• liiminf N1p∗CNp>0. i
6
TherandommodeldescribedinAssumption2isthatofCES for each ǫ > 0, the above convergence does not suffice
distributions which encompass a wide range of observation to obtain the convergence of most of the commonly used
distributions obtained for different settings of the statistics of functionals which involve fluctuations of order N−12 or N−1
τ . Prior to stating our main findings, we shall first review (e.g.quadraticformsofCˆ (ρ)orlinearstatisticsoftheeigen-
i N
some recent results concerning the asymptotic behaviour of values of Cˆ (ρ)). While a further refinement of the above
N
the RTE in the asymptotic regime. convergence seems to be out of reach, it has recently been
established in [28] that the fluctuations of special functionals
can be proved to be much faster, mainly by virtue of an
A. Background
averaging effect which cancels out terms fluctuating at lower
This section reviews the recent results in [28] about the speed.Inparticular,bilinearformsofthetypea Cˆk (ρ)bwere
∗ N
asymptotic behaviour of the RTE estimator. studied in [28], where the following proposition was proved:
RecallthattheRTEisdefined,forρ max 0,1 n ,1 ,
∈ −N Proposition 7. Let a,b CN with a = b = 1
as the unique solution to the following equation: ∈ k k k k
(cid:0) (cid:8) (cid:9) (cid:3) deterministic or random independentof x , ,x . Then, as
1 n
CˆN(ρ)=(1−ρ)n1 i=n1 N1x∗iCxˆi−Nx1∗i(ρ)xi+ρIN. Nk ∈,nZ→, ∞, with cN → c ∈ (0,∞), for any··ǫ·> 0 and every
X sup N1 ǫ a Cˆk (ρ)b a Sˆk (ρ)b a.s. 0.
The study of the asymptotic behaviour of robust-scatter esti- − ∗ N − ∗ N −→
ρ RTE
mators is much more challenging than that of the traditional ∈Rκ (cid:12) (cid:12)
sample covariance matrices. The main reasons are that, first, wpohwereerRofRκtThEe misadtreificense(cid:12)(cid:12)dCˆas iannTdhSˆeor.em 6, wher(cid:12)(cid:12)e k ∈Z in any
robust estimators of scatter do not have closed-form expres- N N
sionsand,second,thedependencebetweenthe outer-products Some important consequences of Proposition 7 need to be
involved in their expressions is non-linear, which does not stated. First, we shall recall that, while the crude study of the
allow for the use of standard random matrix analysis. In randomvariatesa Cˆk (ρ)b seems to be intractable,quadratic
∗ N
order to study this class of estimators, new technical tools forms of the type a Sˆk (ρ)b are well-understood objects
∗ N
based on different rewriting of the robust-scatter estimators whose behavior can be studied using standard tools from
have been developed by Couillet et al. [29], [37], [38]. The random matrix theory [39]. It is thus interesting to transfer
importantadvantageofthesetechniquesisthattheysuggestto the study of the fluctuations of a Cˆk (ρ)b to a Sˆk (ρ)b.
∗ N ∗ N
replacerobustestimatorsbyasymptoticallyequivalentrandom Proposition 7 achieves this goal by taking ǫ < 1. Not only
2
matrices for which many results from random matrix theory does it entail that a∗CˆkN(ρ)b fluctuates at the order of N−21
are applicable. In particular, the RTE estimator defined above (since so doesa Sˆk (ρ)b) but also it allows one to provethat
∗ N
has been studied in [28] and has been shown to behave in the a Cˆk (ρ)b and a Sˆk (ρ)b exhibit asymptotically the same
∗ N ∗ N
regimewhereN,n insuchawaythatcN c (0, ) fluctuations.Similarto[28],ourconcernwillberatherfocused
similar to SˆN(ρ) gi→ven∞by: → ∈ ∞ onthecasek = 1.Inthenextsection,wewillshowhowthis
−
n resultcanbeexploitedinordertoderivethereceiveroperating
1 1 ρ 1
SˆN(ρ)= γ (ρ)1 (1−ρ)c n ziz∗i+ρIN, (8) characteristic (ROC) of the ANMF-RTE detector.
N N
− − i=1
X
B. Optimal design of the ANMF-RTE detector
where γ (ρ) is the unique solution to:
N
As explained above, in order to allow for an optimal
t design of the ANMF-RTE detector, one needs to characterize
1= ν (dt).
Z γN(ρ)ρ+(1−ρ)t N the distribution of TNRTE(ρ) under hypotheses H0 and H1.
Using Proposition 7, we know that the statistic TRTE(ρ)
More specifically, the following theorem applies: N
which cannot be habndled directly, has the same fluctuations
Theorem 6 ( [29]). For any κ > 0 small, define RRκTE , as TNRTE(ρ) obtained by replacingCˆN(ρ) by SˆN(ρ).bThat is:
κ+max(0,1 c 1),1 . Then, as N,n with c c
(0, ), we ha−ve:− →∞ N → ∈ e y∗Sˆ−N1(ρ)p
(cid:2) ∞ (cid:3) TRTE(ρ)=
N (cid:12) (cid:12)
sup CˆN(ρ) SˆN(ρ) a.s. 0. p∗Sˆ−N(cid:12)(cid:12)1(ρ)p y∗Sˆ(cid:12)(cid:12)−N1(ρ)y
ρ∈RRκTE(cid:13) − (cid:13)−→ where Sˆ (eρ) is given bqy (8). q
(cid:13) (cid:13) N
Theorem 6 establish(cid:13)es a convergence(cid:13)in the operator norm Let ρ = ρ ρ+γN1(ρ)1−1(1−−ρρ)c −1. Then, SˆN(ρ) =
of the difference CˆN(ρ) SˆN(ρ). This result allows one to ρρ−1RN(ρ), wh(cid:16)ere, with a slight(cid:17)abuse of notation, we
transfertheasymptoticfirs−torderanalysisofmanyfunctionals denotebyR (ρ)thematrix(1 ρ)1 n z z +ρI .Since
of CˆN(ρ) to SˆN(ρ). However, when it comes to the study TNRTEb(ρ) remNains unchanged a−fter nscPaliin=g1oif S∗iˆN(ρ)Nand y,
of fluctuations, this result is of little help. Indeed, although we also havbe:
Theorem 6ρ∈csaRunRκpTbEeNeas12i−lyǫ(cid:13)reCˆfiNne(dρ)a−sSˆN(ρ)(cid:13)−a→.s. 0. e TNRTE(ρ)= p∗R(cid:12)(cid:12)(cid:12)−N√11τ(ρy)∗pRb−N1τ1(yρ∗)Rp(cid:12)(cid:12)(cid:12)−N1(ρ)y
(cid:13) (cid:13) e q q
(cid:13) (cid:13)
b b
7
whereτ =1underH .Itturnsoutthat,conditionallytoτ,the and resorting to the dominated convergencetheorem.
0
fluctuations of the robust statistic TRTE(ρ) under H or H Similar to the Gaussian case, we need to build consistent
N 0 1
are the same as those obtained in Theorem 1 and Theorem 3 estimates for σ2 (ρ) and f (ρ) given by:
N,RTE RTE
once a is replaced by a and ρ bybρ5. As a consequence,we τ
have the following resu√lτts: fRTE(ρ)= 2a2gR2TE(p)
Theorem 8 (False alarm probability, [28]). As N,n A consistent estimate for σ2 (ρ) was provided in [28]:
→ ∞ N,RTE
with c c (0, ),
N → ∈ ∞ Proposition 10 (Proposition 1 in [28]). For ρ
wheρr∈esRuρRκpT7→E(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)Pρ(cid:20)isTbNtRhTeEa(fρo)re>me√nrtNio|nHed0(cid:21)m−aep−pi2nσgN2,aRr2nTEd(ρ)(cid:12)(cid:12)(cid:12)(cid:12)(cid:12)→0, (cid:0)max((cid:8)0,1−σˆN2c,−NR1T(cid:9)E,(1ρ(cid:1)).=De12fi(n1e,1c−Nρ+pp∗∗cCCNˆˆ−N−Nρ21)(((ρρ1))pp ρ) ∈
− −
σ2 (ρ), 1 p∗CNQ2N(ρ)p
N,RTE 2p Q (ρ)p1 trC Q (ρ)
∗ N N N N and let σˆ2 (1),lim σˆ2 (ρ). Then, we have:
1 N,RTE ρ↑1 N
× 1−c(1−ρ)2m(−ρ)2N1 trC2NQ2N(ρ) ρ suRpTE σN2,RTE(ρ)−σˆN2,RTE(ρ) −a.→s. 0.
with QN(ρ), IN(cid:0)+(1 ρ)m( ρ)CN −1. (cid:1) Similar to∈Rthκe G(cid:12)(cid:12)aussian clutter case, acq(cid:12)(cid:12)uiring a consistent
− −
estimate for f (ρ) is mandatory for our design. We thus
Theorem9(De(cid:0)tectionprobability). AsN(cid:1) ,n withcN RTE
→∞ → prove the following Proposition:
c (0, ),
∈ ∞
sup P TRTE(ρ)> r H Proposition 11. For ρ∈ max 0,1−c−N1 ,1 , let
ρ−∈ERRκ(cid:20)TQE1(cid:12)(cid:12)(cid:12)(cid:12)(cid:18)(cid:20)gRbTNE(p), σN,√RrTNE(|ρ)1(cid:19)(cid:21)(cid:21)(cid:12)→0, fˆRTE(ρ)=(cid:16)p∗Cˆ(1−N(cid:0)−1(cρN)p+(cid:8)(cid:17)c2N(cid:18)ρN)12 trC(cid:9)ˆN((cid:1)ρ)−ρ(cid:19)
where the expectation is taken over the di(cid:12)(cid:12)(cid:12)stribution of τ, ×p∗Cˆ−N1(ρ)p−ρp∗Cˆ−N2(ρ)p
σ (ρ) has the same expression as in Theorem 8 and
N,RTE and fˆ ,lim fˆ (ρ). Then, we have:
RTE ρ 1 RTE
↑
g (p)= 1−c(1−ρ)2m(−ρ)N1 trC2NQ2N(ρ) sup fˆRTE(ρ) fRTE(ρ) a.s. 0.
RTE q p∗CNQ2N(ρ)p ρ∈RRκTE(cid:12)(cid:12) − (cid:12)(cid:12)−→
(cid:12) (cid:12)
q
×rN2τa p∗QN(ρ)p . R−N1(Pρr)ooafn:dTρhebyprρooinf ftohlelorwessulbtsyofifrsPtroreppolsaictiionng4Ra−Nn1d(ρu)sibnyg
and Q is the Marcum Q(cid:12)-function. (cid:12) the convergences[28]: b
1 (cid:12) (cid:12)
b
Proof: Since the fluctuations of the robust statistic Cˆ (ρ)
sup N R (ρ) a.s. 0.
TbyNRTaE(,ρw)eishtahveesafomreaansythfiaxteodfτT,NRSCM(ρ)whenaisreplaced ρ∈RRκTE(cid:13)(cid:13)N1 trCˆN(ρ)− N (cid:13)(cid:13)−→
b √τ b sup (cid:13)(cid:13) ρCˆ (ρ) ρRb(ρ) (cid:13)(cid:13)a.s. 0.
r (cid:13) N − N (cid:13)−→
ρ−∈sQRuRκ1pT(cid:18)Eg(cid:12)(cid:12)(cid:12)(cid:12)RPT(cid:20)ETb(NpRT),Eσ(Nρ),R>rTE√(ρN)(cid:19)|H(cid:12)1−a,.→τs.(cid:21)0. and ρ∈RRκT(cid:12)Eρρ(cid:13)(cid:13)(cid:13)−N1 trCˆN(ρb)(cid:12)−a→.s.(cid:13)(cid:13)(cid:13)0.
(cid:12) (cid:12) (cid:12)
The result thus follows by noticing the fol(cid:12)lowing inequality (cid:12) (cid:12)
(cid:12) Since the resul(cid:12)ts in Proposition(cid:12) 11 and Theorem 9 are
r
sup P TRTE(ρ)> H uniform in ρ, we have the following corollary:
ρ−∈ERRκTQE1(cid:12)(cid:12)(cid:12)(cid:12) g(cid:20)RbTNE(p), σ √rN(|ρ)1(cid:21) DCeofironlelaρˆr∗Ny a1s2.anLyetvafˆlRuTeEs(aρt)isfbyeindge:fined as in Proposition 4.
(cid:20) (cid:18) N,RTE (cid:19)(cid:21)(cid:12)
≤Eρ∈sRuRκpTE(cid:12)(cid:12)P(cid:20)TNRTE(ρ)> √rN|H(cid:12)(cid:12)(cid:12)1,τ(cid:21) Then, for everyρˆr∗N>∈0a,rgρ∈mRaRκxTEnfˆRTE(ρ)o.
(cid:12) b r
−Q1(cid:18)gRTE((cid:12)p),σN,RTE(ρ)(cid:19)(cid:12) P √NTN(ρˆ∗N)>r|H1
5NotethatvectorycanbeassumedtobeGauss(cid:12)(cid:12)(cid:12)ianwithoutimpactingthe (cid:16)max P √NTN(ρ(cid:17))>r H1 a.s. 0.
asymptotic distributions of√NTNRTE underH0 andH1. −ρ∈RRκTEn (cid:16) (cid:17)| o−→
b
8
Using the same reasoningas the one followed in the Gaus-
sian clutter case, we propose the following design strategy:
First,settheregularizationparametertooneofthevalues Proposed design
• maximizing fˆ (ρ): Theory
RTE
Design using ρˆ and rˆ [22]
chen chen
ρˆ∗N ∈argρmaRxTE fˆRTE(ρ) ; Design without regularization ρˆ=0
∈Rκ n o 1
Second, set the threshold to rˆ a=0.5
•
rˆ=σˆN,RTE(ρˆ∗N) −2logη 0.8
y
where η is the required false alaprm probability. bilit a=0.25
ba 0.6
o
r
V. NUMERICAL RESULTS P
n
A. Gaussian clutter ctio 0.4
e
In a first experiment, we consider the scenario where the Det
clutter is Gaussian with covariance matrix C of Toeplitz 0.2
N a=0.1
form:
bj i i j 0
[CN]i,j = b−i−j ∗ i≤>j , |b|∈]0,1[, (9) 0 0.1 0.2 0.3 0.4
(cid:26) False Alarm Probability
where we set b =(cid:0)0.96(cid:1) N = 30 and n = 60. The steering
vector p is given by
Fig.1. ROCcurves ofANMF-RSCMdesigns fora=0.1,0.25,0.5,p=
p=a(θ) (10) a(θ)withθ=20o,N =30,n=60:Gaussiansetting
where θ [a(θ)] = e πksin(θ). In this experiment, θ is
7→ k −
set to 20o. For each Monte Carlo trial, the simulated data
consists of y = αp+x and the secondary data y , ,y
1 n
which are used to estimate ρˆ and to compute Rˆ (ρ·ˆ··). In
∗N N ∗N
particular,theshrinkageparameterandthethresholdvalueare
Proposed design
determined using (6) and (7). We have observed from the
Theory
considered numerical results that fˆ (ρ) is unimodal and
SCM Design using ρˆ and rˆ [22]
thus the maximum can be obtained using efficient line search chen chen
Design without regularization ρˆ=0
methods. For comparison, we consider two other designs: the
first one is based on the regularization parameter derived in 1
the work of Chen et al. [22, Equation (19)] (we denote by
ρˆ the corresponding coefficient) while the second one
chen
correspondsto the non-regularizedANMF (ρˆ=0). . In order
y 0.8
ttohesafitrisstfydethseignreqthuairtetdhefatlhsereashlaorlmd rˆprobabisiligtyiv,ewnebays:surˆme fo=r bilit
chen chen a
σˆN,SCM(ρˆchen)√ 2logη. For the non-regularizedANMF to ob
− r
satisfy the false alarm probability, the threshold value is set P 0.6
n
based on Equation 11 in [27]. Figure 1 represents the ROC o
curves of both designs for different values of a = α√N, cti
e
namely a = 0.1,0.25,0.5, along with that of the theoretical et
D 0.4
performancesof our design. We note that for all SNR ranges,
the proposed algorithm outperforms the design based on the
regularization parameter ρˆ and the gain becomes higher
chen
0.2
asa increases.Italsooutperformsthenon-regularizedANMF
0 1 10 2 2 10 2 3 10 2 4 10 2 5 10 2
− − − − −
detector. Moreover, the performances of the proposed design · · · · ·
False Alarm Probability
correspond with a good accuracy to what is expected by our
theoretical results.
In order to highlight the gain of the proposed design over Fig.2. ROCcurves ofANMF-RSCMdesigns fora=0.9,p=a(θ)with
themostinterestingrangeoflow falsealarmprobabilities,we θ=20o,N =30,n=60:Gaussiansetting
representinFig.2theobtainedROCcurveswhena=0.8and
the false alarm probabilityspanning the interval [0.001,0.05].
9
B. Non-Gaussian clutter Proposed design
Design using ρˆ , rˆ [15]
In a second experiment, we proceed investigating the per- olilla olilla
formance of the proposed design in the case where a non- 1
Gaussian clutter is considered. In particular, we consider the a=0.5
casewheretheclutterisdrawnfromaK-distributionwithzero
mean, covarianceCN, and shape ν =0.5 [3]. The covariance 0.8
y
mNa=trix30haasndthne =sam60e.form as in (9) with b = 0.96 but with bilit
a
b
Similar to the Gaussian clutter case, we consider for the o 0.6
sake of comparison the concurrent design based on the reg- Pr a=0.25
n
ularization parameter derived in the work of Olilla and Tyler o
in [15, Equation (19)]. We denote by ρˆollila and rˆollila the ecti 0.4 a=0.1
corresponding regularization coefficient and threshold. Note et
D
that, accordingto our theoreticalanalysis, the thresholdrˆ
ollila
should be set to rˆollila = σˆN,RTE(ρˆollila)√ 2logη in order 0.2
−
to satisfy the required false alarm probability. The results are
depicted in Figure 3. We note that for all SNR ranges, the 0 0.1 0.2 0.3 0.4
proposed method achieves a gain over the design based on
False Alarm Probability
theregularizationcoefficientproposedbyOlilla etal.We also
observethat,similartothe firstexperiment,the gainincreases
as a grows but with a lower slope6. Fig.3. ROCcurvesofANMF-RTEdesignsfora=0.1,0.25,0.5,p=a(θ)
In a last experiment, we investigate the impacts of a and withθ=20o,N =30,n=60:Kdistributed clutter setting
the distribution shape ν. Figure 4 represents the detection
probability with respect to a when the false alarm probability 1
is fixed to 0.05. We note that for small values of a, higher
detection probabilities are achieved when the distribution of
the clutter is heavy-tailed (small ν), whereas the opposite y 0.9
occurs for large values of a. In order to explain this change bilit
in behavior, we must recall that heavy-tailed clutters (small a
b
ν) are characterized by a higher number of occurrences of o
r
P 0.8
τ in the vicinity of zero and at the same time more frequent n ν =0.1
o
realizationsoflargevaluesofτ.Ifaissmall,theimprovement cti ν =0.9
in detection performancesachieved by heavy-tailedclutters is ete ν =30
attributed to the artificial increase in SNR over realizationsof D 0.7
small values of τ. As a increases, the power of the signal of
interest is high enough so that the effect of realizations with
large values of τ becomes dominant. The latter, which are
0.6
morefrequentforsmallvaluesofν,are characterizedbyhigh 0.6 0.8 1 1.2 1.4 1.6 1.8
levels of noises, thereby entailing a degradation of detection a
performances.
Fig.4. Detectionprobabilitywithrespecttoa,p=a(θ)withθ=20o,N =
VI. CONCLUSION 30, n = 60, ν = 0.1,0.9,30: K-distributed clutter, false alarm probability
=5%.
In this paper, we address the setting of the regularization
parameterwhentheRSCM ortheRTEareusedintheANMF
detector statistic as a replacement of the unknown covari- growlargesimultaneously.Basedontoolsfromrandommatrix
ance matrix,therebyyieldingthe schemesANMF-RSCM and theoryalongwithrecentasymptoticresultsonthebehaviourof
ANMF-RTE. One major bottleneck toward determining the theRTE,wederivedtheasymptoticdistributionoftheANMF
regularization parameter that optimizes the performances of detector under hypothesis H and H . The obtained results
0 1
the ANMF detector, is linked to the difficulty to clearly have allowed us to propose an optimal design of the regu-
characterize the distribution of the ANMF statistics under the larization parameter that maximizes the detection probability
cases of presence or absence of a signal of interest (H and while keeping fixed the false alarm probability through an
1
H ).Inordertodealwiththisissue,weconsideredtheregime appropriate tuning of the threshold value. Simulations results
0
under which the number of samples and their dimensions clearly illustrated the gain of our method over previously
proposed empirical settings of the regularization coefficient.
6Note that we do not compare with the zero-regularization case as in the One major advantageof our approach is that, contrary to first
firstexperiment, since,contrary totheGaussiancase,wedonothaveinour
intuitions, it leads to simple closed-form expressions that can
disposal theoretical results allowing the tuning of the threshold to the value
thatachieves therequiredfalsealarm probability. be easily implemented in practice. This is quite surprising
10
giventhat the handlingof the classical regime where n grows Arguing in a similar way to that in (11), we know that the
to infinity with N fixed has been shown to be delicate. As quantity 1ap R 1p does not fluctuate and converges to:
N ∗ −N
a matter of fact, it has thus far been considered only for the
1 a
non-regularizedTyler estimator where intricate expressionsin Nabp∗R−N1p−Nρp∗QN(ρ)p−a.→s. 0,
the form of integrals were obtained [40]. Building the bridge
while, from [28]:
between both approaches is an open question that deserves b
investigation. 1 1 T
√Nℜ(x∗R−N1p),√Nℑ(x∗R−N1p)
(cid:20) (cid:21)
VII. ACKNOWLEDGMENTS 1 b p∗CNQb2N(ρ)p Z′ =o (1)
DICOoNuIiSllOetS’spwroojrekctis(AjoNinRt-ly12s-uMpOpoNrUte-dObOyOt3h)eaFnrdenthche AGNDRR −s2ρ2N 1−cm(−ρ)2(1−ρ)2N1 trC2NQ2N(ρ) p
ISIS–GRETSI“JeunesChercheurs”Project.Pascal’sworkhas for some Z(cid:0)′ (0,I ). (cid:1)
2
bperoejnecptaorftiathlleyIdseuxppPoarrtiesd-Sbacylatyh.e ICODE institute, research Let r = ∼√1NNℜ(x∗R−N1p), √1Nℑ(x∗R−N1p) T. Denote by
ΥN and ωNhthe quantities: i
b b
APPENDIX A Υ = 1 p∗CNQ2N(ρ)p
PROOF OF THEOREM3 N s2ρ2N 1−cm(−ρ)2(1−ρ)2N1 trC2NQ2N(ρ)
a
The proof of Theorem 3 consists of two steps. First, we ωN = p∗QN(cid:0)(ρ)p (cid:1)
study the asymptotic behaviour of the detection probability Nρ
forfixedρ.Then,byasimilarargumenttotheoneconsidered Recall the following distance between probability distribu-
in [28], we establish the uniformity of the result over the tions:
considered set of ρ. Assume that the received signal vector
y is given by: β P,P˜ =sup fdP fdP˜, f BL 1
a − k k ≤
y= p+x (cid:16) (cid:17) (cid:26)Z Z (cid:27)
√N where f BL = f Lip+ f , f Lip being the Lipschitz
normakndk. , tkheksupremkumk∞norkmk[?].Assume forthe mo-
with kpk2 =N and let us write √NTNRSCM(ρ) as: ment that klimk∞supΥN <∞ and limsupNaρp∗QN(ρ)p<∞.
√NTRSCM(ρ)=√N √1Nyb∗R−N1(ρ)√pN . TfrhoempTrohoeforfeomr t1h1e.s7e.1stiante[m?]e,nts will be provided later. Then,
bN qN1y(cid:12)(cid:12)(cid:12)∗R−N1(bρ)yqp∗Rb−NN(cid:12)(cid:12)(cid:12)1(ρ)p β L r,N1 ap∗R−N1p ,L ΥNZ′,ωN →0.
A close inspection of the expressibon of √NTNRSCM(ρ) re- where (cid:18)(X(cid:18)) stands bfor t(cid:19)he p(cid:16)robability d(cid:17)i(cid:19)stribution of
veals that the fluctuations will be governed by the numerator L
X. This in particular establishes that the random vari-
y∗R−N1(ρ)√pN since, from classical results of brandom matrix able r, 1ap R 1p converges uniformly in distribu-
theory, we know that quantities in the denominator exhibit N ∗ −N
a dbeterministic behaviour, being well-approximated by some tion to(cid:16) Υ Z′,ω .(cid:17)From the uniform continuous map-
N N
b
deterministic quantities. In effect, ping Th(cid:16)eorem in [(cid:17)?, Theorem 1], we thus prove that
N1 p∗RN−1(ρ)p−ρ1Np∗QN(ρ)p−a.→s. 0, (11) √doNm(cid:12)(cid:12)N1vyar∗iaRbb−Nle1p(cid:12)(cid:12)witbhehalvoecsatiaosnympNatρoptic∗aQllNy(ρa)spa aRnidce srcaanle-
while: b 1(cid:12) (cid:12)p∗CNQ2N(ρ)p . Using this result
N1 y∗R−N1(ρ)y−N1ρtrCNQN(ρ)−a→.s. 0. (12) arlo2nρg2Nw(i1t−hcmSl(u−tρs)k2y(1−Lρe)m2N1mtar,C2NwQe2Nc(ρo)n)clude that under H1,
TRSCM(ρ) is also asymptotically equivalent to a Rice ran-
N
The first converbgence(11) follows from Theorem 1.1 of [41] dom variate but with location a √p∗QN(ρ)p and scale
whereasthesecondoneisobtainedbyobservingthat,because b √N √N1 trCNQN(ρ)
of the low-SNR hypothesis: σN,SCM. We therefore get, for a fixed ρ,
N1 y∗RN−1(ρ)y−N1 x∗R−N1(ρ)x−a.→s. 0 (cid:12)P(cid:20)TNSCM > √rN|H1(cid:21)−Q1(cid:18)gSCM(p), σN,SrCM(cid:19)(cid:12)−a.→s. 0
(cid:12) (cid:12)
and then using thbe well-known conbvergence result [42]: T(cid:12)hegbeneralizationofthisresulttouniformconvergenc(cid:12)eacross
ρ(cid:12) SCM canbederivedalongthesamestepsasin[(cid:12)28].We
N1 x∗R−N1(ρ)x−N1ρtrCNQN(ρ)−a.→s. 0. no∈wRprκovide details about the control of the limsupΥN and
limsupω . The fact that limsupω < follows directly
N N
We will now debal with the fluctuations of the numerator. We from the last item in Assumption 1, wh∞ile the control of
have: limsupΥ < requires one to check that:
N
∞
√N(cid:12)N1 y∗RN−1p(cid:12)=(cid:12)aNp∗R−N1p+x∗R−N1√pN(cid:12). liminf1−cm(−ρ)2(1−ρ)2N1 trC2NQ2N(ρ).
(cid:12) (cid:12) (cid:12) (cid:12)
(cid:12) b (cid:12) (cid:12) b b (cid:12)
(cid:12) (cid:12) (cid:12) (cid:12)