1 Robust Hypothesis Testing with α-Divergence Go¨khan Gu¨l, Student Member, IEEE, Abdelhak M. Zoubir, Fellow, IEEE, Abstract—Arobustminimaxtestfortwocompositehypotheses, effects that go unmodeled [2]. which are determined by the neighborhoods of two nominal dis- Imprecise knowledge of F or F leads, in general, to per- 0 1 tributionswithrespecttoasetofdistances-calledα−divergence formance degradation and a useful approach is to extend the distances, is proposed. Sion’s minimax theorem is adopted to knownmodelbyacceptingasetofdistributionsF ,undereach characterize the saddle value condition. Least favorable distri- i butions, the robust decision rule and the robust likelihood ratio hypothesis Hi, that are populated by probability distributions test are derived. If the nominal probability distributions satisfy Gi, which are at the neighborhood of the nominal distribution 6 a symmetry condition, the design procedure is shown to be Fi basedonsomedistanceD[1].Undersomemildconditions simplifiedconsiderably.Theparameterscontrollingthedegreeof 1 onD,itcanbeshownthatthebest(errorminimizing)decision 0 robustnessareboundedfromaboveandtheboundsareshownto ruleδˆfortheworstcase(errormaximizing)pairofprobability beresultingfromasolutionofasetofequations.Thesimulations 2 performed evaluate and exemplify the theoretical derivations. distributions (Gˆ0,Gˆ1) ∈ F0 × F1 accepts a saddle value. n Therefore, such a test design guarantees a certain level of a Index Terms—Detection, hypothesis testing, robustness, least detection at all times. This type of optimization is known J favorable distributions, minimax optimization, likelihood ratio test. as minimax optimization and the corresponding worst case 4 distributions (Gˆ ,Gˆ ) are called least favorable distributions 2 0 1 (LFD)s [3]. ] I. INTRODUCTION The literature in this field is unfortunately not rich. One of T Decision theory has been an active field of research ben- the earliest and probably the most crucial work goes back O efiting from contributions from several disciplines, such as to Huber, who proposed a robust version of the probability t. economics, engineering, mathematics, or statistics. A decision ratio test for the (cid:15)−contamination and total variation classes a maker (or a detector) chooses a course of action from several of distributions [4]. He proved the existence of least favorable t s possibilities. A detector is said to be optimal or to be giving distributions and showed that the corresponding robust test [ thebestdecisionforaparticularproblemifthedecisionruleof was a censored version of the nominal likelihood ratio for 3 interestminimizes(ormaximizes)awelldefinedcostfunction, both uncertainty classes. In a later work, Huber and Strassen v e.g., the error probability (or the probability of detection) [1]. extended the (cid:15)−contamination neighborhood to a larger class, 9 In addition to the fact that decision theory is truly an in- which includes five different distances as special cases [5]. 1 terdisciplinary subject of research, there are many areas of It was also shown that the robust test resulting from this 0 5 engineering, where decision theory finds applications, e.g., new neighborhood was still a censored likelihood ratio test. 0 radar, sonar, seismology, communications and biomedicine. AlthoughitwasfoundtobelessengineeringorientedbyLevy 1. For some applications, such as image and speech classifi- [1],thelargestclassesforwhichsimilarconclusionshavebeen 0 cation or pattern recognition, interest is in a statistical test made was for the 2−alternating capacities proposed by Huber 5 that performs well on average. However, for safety oriented and Strassen [6]. 1 applicationssuchasseismologyorforestfiredetection,aswell Another approach for robust hypothesis testing was proposed : v as for biomedical applications such as early cancer detection by Dabak and Johnson based on the fact that the choice i from magnetic resonance images or X-ray images, interest of measures defining the contamination neighborhoods was X is in maximizing the worst case performance because the arbitrary [7]. They chose the relative entropy (KL-divergence) r a consequences of an incorrect decision can be severe [1]. because it is a natural distance between probability measures In general, any practical application of decision theory can and therefore a natural way to define the contamination be formulated as a hypothesis testing problem. For binary neighborhoods. Somewhat surprisingly, the robust test which hypothesis testing, it is assumed that under each hypothesis minimizes the KL-divergence between the LFDs obtained H , the received data y = (y ,...,y ) ∈ Ω follows a par- from the closed balls with respect to the relative entropy i 1 n ticular distribution F corresponding to a density function f , distance was not a clipped likelihood ratio test, but a nominal i i i∈{0,1}. A decision rule δ partitions the whole observation likelihood ratio test with a modified threshold. It was noted space Ω into non-overlapping regions corresponding to each that their approach was not robust for all sample sizes but hypothesis. The optimality of the decision rule δ depends on when Kullback’s theorem is valid, that is for a large number the correctness of the assumption that the data y follows F . of observations [7]. The difference in the robust tests for i However, in many practical applications either F and/or F (cid:15)−contamination and relative entropy neighborhoods lies in 0 1 arepartiallyknownorareaffectedbysomesecondaryphysical thefactthatallthedensitiesintheclassofdistributionsbased on relative entropy are absolutely continuous with respect G. Gu¨l and A. M. Zoubir are with the Signal Processing Group, Institute to the nominal distributions, but not for the case of the ofTelecommunications,TechnischeUniversittDarmstadt,64283,Darmstadt, (cid:15)−contamination class. Germany(e-mail:ggul@spg.tu-darmstadt.de;zoubir@spg.tu-darmstadt.de) ManuscriptreceivedApril19,2005;revisedJanuary11,2007. AquestionleftopenbyDabakandJohnsonwasthedesignof 2 a robust test for a finite number of samples. Levy answered consideringthesquaredHellingerdistance.In[20],thenumber this question under two assumptions; monotone increasing of non-linear equations that needs to be solved to be able nominallikelihoodratioandsymmetricnominaldensityfunc- to design the robust test was reduced and a formula from tions (f (y) = f (−y)), where y ∈ R. He implied that where the maximum robustness parameters could be obtained 0 1 a robust test based on the relative entropy would be more was derived. In [21], robust approaches were extended to suitable for modeling errors rather than outliers, due to the distributeddetectionproblemswherecommunicationfromthe smoothness property (absolute continuity). He also showed sensors to the fusion center is constrained. In a recent work that the resulting robust test was neither equivalent to Huber’s [22], based on the KL-divergence, the monotone increasing nor to Dabak’s robust test; it was a completely different test likelihood ratio constraint was removed. [2]. In this paper, A minimax robust test for two composite AlthoughKL-divergenceisasmoothandanaturaldistancebe- hypotheses, which are formed by the neighborhoods of two tweenprobabilitymeasures,itisnotclearwhyKLdivergence nominal distributions with respect to the α−divergence, is shouldbeconsideredtobuilduncertaintysets,especiallysince designed. It is shown that for any α, the corresponding robust there are many other divergences, which are also smooth and test is the same and unique. There is no constraint on the have nice theoretical properties, e.g. the symmetry property, choiceofnominaldistributions.Therefore,ourdesigngeneral- which KL-divergence does not have. Besides, theoretically izes [2]. Since the α−divergence includes the KL-divergence nice properties do not always lead to preferable engineering or the squared Hellinger distance as a special case, cf. [9], applications, see for example [8, p.7]. In this respect, KL- our work also generalizes the works in [3], [20] and [22]. divergence can be replaced by the α−divergence because The advantage of considering the α−divergence for modeling α−divergenceincludesuncountablymanydistancesasspecial errorsisthatitallowsthedesignertochooseasingleparameter cases, e.g. χ2 distance for α = 2 [9], it reduces to the KL- that accounts for the distance without carrying out tedious divergence as α → 1 and shares similar theoretical proper- stepsofderivationsforthedesignofarobusttest.Additionally, ties with the KL-divergence such as smoothness, convexity the a priori probabilities in our work are not required to be or satisfiability of (generalized) Pythagorean inequality [10]. equal, which was assumed in all previous works on model Moreover,theflexibilityprovidedbythechoiceofαresultsin mismatch. An example is cognitive radio where the primary performanceimprovementsinvarioussignalprocessingappli- user may be idle for most of the time, i.e. P(H ) (cid:29) P(H ) 0 1 cations and implies the sub-optimality of the KL-divergence. [23]. Last but not least, the work in this paper allows vector For example, in the design of distributed detection networks valued observations. withpowerconstraints,α−divergenceisconsideredasthedis- The organization of this paper is as follows. In the following tance between the probability measures, and error exponents section, some background to the minimax optimization prob- of both kinds are maximized over all α∈(0,1) [11]. In non- lemisgivenandcharacterizationthesaddlevalueconditionis negative matrix factorization [12], and indexing and retrieval detailed, before the problem definition is stated. Section III is [13], the optimal value of α (with respect to some objective dividedintothreeparts.Inthefirstpart,theminimaxoptimiza- function) is found to be 1/2 corresponding to the squared tion problem is solved and the least favorable distributions, Hellinger distance. In medical applications; e.g. in medical the robust decision rule as well as the robust likelihood ratio, imagesegmentation[14],restoration[15]andregistration[16], which are later shown to be determined via solving two non- the α−divergence is considered and the optimal value of α is linear equations, are obtained. The second part shows how found to be a non-standard value, i.e. a value which does not the problem is simplified if the nominal probability density correspond to any known distance. There are also theoretical functions satisfy the symmetry condition. In the third part, works which take advantage of the α−divergence in the the maximum of the robustness parameters, above which design of statistical tests. It is reported for parametric models a minimax robust test cannot be designed, are derived. In [17], [18] as well as for non parametric models [19] that Section IV simulation results that illustrate the validity of the use of α−divergence as the distance between probability the theoretical derivations are detailed. Finally, the paper is measures, again with some non-standard values of α, e.g. concluded in Section V. α = 1.6 in [18] and α = 1.3 or α = 1.5 in [19], leads to promising results. However, non of these aforementioned II. PROBLEMFORMULATION works have the property of minimax robustness. Furthermore, A. Background in non of the aforementioned works, it is possible to adjust the tradeoff between robustness and detection performance. Let (Ω,A) be a measurable space with the probability Additionally, the parametric models have a possibly invalid measures F0, F1, G0, G1, and G on it, having the density assumption that the actual probability distributions can be functions f0, f1, g0, g1 and g respectively, with respect to representedbyaparametricmodel.Thismotivatestheworkin some dominating measure µ, i.e., Fi,Gi,G (cid:28) µ, i ∈ {0,1}. thispaper:aminimaxrobustdesignofhypothesistestingwith It is assumed that the nominal measures are distinct, i.e. the the α−divergence distance,where the robustness isadjustable conditionF0 =F1 µ−almosteverywhereisnottrue.Consider withrespecttothedetectionperformancebythechoiceoftwo the binary composite hypothesis testing problem robustness parameters, (cid:15) and (cid:15) . 0 1 Hc :G=G The related literature can be summarized as follows: In [3], 0 0 Hc :G=G (1) thesymmetryconstraintthatwasimposedin[2]wasremoved, 1 1 3 wherethemeasuresG aredefinedwhenevertheircorrespond- which removes the compactness constraint on the set over i ing density functions g belong to the closed ball which maximization is performed. In order for (8) to be valid i the following conditions must hold: G ={g :D(g ,f )≤(cid:15) } i∈{0,1}, (2) i i i i i • TheobjectivefunctionPE(δ,·)isrealvalued,uppersemi- where D is a distance between the density functions. In other continuous and quasi-concave on G ×G for all δ ∈∆ 0 1 words,everydensityfunctiong whichisatleast(cid:15) closetothe i i • The objective function PE(·,(g0,g1)) is lower semi- nominaldensityf isamemberoftheuncertaintyclassG and i i continuous and quasi-convex on ∆ for all (g ,g ) ∈ 0 1 defines G , i∈{0,1}. We choose D to be the α−divergence i G ×G 0 1 i.e., • ∆ is a compact convex subset of a linear topological 1 (cid:18) (cid:90) (cid:19) space D(g,f;α):= 1− gαf1−αdµ ,α∈R\{0,1} α(1−α) Ω • G0×G1 is a convex subset of a linear topological space (3) The first two conditions hold true because P is a real valued E sinceitisaconvexdistanceforeveryαanditincludesvarious continuous function, and linear on all three terms δ,g ,g , distances as special cases [9, p.1536].1. Given that y ∈Ω has 0 1 therefore both convex and concave. The last condition is also been observed, a randomized decision rule δ : Ω (cid:55)→ [0,1] true because, all convex combinations of g0 ∈G and g1 ∈G maps each y to a real number in the unit interval. Let ∆ be i i i i are in G since D is a convex distance and the Cartesian i thesetofalldecisionrules(functions).Then,foranypossible product of convex sets is again a convex set. Similarly, ∆ is choice of δ ∈ ∆, the following error types are well defined: a convex set because for any t∈[0,1] and for all δ ,δ ∈∆, 0 1 first, the false alarm probability tδ +(1−t)δ ∈∆. Note that any continuous function is also 0 1 (cid:90) upper or lower semi-continuous and any convex function is P (δ,f )= δf dµ, (4) F 0 0 also quasi-convex. Lastly, ∆, which is equivalent to [0,1]Ω Ω in infinite dimensional vector space, is the product of un- second, the miss detection probability countablymanycompactsets[0,1].AccordingtoTychonoff’s (cid:90) P (δ,f )= (1−δ)f dµ, (5) theorem, ∆ is compact with respect to the product topology M 1 1 Ω [26], [27]. Note that any finitely supported discretization of and third, the overall error probability g and g makes both G ×G and ∆ compact with respect 0 1 0 1 to the standard topology. This is a straightforward result of P (δ,f ,f )=P(H )P (δ,f )+P(H )P (δ,f ). (6) E 0 1 0 F 0 1 M 1 Heine-Borel theorem [28, Theorem 2.41]. It is well known that P is minimized if the decision rule is Accordingly, based on Sions’s minimax theorem, there exists E chosen to be the likelihood ratio test a saddle value for the objective function P , i.e., E  0, l(y)<ρ PE(δ,gˆ0,gˆ1)≥PE(δˆ,gˆ0,gˆ1)≥PE(δˆ,g0,g1). (9) δ(y)= κ(y), l(y)=ρ, (7) 1, l(y)>ρ Since P is distinct in g and g , we also have E 0 1 where ρ = P(H0)/P(H1) is some threshold, l(y) := P (δˆ,g )≤P (δˆ,gˆ ) F 0 F 0 f /f (y) is the likelihood ratio at observation y and κ:Ω→ 1 0 P (δˆ,g )≤P (δˆ,gˆ ). (10) [0,1]. M 1 M 1 B. Saddle value specification C. Problem definition In this section, the existence of a saddle value condition Based on (10), the minimax optimization problem (8) can due to the functional topology of the minimax optimization be solved considering the Karush-Kuhn-Tucker (KKT) multi- problem is shown. Minimax theorem, which is attributed to pliers. Hence, the problem formulation can be restated as John von Neumann, gives the necessary conditions such that theexistenceofasaddlevalueisguaranteed[24].However,it Maximization: gˆ =arg sup P (δ,g ) 0 F 0 isapplicableifandonlyifbothsetsoverwhichthemaximiza- g0∈G0 tionandminimizationisperformedarecompact.Notethatthe (cid:90) s.t. g >0, Υ(g )= g dµ=1 closed balls (G and G ) with respect to the α−divergence 0 0 0 0 1 R distance are not compact, therefore Von Neumann’s minimax gˆ =arg sup P (δ,g ) 1 M 1 theorem is not applicable in our case. Here, we adopt Sion’s g1∈G1 minimax theorem [25], (cid:90) s.t. g >0, Υ(g )= g dµ=1 1 1 1 sup minP (δ,g ,g ) R E 0 1 (g0,g1)∈G0×G1δ∈∆ =min sup P (δ,g ,g ), (8) Minimization: δˆ=argminP (δ,gˆ ,gˆ ). (11) E 0 1 E 0 1 δ∈∆(g0,g1)∈G0×G1 δ∈∆ In (11), there are two separate maximization problems, which 1Noticethatα−divergenceispreferredagainsttheReńyi’sα−divergence becauseReńyi’sα−divergenceisconvexonlyforα∈[0,1][9,p.1540] are coupled with the minimization problem through the deci- 4 sion rule δ2 implying the robust likelihood ratio function  l−1l, l<ρl III. ROBUSTDETECTIONWITHα−DIVERGENCE ˆl= gˆ1 =ρl, ρl ≤ll ≤ρl (18) gˆ l u Thefollowingtheoremprovidesasolutionfor(11),whichis 0 l−1l, l>ρl u u composedoftheleastfavorabledensitiesgˆ andgˆ ,therobust decision rule δˆ, the robust likelihood ratio0functio1n ˆl=gˆ /gˆ provideauniquesolutionto(11).Furthermore,theparameters 1 0 l and l can be determined by solving in parametric forms, as well as two non-linear equations from l u (cid:32) which the parameters can be obtained. Before the statement 1 (cid:90) (cid:90) l α f dµ+ Φ(cid:48)(l ,l ;α,ρ)αf dµ lof≤th1e≤thelorρlu}≡{y :ˆl(y)>ρ} (14) gˆ0 and gˆ1, and the robust likelihood ratio function ˆl. In the minimization stage, the LFDs and the robust decision rule and δˆ are made explicit. Finally, in the optimization stage, four Φ (l,l ,l ;α,ρ)= parameters that are needed to design the test are reduced to 1 l u 1 two parameters without loss of generality. · Proof: z(l ,l ;α,ρ) l u (cid:18) k(l ,l )α−1(lα−1−lα−1) (cid:19)α−11 A. Derivation of LFDs and the robust decision rule l u l u llα−1−(k(ll,lu)lu)α−1+(k(ll,lu)α−1−1)(l/ρ)α−1 1) Maximization step: Consider the Lagrangian function with Φ =Φ lρ−1. L(g ,λ ,µ )=P (δ,g )+λ ((cid:15) −D(g ,f ;α))+µ (1−Υ(g ))), 0 1 0 0 0 F 0 0 0 0 0 0 0 (21) Theorem III.1. The least favorable densities where µ and λ ≥0 are the KKT multipliers. It can be seen 0 0 z(ll,llul;α,ρ)f0, l<ρll tfhoarteLveirsyaλstri>ctly0.cTonhcearevfeorfeu,ncthtieornealexoifstgs0,aausn∂iq2uLe/∂sogl02utρl z(ll,lu;α,ρ) 0 u More explicitly the Lagrangian can be stated as (cid:32) (cid:90) λ z(ll,lu1;α,ρ)f1, l<ρll L(g0,λ0,µ0)= Rδg0−µ0g0+ α(1−0 α) (1−α)f0 gˆ1 =Φz(1kll((,llllu,,ll;αul,),ρlu)f;α1,,ρ)f1, lρl>l ≤ρllu≤ρlu , (16) +αg0−(cid:18)fg0(cid:19)αf0(cid:33)+λ0(cid:15)0+µ0dµ. (22) 0 and the robust decision rule Note that similar to [2], the positivity constraint g ≥ 0 (or  0 δˆ=01(l,,lα−1−(k(ll,lu)llulα)−α1−(1l/)ρ(l)/1ρ−)α1−−α1+k(ll,lu)α−1−1, llρl<>l ≤ρρllllu≤ρlu , gsgtha1retai≥nsmgfi0ieaa)dxniimasoupnuttoomitmmioaimztfiapct(oai2osl2lney)d,,m,wtbuhhesectiladebuiersfeeoccrhftoieoocrtnhksaeeoldrms(fGeeoaârαct,ephaothussixoist’lisuvc)tioitonyd.nsetrTrioaovfianfitLtinvaides- (17) of the Lagrangian L with respect to g in the direction of a 0 function ψ is taken: com2Ipnagcetnseertaslianrtghseutpopmolaoygineostianldwuacyesdbbeyatchheieαve-ddivsienrcgeenGc0eadnisdtaGn1cea.rIenntohnis- (cid:90) (cid:34) λ (cid:32)(cid:18)g (cid:19)α−1 (cid:33)(cid:35) paper, existence of gˆ0 and gˆ1 is due to the KKT solution of the minimax δ−µ0+ 1−0α f0 −1 ψdµ. (23) optimizationproblem,whichisintroducedinSectionIII. Ω 0 5 Since ψ is arbitrary, L is maximized whenever Υ(gˆ) = 1, i ∈ {0,1} are imposed. This leads to four non- i linear equations: δ−µ + λ0 (cid:32)(cid:18)g0(cid:19)α−1−1(cid:33)=0. (24) (cid:90) (cid:90) (cid:90) 0 1−α f0 c1 ˆl<ρf0dµ+ ˆl=ρΦ0f0dµ+c2 ˆl>ρf0dµ=1, (cid:90) (cid:90) (cid:90) Solving (24) the density function of the LFD Gˆ0, c3 f1dµ+ Φ1f1dµ+c4 f1dµ=1, ˆl<ρ ˆl=ρ ˆl>ρ (cid:90) (cid:90) (cid:90) (cid:18) (cid:19) 1 1−α α−1 cα f dµ+ Φαf dµ+cα f dµ=x(α,(cid:15) ), gˆ = (µ −δ)+1 f (25) 1 0 0 0 2 0 0 0 λ 0 0 ˆl<ρ ˆl=ρ ˆl>ρ 0 (cid:90) (cid:90) (cid:90) cα f dµ+ Φαf dµ+cα f dµ=x(α,(cid:15) ), is obtained. Writing the Lagrangian for P , in a similar way, 3 1 1 1 4 1 1 M ˆl<ρ ˆl=ρ ˆl>ρ with the KKT multipliers µ0 := µ1 and λ0 := λ1 it follows (32) that in four parameters, where x(α,(cid:15))=1−α(1−α)(cid:15). (cid:18) (cid:19) 1 1−α α−1 gˆ = (µ −1+δ)+1 f . (26) 3) Optimization Step: In this section, the number of equa- 1 λ1 1 1 tions as well as the number of parameters are reduced. This allows the re-definition of ˆl, δˆ, gˆ and gˆ in a more compact Accordingly, the robust likelihood ratio function can be ob- 0 1 form. Let l = c /c and l = c /c , then ˆl = gˆ /gˆ from tained as l 1 3 u 2 4 1 0 (29) indicates the equivalence of integration domains, I , I 1 2 (cid:34) (cid:35) 1 and I as defined by (14). Applying the following steps in ˆl= gˆ1 = 1λ−1α(µ1−1+δ)+1 α−1 l. (27) (32): 3 gˆ 1−α(µ −δ)+1 0 λ0 0 • Consider new domains I1, I2, I3 2) Minimization step: The minimizing decision function is • Use the substitutions c1 :=c3ll and c2 :=c4lu knowntobeoftype(7)withl tobereplacedbyˆl andκtobe • Divide both sides of the first two equations by c3 determined from (27) via solving ˆl =ρ for δ :=δˆ. For every • Equate the resulting equations to each other via 1/c3 leads to c = k(l ,l )c , where k(l ,l ) is as defined by ρ, this results in 4 l u 3 l u (12). Next, the goal is to find a functional f s.t. Φ = 1 0, ˆl<ρ c3f(l,ll,lu,α). Since Φ0f0ρ = Φ1f1, it follows that Φ0 = δˆ=λ0(−1+α+λ1+µ1−αµ1) − λ1(λ0+µ0−αµ0)(l/ρ)1−α, ˆl=ρ. c3f(l,ll,lu,α)lρ−1, therefore it suffices to evaluate only Φ1. 1(−,1+α)(λ0+λ1(l/ρ)1−α) (−1+α)(λ0+λ1(l/ρ)1−α) ˆl>ρ AAppsteenpdibxyAs.teApccdoerrdiivnagtiloy,nΦofisthaelsfounfucltliyonsaplecfifiiesdginivetenrmins 0 (28) ofthedesiredparametersandfunctions.InsertingΦ (whichis 1 Inserting (28) in (25) and (26), the least favorable density now a functional of c ,l,l ,l ,α), c.f., (51), into the second 3 l u functions can be obtained as equation in (32) and noticing that c = k(l ,l )c leads to 4 l u 3 c f , ˆl<ρ c f , ˆl<ρ c3 = 1/z(ll,lu;α,ρ), where z(ll,lu;α,ρ) is as defined by  1 0  3 1 (13). Applying a similar procedure, which can be found in gˆ0 = Φ0f0, ˆl=ρ, gˆ1 = Φ1f1, ˆl=ρ, (29) Appendix B, to δˆ, c.f., (28), for the case ˆl = ρ leads to the c2f0, ˆl>ρ c4f1, ˆl>ρ robust decision rule δˆ as given by Theorem III.1. The least favorable densities, gˆ and gˆ , and the robust likelihood ratio 0 1 where function ˆl are obtained similarly, by exploiting the connection c =(cid:18)(1−α)µ0+λ0(cid:19)α−11 , c =(cid:18)(1−α)(µ0−1)+λ0(cid:19)α−11sb,iemtwpeliefincatthioenpsaervaemnetutearlslycl1e,tcth2,ecfo3,urc4eqaunadtiolnl,slgui.veTnhbeys(a3m2e) 1 λ 2 λ 0 0 to be rewritten as the two equations stated by Theorem III.1. As it was mentioned earlier, both gˆ and gˆ are obtained 0 1 c3 =(cid:18)(1−α)(µλ1−1)+λ1(cid:19)α−11 , c4 =(cid:18)(1−αλ)µ1+λ1(cid:19)α−11aunrieqsuuellty, δˆfraorme atlhseoLuangiqrauneg.iIatnfoLll.oHwesntchea,tˆlth=e sgô1l/ugˆti0o,nafnodunads 1 1 for(11)bytheKKTmultipliersapproachisuniqueasclaimed. and TheoremIII.1canbesummarizedasillustratedinFigure1. (cid:18) (cid:19) 1 Φ = −1+λ0+λ1+µ0+µ1−α(−1+µ0+µ1) α−1 , In other words, for any choice of pair of nominal density 0 λ0+λ1(l/ρ)1−α functions f0 and f1, the robustness parameters (cid:15)0 and (cid:15)1, the (30) Bayesian threshold ρ and the distance parameter α, the robust Φ1 =(cid:18)−1+λ0+λ1λ+µ+0λ+(µl1/ρ−)αα−(1−1+µ0+µ1)(cid:19)α−11 . adnedsigthneoruotpbuutsst tdheecilseiaosnt frauvleorδâ.blNeodteicnesitthyaftugˆn0ctiaonndsgˆgˆ10aarnedthgˆ1e 1 0 (31) scaled versions (with different scaling factors) of the nominal distributions on l < ρl and l > ρl , and in between, they l u Inordertodeterminetheunknownparameters,theconstraints areacompositionofbothnominals,sinceΦ andΦ areboth 0 1 in the Lagrangian definition, i.e., D(gˆ,f ,α) = (cid:15) and functionalsoff andf .Interpretationofthedecisionruleδîs i i i 0 1 6 B. Simplified model with additional constraints Insomecases,evidencethatthefollowingassumptionholds f0, f1 (cid:96) may be available: Minimax Robust ∆ Ε Ε AssumptionIII.2. Thenominallikelihoodratiolismonotone 0, 1 (cid:96) (cid:96) Test Design g g andthenominaldensityfunctionsaresymmetric,i.e.,f (y)= Ρ Α 0, 1 1 , f0(−y)∀y If, additionally, the robustness parameters are set to be equal, (cid:15) = (cid:15) = (cid:15) , or in other words x(α,(cid:15)) = x(α,(cid:15) ) = 0 1 0 Fig. 1. Summary of the robust hypothesis testing scheme given by x(α,(cid:15)1), it follows that TheoremIII.1. (cid:96) δ(y)=1−δ(−y) l ⇑ (cid:40) (cid:41) (cid:40) (cid:41) (cid:40) (cid:41) l =1/l c =c λ =λ u l 2 3 0 1 ⇐⇒ ⇐⇒ l lu yu =−yl c1 =c4 µ0 =µ1 (cid:109) g (y)=g (−y) (33) Ρ 1 0 (cid:144) where y =l−1(l ) and y =l−1(l ). These relationships are l l u u straightforward and therefore the proofs are omitted. Notice l l that, due to monotonicity of l, the limits of integrals I , I l 1 2 and I should be re-arranged e.g., 3 I :={y :l(y)<ρl } 1 l (cid:144) ≡{y :y 1, is Gˆ =Gˆ µ-a.e. It is clear from the saddle value condition 0 1 which implies µ =λ /(α−1) and µ =λ /(α−1), can be 0 0 1 1 (30) that for any possible choice of ((cid:15) ,(cid:15) ), which results in 0 1 examined similarly. Gˆ =Gˆ ,itistruethatP ≤1/2forall(g ×g )∈G ×G . 0 1 E 0 1 0 1 Since infinitesimally smaller parameters guarantee the strict IV. SIMULATIONS inequality P < 1/2, it is sufficient to determine all pos- E sible pairs which result in Gˆ = Gˆ . A careful inspection In this section, some simulations are performed to illustrate 0 1 suggests that the LFDs are identical whenever l → infl the theoretical derivations. Consider a simple hypothesis test- l and l → supl. For this choice I and I are empty sets ing problem u 1 3 and the density functions under each hypothesis are defined Hs :Y =W only on I . Without loss of generality, assume that α < 1, 0 2 Hs :Y =W +A (47) infl = 0 and supl = ∞. For this choice ll → 0 implies 1 µ1 =λ1/(α−1)+1andlu →∞impliesµ0 =λ0/(α−1)+1. where A>0 is a known DC signal, W is a random variable Insertingtheseintooneofthefirsttwoequationsin(32),gives which follows a symmetric Gaussian mixture distribution (cid:90) (cid:0)λ0f0(y)1−α+λ1ρα−1f1(y)1−α(cid:1)1−1α dy =(1−α)1−1α . W ∼ 1(cid:0)N(−µ,σ2)+N(µ,σ2)(cid:1), (48) Ω 2 (38) Similarly, from the third and fourth equations it follows that where N(µ,σ2) is a Gaussian distribution with mean µ and (cid:90) (cid:18) (cid:19) α variance σ2 and Y is a random variable on Ω=R, which is λ0f0(y)1−αα +λ1ρα−1f1(y)1−αf0(y)(α−α1)2 1−α dy consistentwiththedatasampley.Toaccountforuncertainties R on Y under both hypotheses, let α =(1−α)1−α x(α,(cid:15)0) (39) F (y):=P(Y 0 and for all allowable (cid:15) , cf. false alarm and miss detection probabilities for all SNRs, i i Section III-C. Assume that g is fixed and it does not depend as expected. The difference between false alarm and miss on α. Then, the integral (cid:82) gαf1−αdµ is 1 at α = 0 and detection probabilities for the same robust test is small and R α=1, convex in α, and it is positive for all α>0, f and g. it is more pronounced for low SNRs. For high SNRs the Hence, lim (cid:82) gαf1−αdµ = ∞ and lim D(f,g;α) performance of two robust tests are close to each other. The α→∞ R α→∞ is indeterminate. Using L’Hospital’s rule twice we obtain reason is that for high SNRs maximum allowable robustness (cid:82) log2(g/f)(g/f)αfdµ parametersbecomerelativelyhighcomparedtotheparameters K = lim D(g,f;α)= lim R . (50) of both robust settings. Although the nominal test has the α→∞ α→∞ 2 lowest error rates, its performance can degrade considerably The integral (cid:82)Rlog2(g/f)(g/f)αfdµ is also positive and under uncertainties in the nominal model. The robust tests, on convex in α. This implies K →∞ for α→∞. Now, assume theotherhand,haveslightlyhighererrorrates,butguaranteed thatg dependsonα andtendstoalimitingdistributiong∗ for power of the test, which indicates the trade-off between ||g∗−f||>0, when α→∞. Then, our conclusion does not performanceandrobustness.Finally,inthelastsimulation,the change, i.e., K → ∞ for α → ∞. Since D(f,g;α) is finite, 3D boundary surface of the maximum robustness parameters werequirethatα→∞=⇒g∗ →f.Consequently,from(15) is determined for α = 0.5 (45) and is shown in Figure 10. and (16), gî → fi whenever ll → 1 and lu → 1 explains the This surface has a cropped rotated cone like shape, which is asymptotic of Figure 6 for any pair (f0,f1). symmetric about its main diagonal, i.e., with respect to the Based on simulation results the following are conjectured: plane (cid:15) = (cid:15) on the space ((cid:15) ,(cid:15) ,a). Notice that except 0 1 0 1 • For a fixed (cid:15)0 and (cid:15)1, increasing α leads to a monotone for the points on the cone like shape that intersect with the decreaseinlu andmonotoneincreaseinll onR+\{0,1}. ((cid:15)0,(cid:15)1,a=0)plane,allotherpointson((cid:15)0,(cid:15)1,a=0)thatare • For a fixed α, increasing (cid:15)0, (cid:15)1 or both introduces a non- plotted in blue color are un-defined (rather than being valid decrease in lu, non-increase in ll, or both, given that pointswitha=0),implyingthatforthosepointsnominimax (cid:15)0 and (cid:15)1 are less than their allowable maximum, cf. robust test exists. Section III-C. The proof of these conjectures is an open problem. V. CONCLUSION From (19) and (20), it is clear that given a pair ((cid:15) ,(cid:15) ), 0 1 a slight change in α changes the equations completely and A robust version of the likelihood ratio test considering in general l and l are functions of α. In Figure 7, the α−divergence as the distance to characterize the uncertainty l u robust decision rule δˆfor various α values is plotted, without sets has been proposed. The existence of a saddle value to the considering the dependency of l and l on α. To do this, minimax optimization problem was shown by adopting Sion’s l u l ≈ 0.605 and l ≈ 1.618, that are found for ρ = 1, α = 4, minimaxtheorem.Theleastfavorabledistributions,therobust l u (cid:15) = 0.02 and (cid:15) = 0.03, are fixed constants in (17). Then, decision rule as well as the robust version of the likelihood 0 0 9 (cid:96) (cid:96) fi,gi l, l 0.20 f0 f g(cid:96) g(cid:96)(cid:42) (cid:56) (cid:60) f f1 g(cid:96)1 g(cid:96)1(cid:42) 5 1 0 0 0 (cid:57) (cid:61) (cid:96) 0.15 g 0 4 (cid:96) g 1 0.10 3 2 0.05 1 y (cid:45)6 (cid:45)4 (cid:45)2 0 2 4 6 Fig.3. Nominaldensitiesandthecorrespondingleastfavorabledensitiesfor y ρ=1,α=4,(cid:15)0=0.02and(cid:15)1=0.03. (cid:45)4 (cid:45)2 2 4 (cid:96) fi,gi Fig.5. Nominalandleastfavorablelikelihoodratios(gˆ1/gˆ0 forρ=1and gˆ1∗/gˆ0∗ forρ=1.2)forα=4,(cid:15)0=0.02and(cid:15)1=0.03. f 0.20 0 l,l l u (cid:56) (cid:60) f 1 l (cid:96) l 0.15 g0 1(cid:56).6 (cid:60) lu (cid:96) g 1 1.4 0.10 1.2 0.05 1.0 y (cid:45)6 (cid:45)4 (cid:45)2 0 2 4 6 0.8 Fig.4. Nominaldensitiesandthecorrespondingleastfavorabledensitiesfor ρ=1.2,α=4,(cid:15)0=0.02and(cid:15)1=0.03. 0.6 Α 50 100 150 200 ratio test were derived in two parameters and in three distinct Fig. 6. Lower and upper thresholds, ll and lu, for a variable α, ρ = 1, regions on the co-domain of the nominal likelihood ratio. (cid:15)0=0.02and(cid:15)1=0.03. Two equations from where the parameters can be determined were also derived. It was found that the robust likelihood in ratio doesn’t depend on the parameter α that characterizes the distance between the probability measures. When the nominal Φ =(cid:18)c1α−1+(λ1−1+µ1+α−αµ1)/λ0(cid:19)α−11 . density functions satisfy a symmetry constraint, the two non- 1 (λ /λ )+(l/ρ)α−1 1 0 linearequationswerecombinedintoasingleequation.Finally, • Multiplying the numerator and the denominator of the the upper bounds on the parameters that control the degree result of the previous step by λ /λ , replacing the term of robustness were derived. Open problems include proving 0 1 1−1/λ +µ /λ +α/λ −αµ /λ by cα−1 and again the monotonicity of the parameters l and l for increasing 1 1 1 1 1 1 3 l u multiplying both the numerator and the denominator by ((cid:15) ,(cid:15) ), or α. It was shown that simulation results illustrate 0 1 λ gives the theoretical results. 1 (cid:18)λ c α−1+λ c α−1(cid:19)α−11 Φ = 0 1 1 3 . APPENDIXA 1 λ1+λ0(l/ρ)α−1 SIMPLIFICATIONOFΦ1 • The result of the previous step is free of parameters µ0 From (31) consider the following steps for andµ1,butstillparameterizedbyλ0andλ1.Toeliminate them, using the identities λ = (1−α)/(cα−1 −cα−1) 0 1 2 Φ =(cid:18)−1+λ0+λ1+µ0+µ1−α(−1+µ0+µ1)(cid:19)α−11 and λ1 =(1−α)/(cα4−1−c3α−1) leads to 1 λ +λ (l/ρ)α−1 1 0 (cid:32) (cid:33) 1 (c c )α−1+(c c )α−1 α−1 • Dividing the numerator and the denominator by λ0 and Φ1 = c α−1−c α−114+(c α−12−3c α−1)(l/ρ)α−1 . replacing the term 1+µ /λ −αµ /λ by cα−1 results 1 2 4 3 0 0 0 0 1 10 ∆ y Α(cid:61)100 (cid:72) (cid:76) Α(cid:61)10 1.0 0.8 Α(cid:61)0.01 0.6 a0.5 0.4 0.2 0.0 y 0 4 (cid:45)2 2 4 1 3 2 2 F(cid:15)1ig=.70..0T3h.edecisionruleδˆforα={0.01,10,100},ρ=1,(cid:15)0=0.02and Ε0 3 40 1 Ε1 (cid:96) f,g i i Fig. 10. All allowable pairs of maximum robustness parameters, ((cid:15)0,(cid:15)1), f0 w.r.t.alldistancesa∈[0,1]forα=0.5. f1 (cid:56) (cid:60) g(cid:96) Α(cid:61)10 0.3 and c4 =k(ll,lu)c3 yields 0 g(cid:96)1Α(cid:61)10 Φ1(l,ll,lu,c3;α,ρ)= g(cid:96)0(cid:72)Α(cid:61)0.0(cid:76)1 0.2 c (cid:18) k(ll,lu)α−1(llα−1−luα−1) (cid:19)α−11 . g(cid:96)1(cid:72)Α(cid:61)0.0(cid:76)1 3 llα−1−(k(ll,lu)lu)α−1+(k(ll,lu)α−1−1)(l/ρ)α−1 (cid:96) (cid:72) (cid:76) (51) g Α(cid:61)(cid:45)10 0 0.1 (cid:96) (cid:72) (cid:76) g Α(cid:61)(cid:45)10 1 APPENDIXB (cid:72) (cid:76) SIMPLIFICATIONOFδˆ y (cid:45)(cid:72)4 (cid:76) (cid:45)2 2 4 Since the equivalence of integration domains are given by Fig.8. Nominaldensitiesandthecorrespondingleastfavorabledensitiesfor (14), only ρ=1,(cid:15)0=0.1and(cid:15)1=0.1. λ (−1+α+λ +µ −αµ ) λ (λ +µ −αµ )(l/ρ)1−α δˆ= 0 1 1 1 − 1 0 0 0 , ˆl=ρ (−1+α)(λ +λ (l/ρ)1−α) (−1+α)(λ +λ (l/ρ)1−α) 0 1 0 1 • Theresultfromthepreviousstepdependsonlyonc1,c2, isrequiredtobesimplified.Inthefollowing,thesimplification c ,c andα.Usingthesubstitutionsc =c l ,c =c l is performed in three steps and the domain term ˆl = ρ is 3 4 1 3 l 2 4 u omitted for the sake of simplicity: • Dividing the numerator and the denominator of the first term by λ and the second term by λ , and replacing the 1 0 P ,P related terms by cα−1 and cα−1 results in F M 1 3 λ cα−1 λ cα−1(l/ρ)1−α δˆ= 0 · 3 − 1 · 1 (cid:56)0.1 (cid:60) −1+α λ0 +(l/ρ)1−α −1+α 1+ λ1(l/ρ)1−α λ1 λ0 cα−1−cα−1(l/ρ)1−α = 3 1 . (cid:16) (cid:17) (−1+α) 1 + 1 (l/ρ)1−α λ1 λ0 0.001 Nominal P F • The result of the previous step is free of parameters µ0 Nominal PM andµ ,butstillparameterizedbyλ andλ .Toeliminate 1 0 1 Ε0(cid:61)0.02,(cid:72)Ε1(cid:61)(cid:76)0.03, PF them, using the identities λ0 = (1−α)/(c1α−1 −cα2−1) 10(cid:45)5 Ε0(cid:61)0.02,(cid:72)Ε1(cid:61)(cid:76)0.03, PM and λ1 =(1−α)/(cα4−1−c3α−1) leads to (cid:72)Ε (cid:61)0.3,Ε (cid:61)0.2, (cid:76)P(cid:72) (cid:76) 0 1 F (l/ρ)1−αcα−1−cα−1 (cid:72)Ε0(cid:61)0.3,Ε1(cid:61)0.2, (cid:76)PM(cid:72) (cid:76) δˆ= 1 3 . SNR cα−1−cα−1+(cα−1−cα−1)(l/ρ)1−α (cid:72) 0 (cid:76) 5(cid:72) (cid:76) 10 15 4 3 1 2 Fig.9. Fa(cid:72)lsealarmand(cid:76)m(cid:72)iss(cid:76)detectionprobabilitiesofδ,(2),(ρ=1)for • Theresultfromthepreviousstepdependsonlyonc1,c2, (f0,f1) compared to that of the robust decision rule δˆfor (gˆ0,gˆ1) when c3,c4 andα.Usingthesubstitutionsc1 =c3ll,c2 =c4lu SNRisvaried.

Robust Hypothesis Testing with $\alpha$-Divergence

Similar Robust Hypothesis Testing with $\alpha$-Divergence