Table Of ContentBootstrap Testing of the Rank of a Matrix via Least Squared
Constrained Estimation.
Franc¸ois Portier and Bernard Delyon
Abstract In order to test if an unknown matrix has a given rank (null hypothesis), we
consider the family of statistics that are minimum squared distances between an estimator
and the manifold of fixed-rank matrix. Under the null hypothesis, every statistic of this
3 family converges to a weighted chi-squared distribution. In this paper, we introduce the
1
constrained bootstrap to build bootstrap estimate of the law under the null hypothesis of
0
such statistics. As a result, the constrained bootstrap is employed to estimate the quantile
2
for testing the rank. We provide the consistency of the procedure and the simulations
n
shed light one the accuracy of the constrained bootstrap with respect to the traditional
a
J asymptotic comparison. More generally, the results are extended to test if an unknown
4 parameter belongs to a sub-manifold locally smooth. Finally, the constrained bootstrap is
easy to compute, it handles a large family of tests and it works under mild assumptions.
]
T
Keywords. Rankestimation,Leastsquaredconstrainedestimation,Bootstrap,Hypothesis
S
testing.
.
h
t
a
1 Introduction
m
[
Let M Rp×H be an unknown matrix (arbitrarily p H). To infer about the rank of M
0 0
1 ∈ ≤
with hypothesis testing, the general framework usually considered is the following: there exists
v
8 an estimator M Rp×H of M0 such that
∈
6
7 c n1/2(M M ) d W, with vec(W) = (0,Γ) (1)
0 − 0 −→ N
.
1
where vec() vectorizes acmatrix by stacking its columns. In the whole paper the hatted quan-
0 ·
3 tities are random sequences that depends on the sample number n, all the limit are taken with
1 respect to n. Moreover there exists an estimator Γ such that
:
v
P
i Γ bΓ, (2)
X
−→
r
a and in some cases, one may ask that b
Γ is full rank. (3)
Let d be the rank of M and m 1,...,p , we consider the set of hypotheses
0 0
∈ { }
H : d = m against H : d > m, (4)
0 0 1 0
Thus d can be estimated the following way: we start by testing m = 0, if H is rejected we go
0 0
a step further m := m+1, if not we stop the procedure and the estimated rank is d = m. In
this paper, by considering the hypotheses (4) we focus on each step of this procedure.
Many different statistical tests appeared in the literature for this purpose. Forb instance
Cragg and Donald [11] introduced a statistic based on the LU decomposition of M, Kleibergen
c
1
and Paap [19] studied the asymptotic behaviour of some transformation of the singular values
of M, and Cragg and Donald [12] considered the minimum of a squared distance under rank
constraint. In some other fields with similar issues, close ideas have been developed : Bura and
Yancg [7] examined a Wald type statistic depending on the singular decomposition of M and
Cook and Ni [10] also considered the minimum of a squared distance under rank constraint.
Althoughbasedondifferentconsiderations,eachofthepreviousworkreliesonthetestdesccribed
by (4). For comprehensiveness, in this paper we consider the following three statistics. The
first one is introduced by Li [21] as
p
Λ = n λ2 (5)
1 k
k=m+1
X
b b
where (λ ,...,λ ) are the singular values of M arranged in descending order. Under H and (1),
1 p 0
this statistic converges in law to a weighted chi-squared distribution [7]. The main drawback
of suchba testbis that Λ is not pivotal, i.e.cits asymptotic law depends on unknown quantities
1
that are M and Γ. Accordingly the consistency of the associated test requires assumptions (1)
0
and (2). In [7] a standbardized version of Λ is studied with
1
Λ1 = nvec(Q1MQ2)T[(Qb2 Q1)Γ(Q2 Q1)]+vec(Q1MQ2) (6)
⊗ ⊗
where M+ standsbfor the Mboorce-Pbenrosbe invebrsebofbM abnd Q andb Qcbare respectively the
1 2
orthogonal projectors on the left and right singular spaces associated with the p m smallest
−
singular values of M. The authors proved that under H , bif (1) anbd (2) hold, the Wald-
0
type statistic Λ is asymptotically chi-squared distributed. Besides, [12] and [10] proposed a
2
constrained estimatcor by minimizing a squared distance under a fixed-rank constraint as
b
Λ = n min vec(M M)TΓ−1vec(M M), (7)
3
rank(M)=m − −
b c b c
which is also asymptotically chi-squared distributed under H , assuming (1), (2) and (3). We
0
will refer the minimum discrepancy approach. Although the statistics Λ and Λ have the
2 3
convenience of being pivotal, they both require the inversion of a large matrix and this may
cause robustness problems when the sample number is not large enoughb. For αb ]0,1[ and
∈
under the relevant assumptions, each of these statistics Λ , Λ and Λ , is consistent at level α
1 2 3
in testing (4), i.e. the level goes to 1 α and the power goes to 1 as n goes to .
− ∞
Nevertheless the estimation of the quantile is difficultbbecbause eitbher the asymptotic distri-
bution depends on the data (non pivotality represented by Λ ), or the true distribution may
1
be quite different than the asymptotic one (slow rates of convergence represented by Λ and
2
Λ ). The objective of the paper is to propose a bootstrap mbethod for quantile estimation in
3
this context. b
b An important remark which instigates the sketch of the paper is that all the previous statis-
tics share the form
Λ = n Bvec(M M ) 2 with M = argmin Avec(M M) 2 (8)
c c
k − k k − k
rank(M)=m
b b c c c b c
where is the Euclidean norm, A RpH×pH, B RpH×pH. The values of A and B
k · k ∈ ∈
corresponding to the statistics Λ , Λ and Λ are summarized in the Table 1 (See Section 2 for
1 2 3
the details). b b b b
We refer to traditional testibng (bresp. bbootstrap testing) when the statistic is compared to
its asymptotic quantile (resp. bootstrap quantile). The bootstrap test is said to be consistent
2
Λ Λ Λ
1 2 3
A I I Γ−1/2
b b b
Bb I [(Q Q )Γ(Q Q )]+1/2 Γb−1/2
2 1 2 1
⊗ ⊗
b b b b b b b
Table 1: Values of A and B in (8) for Λ , Λ and Λ .
1 2 3
b b b b b
at level α if
P Λ> q(α) 1 α and P Λ > q(α) 1, (9)
H0 −→ − H1 −→
(cid:16) (cid:17) (cid:16) (cid:17)
whereq(α)isthequantbileobflevelαcalculatedbybootstrap. Tbheadbvantageofbootstraptesting
isits highlevel ofaccuracy underH withrespecttotraditionaltesting. Thisfactis emphasized
0
by conbsidering the two possibilities: when the statistic is pivotal and when the asymptotic law
of the statistic depends on unknown quantities. First, as highlighted by Hall [15], when the
statistic is pivotal, under some conditions the gap between the distribution of the statistic and
its bootstrap distribution is OP(n−1). Since the normal approximation leads to a difference
O(n1/2), the bootstrap enjoys a better level of accuracy. Secondly if the asymptotic law of the
statistic is unknown, the bootstrap appears even more as a convenient alternative because it
avoids its estimation. In [17], Hall and Wilson give two advices for the use of the bootstrap
testing:
A) Whatever the sample is under H or H , the bootstrap estimates the law of the statistic
0 1
under H .
0
B) The statistic is pivotal.
The first guideline is the most crucial because if it fails it may lead to inconsistency of the
test. The second guideline aims at improving the accuracy of the test by taking full advantage
of the accuracy of the bootstrap. In this paper we propose a new procedure for bootstrap
testing in least square constraint estimation (LSCE) (estimators as (8) are particular cases),
called constrained bootstrap (CS bootstrap). More precisely, the CS bootstrap aims at testing
whether a parameter belongs or not to a submanifold and so generalised the test (4). Our
main result is the consistency of the CS bootstrap under mild conditions. As a consequence
we provide a consistent bootstrap testing procedure for testing (4) with the statistic Λ , Λ
1 2
and Λ . For the sake of clarity, we address the CS bootstrap in the next section. Section 3 is
3
dedicated to rank estimation with special interest to the bootstrap of the statistic Λ , Λb anbd
1 2
Λ . bFinally, the last section emphasizes the accuracy of the bootstrap in rank estimation by
3
providing a simulation study in sufficient dimension reduction (SDR). Accordingly,bthebsketch
obf the paper is as follows:
The CS bootstrap in LSCE
•
Bootstrap testing procedure for Λ , Λ and Λ
1 2 3
•
Application to SDR
•
b b b
2 The constrained bootstrap for LSCE and hypothesis testing
Because of (8) LSCE has a central place in the paper. Moreover since LSCE intervenes in many
statistical fields as M-estimation or hypothesis testing, this section is independentfrom the rest
of the paper.
3
2.1 LSCE
Let θ Rp be called the parameter of interest, and let θ Rp be an estimator of θ . We define
0 0
∈ ∈
the constrained estimator of θ as
0
b
θ = argmin (θ θ)TA(θ θ), (10)
c
− −
θ∈M
b b b b
where is a submanifold of Rp with co-dimension q, and A Rp×p. The constrained statistic
M ∈
is defined as
b
Λ= n(θ θ )TB(θ θ ). (11)
c c
− −
where B Rp×p. Note that if A is fbull ranbk, thbe unbiqbue mbinimizer of (10) without constraint is
∈
θ, hence it could be understood as the unconstrained estimator. We introduce now the notion
of nonsbingular point in . Thbis one is needed to express the Lagrangian first order condition
M
obf the optimization (10). For any function g = (g ,...,g ) : Rp Rq, define its Jacobian as
1 p
→
J = ( g ,..., g ), where stands for the gradient operator.
g 1 q
∇ ∇ ∇
Definition 1. We say that θ is -nonsingular if θ and if there exists a neighbourhood
M ∈ M
V and a function g :Rp Rq continuously differentiable on V with J (θ) full rank such that
g
→
V = g = 0 .
∩M { }
As a consequence any point of a submanifold locally smooth is nonsingular, e.g. any matrix
with rank m is a nonsingular point in the submanifold rank(M) = m. We prove in Proposition
d P
2 that if θ is -nonsingular, √n(θ θ ) (0,∆) and B = A A is full rank, then we have
0 0
M − → N →
p
b Λ d ν W2, b b (12)
−→ k k
k=1
X
b
where the W ’s are i.i.d. Gaussian random variables and the ν ’s are the eigenvalues of the
k k
matrix ∆1/2J (θ )T(J (θ )A−1J (θ )T)−1J (θ )∆1/2. Especially, the case A = ∆−1 is interest-
g 0 g 0 g 0 g 0
ing because Λ is asymptotically chi-squared distributed with q degrees of freedom. Otherwise,
if θ / , Λ goes to infinity in probability. Those facts shed light on a consistent testing
0
∈ M
procedure babsed on LSCE with the hypotheses
b
H : θ against H : θ / (13)
0 0 1 0
∈ M ∈M
andthedecisionruletorejectH ifΛislargerthanaquantileofitsasymptoticlaw. Accordingly
0
the previous framework can be seen as an extension of the Wald test statistic which handles
the simple hypothesis θ = θ with tbhe statistic (θ θ)T∆−1(θ θ).
0
− −
2.2 The bootstrap in LSCE b b
Since LSCE is a particular case of estimating equation, we review the bootstrap literature with
two principal directions: estimating equation and hypothesis testing. For clarity we alleviate
the framework in this section: let X , ,X be an i.i.d. sequence of real random variables
1 n
···
with law P, define γ = var(X ), γ = (X X)2, we put θ = E[X ], θ = X, and A = B = γ−1
1 0 1
−
where stands for the empirical mean.
·
The original bootstrap was ibntroduced in [14] in the following bway. Let X1∗,...,Xn∗ be
an i.i.d. sequence of real random variables with law P = n−1 n δ , define θ∗ = X∗, the
i=1 Xi
P
b
4
distribution of √n(θ∗ θ) conditionally on the sample, that we call the bootstrap distribution,
−
is “close” to the distribution of √n(θ θ ), that we call the true distribution (in the rest of the
0
−
paper we just say “condbitionally” instead of “conditionally on the sample”). For instance, it is
shown in [23] that the bootstrap disbtribution converges weakly to the true distribution almost
surely. One says that √n(θ∗ θ) bootstraps √n(θ θ ) and we will write
0
− −
(n1/2(θ∗ θ)P)= (n1/2(θ θ )) a.s.,
∞ b ∞b 0
L − | L −
where ∞() and ∞( P) both mean bthebasymptotic lawbs with the difference that the later is
conditiLonal·on theLsam·|ple. Equivalently, one has for every x R, P(√n(θ∗ θ) x P) a.s.
∈ − ≤ | →
P(√n(θ θ ) x), butbthe use of the bootstrap is legitimate by a more general results stated
0
− ≤
in [15], which says that b b
b
P(n1/2(θ∗ θ)/γ∗ x P) P(n1/2(θ θ0)/γ x) = OP(n−1) (14)
| − ≤ | − − ≤ |
with γ∗ = (X∗ X∗)2, providbed that Pbis non-latticbe. Besidbes, one has
−
P(n1/2(θ θ0)/γ x) Φ(x) =OP(n−1/2),
| − ≤ − |
where Φ is the cumulative distribution function (c.d.f.) of the standard normal law. Variations
b b
of Efron’s resampling plan are proposed in [2] under the name of weighted bootstrap. For a
complete introduction about the bootstrap we refer to [15]. We now present three different
bootstrap techniques related to LSCE1.
(i) The classical bootstrap (C bootstrap)
The literature about the bootstrap in Z and M-estimation, see respectively [9] and [1],
is based on the following principle: if θ = argmin E[φ(X,θ)] is estimated by θ =
M M
θ∈Θ
argmin 1 n φ(X ,θ) where Θ is an open set, then the bootstrap of √n(θ θ ) is
n i=1 i M − bM
θ∈Θ
carried ouPt by the quantity √n(θ∗ θ ) with
M − M b
n
θ∗ = argmbin n−1 w φ(X ,θ), (15)
M i i
θ∈Θ
i=1
X
where (w ) is a sequence of random variables. The particular case where the vector
i
(w ,...,w ) is distributed as mult(n,(n−1,...,n−1)) leads to a direct application of orig-
1 n
inal Efron’s bootstrap to M-estimation. Since such a bootstrap has been extensively stud-
ied,werefertotheCbootstrap. Totheknowledgeoftheauthors,theCbootstrapwhenΘ
has empty interior has not been studied yet. Nevertheless one may sight its bad behaviour
for the test of equal mean H : θ = µ. The associated least squared constrained statistic
0 0
nγ−1(θ µ)2,
−
is indeed the score statistic associated to the M-estimator with φ(x,θ) = γ−1(x θ)2 and
b b −
Θ = µ . Clearly the C bootstrap through nγ∗−1(θ∗ µ)2 does not work because of its
{ } −
bad behaviour under H1 for instance. In this case it is better to use b
nγ∗−1(θ∗ θ)2,
−
1Abootstrap withaDelta-methodapproach(see[23],chapter23,Theorem5)failsbecausex→ minkx−θk
b
kθk=1
is not continuously differentiable on theunit circle.
5
but it cannot handle the cases of more involved hypotheses2. Whereas the C bootstrap is
not really connected with hypothesis testing, the two following bootstrap procedures are
more related to the present work.
(ii) The biaised bootstrap (B bootstrap)
TheBbootstrap is introduced in [16] and is directly motivated by hypothesis testing. The
originalideaoftheirworkistore-samplewithrespecttothedistributionP = n ω δ ,
b i=1 i Xi
where the ω ’s maximize
i
P
b
n 1 n ω X = µ
log(ω ) under the constraints n i=1 i i . (16)
i n ω = 1
i=1 P i=1 i
X
P
Since the ω ’s minimize the Kulback-Leibler distance between P and P , one can see the
i b
resulting distribution as the closest to the original one satisfying the mean constraint.
The authors presented interesting results for the test of equal meban θ =b µ, essentially the
0
bootstrap statistic nγ∗−1(θ∗ µ)2, with θ∗ = X∗, X∗ sampled from P , has a chi-squared
b − b b b,i b
limiting distribution either H or H is assumed. As a result both guidelines (A) and (B)
0 1
are checked. They go further by showing that the B bootstrap outclabsses the asymptotic
normal approximation for quantile estimation in the sense that q(α) qn(α) = OP(n−1)
| − |
whereas q (α) q (α) = O(n−1/2), whereq , q andq arethequantile functionsof the
n ∞ ∞ n n
| − |
standard normal distribution, the statistic nγ−1(θ µ)2 under Hb0 and the bootstrapped
−
statistic,respectively. AlthoughtheBbootstrapmatchebsthecontextofhypothesistesting,
it has been designed to handle the particularbtestbof equal mean. To the knowledge of the
authors the study of the B bootstrap has not been extended to other tests. Facing (16),
themain drawback of the B bootstrap deals with algorithmic difficulties. Indeed whenthe
constraint becomes more involved, solving (16) is more difficult. As a result it is not sure
that this method could handle other situations such as fixed-rank constraints.
(iii) The estimating function bootstrap (EF bootstrap)
Now X Rp. Some other ideas about the bootstrap of the Z-estimators can be found
i
∈
in [20] and [18], and can be summarized as follows. Considering the score statistic S =
√n n ∂φ(X ,θ ), [18] showed that it could be bootstrapped by
i=1 ∂θ i 0
b
P n
∂φ
S∗ = n−1/2 w (X ,θ),
i i
∂θ
i=1
X
b
where (w ) is a sequence of random variables. This bootstrap is called the EF bootstrap
i
and revealed nice computational properties. Moreover the authors argued for its use in
quantile estimation in order to test if g(θ )= 0, where g : Rp Rq is the constraint func-
0
→
−1
tion,byrecommendingessentially touseS∗TJ (θ)T J (θ)γ∗J (θ)T J (θ)S∗. Applying
g g g g
it to the least squared context φ(x,θ)= γ−1/2(x (cid:16)θ) 2, the EF b(cid:17)ootstrap is carried out
k b − kb b b
by
b
−1
n(θ∗ θ)TJ (θ)T J (θ)γ∗J (θ)T J (θ)(θ∗ θ).
g g g g
− −
(cid:16) (cid:17)
Although it verifies both bguidelibnes (A)band (Bb) (see thebarticlebfor details), one can
see that the good behaviour of such an approach is more based on the rank deficiency
2We refer to [17] for a study of this bootstrap in order to test θ0 =µ.
6
of J (θ) than on the bootstrap of √n(θ θ ). Indeed √n(θ∗ θ) bootstraps the non
g c
− −
constrained estimator √n(θ θ ). Thenas theauthors noticed, it is firstof all a bootstrap
0
−
of thebWald-type statistic nSTJ (θ )T Jb (θb)γJ (θ )T −1J (θ )Sbwhich has fortunately
g 0 g 0 g 0 g 0
the same asymptotic law tbhan the targeted one. This may induce some loss in accuracy.
(cid:0) (cid:1)
Moreover, it requires the knobwledge of the funcbtion Jg which is nobt the case for fixed rank
constraints where the g depends on the limit M (see Remark 1 for some details).
0
Essentially both (i) and (ii) provide a bootstrap for testing simple hypotheses. The EF
bootstrap proposed in (iii) extends this limited scope by including tests of the form g(θ ) = 0
0
where g is known. Nevertheless it does not handle the test (4) as it is highlighted by the
following remark.
Remark 1. Testing (4) with Λ results in an optimization with the constraint rank(M) = m.
3
Since the subspace of fixed rank matrices is a submanifold locally smooth with co-dimension
(p d)(H d), at every pointbM, there exists a neighbourhood V and a ∞ function g : V
− − C →
R(p−d)(H−d) such that V rank(M) = m = g = 0 and J (M) has full rank. Moreover, we
g
∩{ } { }
have
Γ−1/2vec(M M ) 2 Γ−1/2vec(M M ) .
c 0 0
k − k ≤ k − k
P
If now (1) holds, the right-hand cside term goes to 0 in prcobability and M M . As a
c 0
→
consequence, if Γ is invertible, for any neighbourhood of M , from a certain rank, M belongs
0 c
to it with probability 1. Then under H since M has rank m the constraicned estimator has
0 0
the expression c
M = argmin Γ−1/2vec(M M) ,
c c
k − k
g(M)=0
c c
with g depending on M . Unfortunately we do not know neither g nor J (M ). This entails
0 g 0
some problems relating to the later approach.
2.3 The constrained bootstrap
The CS bootstrap is introduced in order to solve all the issues we have raised through the
previous little review which are essentially: computational difficulties and small scope of the
existing methods. The CS bootstrap targets an estimation q(α) of the quantile under H of
0
Λ. The consistency of the procedure, i.e. (9), forms the main result about the CS bootstrap.
Another important issue which occurs beforehand in the sectibon is the bootstrap of the law of
b
n1/2(θ θ ) under H .
c 0 0
−
Basically, weshowthatabootstrapofthebunconstrainedestimator√n(θ θ )allowsabootstrap
0
−
oftheconstrainedestimator√n(θ θ )underH . WepointoutthattheCSbootstrapheuristic
c 0 0
−
is rather different than the C and EF bootstrap. Otherwise it sharesbthe idea to “reproduce”
H even if H is realized with thbe B bootstrap. Assuming that we can bootstrap √n(θ θ ),
0 1 0
−
the CS bootstrap calculation of the statistic is realized as follows:
b
7
The CS bootstrap procedure
Compute
θ∗ = θ +n−1/2W∗, with (W∗ P) = (n1/2(θ θ )) a.s., (17)
0 c L∞ | L∞ − 0
where the simulatbion of W∗ can be done by a standabrd bootstrap pbrocedure3. Calculate
θ∗ = argmin (θ∗ θ)TA∗(θ∗ θ), and Λ∗ = n(θ∗ θ∗)TB∗(θ∗ θ∗), (18)
c 0 − 0 − 0 − c 0 − c
θ∈M
where A∗ Rp×p and B∗ Rp×p 4.
∈ ∈
Intuitively, this choice appears natural because θ∗ equals θ plus a small perturbation going
0 c
to 0. Accordingly θ∗ is somewhat reproducing the behaviour of θ under H , especially because
0 0
W∗ hastherightasymptoticvariance. Asweshouldnotice, A∗bandB∗ couldbechosenasAand
B but this is not the best choice in practice. As it is highlightedbin (14), we should normalized
by the associated bootstrap quantities (e.g. the variance computed on the bootstrap sambple).
Tbhe following lemma gives a first order decomposition of the bootstrap law √n(θ∗ θ ) under
c − c
mild conditions. The following lemma is proved in the Appendix.
b
Lemma 1. Let be a submanifold. Assume there exists θ and θ a -nonsingular
c c
point such that θMa.s. θ . If moreover (√n(θ∗ θ )P) exi∈stsMa.s. and coMnditionally a.s.
c → c L∞ 0 − c |
A∗ P A is full rank, then we have conditionally a.s. b
→ b b b
n1/2(θc∗−θc)= (I −P)n1/2(θ0∗−θc)+oP(1),
with P = A−1JgT(θc)(Jg(θc)A−1JgT(bθc))−1Jg(θc). b
Note that if θ is -nonsingular and (√n(θ θ )P) exists, we can apply Lemma 1 with
0 ∞ 0
M L − |
θ = θ = θ . This gives the following proposition:
c c 0
b b
Proposition 2. Let be a submanifold. Assume that (√n(θ θ )P) exists with θ -
b ∞ 0 0
M P L − | M
nonsingular. Assume also that A A is full rank, then we have
→ b b
n1/2(θcb θ0)= (I P)n1/2(θ θ0)+oP(1),
− − −
with P = A−1JgT(θ0)(Jg(θ0)A−b1JgT(θ0))−1Jg(θ0). b
Proposition 2 leads easily to (12) and extends classical results [6] about constrained esti-
mators with constraint g = 0 to manifold type constraints. Besides statements of Lemma 1
{ }
and Proposition 2 together explain the preceding definition of θ∗ in (17). They also lead to the
0
following theorem.
a.s. P
Theorem 3. Let be a submanifold. Assume that θ θ with θ -nonsingular and A A
0 0
M →P M →
hold. If moreover (17) holds and conditionally a.s. A∗ A is full rank, then we have
→
b b
(n1/2(θ∗ θ )P) = (n1/2(θ θ )) a.s..
L∞ c − c | L∞ c− 0
3The bootstrap procedure to get W∗ is nbot sbpecified because bit depends on θb. For instance, if θbis a mean
b
over some i.i.d. random variables, one can use the Efron’s traditional bootstrap and if θ is a M-estimator, one
should usea bootstrap as detailed by equation (15).
4Assumptions about A∗ and B∗ are provided furtherin thestatements of the propositions.
8
Essentially, Theorem3isanapplicationofLemma1underH ,indeedasweseenintheproof
0
a.s. a.s.
of Lemma 1, equation (24), the assumption θ θ implies that θ θ . Nevertheless
0 c c
→ ∈ M →
under H nothing guarantee such a convergence (see Example 1 below). Roughly speaking,
1
asking for an equality in law under H as inbTheorem 3 may be too mbuch to ask. However
1
as stated in the following theorem we do not require that θ converges a.s. to a constant to
c
provide that the power of the corresponding test goes to 1. This leads to the consistency of the
CS bootstrap for hypothesis testing. For the statement of thbe consistency theorem, we need to
define the quantile function of the bootstrap statistic
q(α) = inf x : F(x) 1 α ,
{ ≥ − }
where F is the c.d.f. of Λ∗ condbitionally on the sbample.
a.s.
Theorem 4. Let be a manifold. Assume that θ θ with θ -nonsingular under H . We
b MP P → 0 0 M 0
assume also that A A is full rank, B B. If moreover (√n(θ∗ θ )P)= (√n(θ θ ))
→ → P b P L∞ 0− c | L∞ − 0
a.s. has a density, and conditionally a.s. A∗ A, B∗ B, then we have
b b → → b b b
P (Λ > q(α)) 1 α, and P (Λ > q(α)) 1.
H0 −→ − H1 −→
Inother words, the test dbescribbed in(13)with statistic ΛandbCSbbootstrap calculation of quantile
is consistent.
b
We provide the following example under H , where θ does not converge to a constant in
1 c
probability. Although we cannot get theconclusion of Theorem 3, theleast squaredconstrained
statistic still converges in distribution. b
d
Example 1. Let (Xi)i∈N be a i.i.d. sequence such that X1 = (0,1). Define θ = X, and
N
H : θ2 = 1. Clearly H does not hold and naturally the statistic n min θ θ 2 goes to infinity
0 0 0 θ2=1k − k b
in probability. One can find that θ = sign(X) which does not converge. Since
c b
θ∗ = argmin θ∗ θ 2 and θ∗ = θ +n−1/2W∗,
c bk 0 − k 0 c
θ2=1
we get that θ∗ = θ a.s. and naturally, we do not have tbhe asymptotic given by Theorem 3.
c c
Besides, the convergence to a chi-squared distribution holds for the quantity n min θ∗ θ 2.
b θ2=1k 0 − k
3 Rank estimation with hypothesis testing
In this section through a review of the literature about rank estimation, we apply the results
obtained in section 2.1 to provide a consistent bootstrap procedurefor the test described by (4)
associated with the statistics Λ , Λ and Λ . We define q = p d the dimension of the kernel
1 2 3 0 0
−
of MT. We denote by (λ ,...,λ ) the singular values of M arranged in descending order and
0 1 p 0
we write the SVD of M as b b b
0
D 0 VT
M = (U U ) 1 1 ,
0 1 0 0 0 VT
(cid:18) (cid:19)(cid:18) 0 (cid:19)
with U Rp×d0, U Rp×q0, V RH×d0, V RH×q0, and D = diag(λ ,...,λ ). For
1 ∈ 0 ∈ 1 ∈ 0 ∈ 1 1 d0
m 1, ,p , we note q = p m and we write the SVD of M as
∈ { ··· } −
M =(U U ) D1 0 V1T c,
1 0 0 D VT
0! 0 !
b b
c b b
b b
9
with U Rp×m, U Rp×q, V RH×m, V RH×q, D = diag(λ ,...,λ ) and D =
1 0 1 0 1 1 m 0
∈ ∈ ∈ ∈
diag(λ ,...,λ ). We also introduce the orthogonal projectors
m+1 p
b b b b b b b b
Q1 =bI −P1 =bU0U0T, Q2 = I −P2 = V0V0T, Q1 = I −P1 = U0U0T and Q2 = I −P2 = V0V0T.
Whereas the link between Λ and LSCE is evidbent, the onbe conbecbting Λ abnd Λ tobLSCEbrbelies
3 1 2
on the following classical lemma, whose proof is avoided.
b b b
Lemma 5. Let M Rp×H, it holds that
∈
p
c
argmin M M 2 = P MP , and M P MP 2 = λ2,
k − kF 1 2 k − 1 2kF k
rank(M)=m
k=m+1
X
c b cb c b cb b
where λ ,...,λ are the singular values of M arranged in descending order, and P and P are
1 p 1 2
orthogonal right and left singular projectors of M associated with λ ,...,λ .
1 m
b b c b b
Note that in the previous lemma, P and P are uniquely determined if and only if λ =
1 c2 b b m
6
λ .
m+1
b b b
3b.1 Nonpivotal statistic
As stated in the introduction, the statistic Λ = n p λ2 can be used to arbitrate between
1 k=m+1 k
the hypotheses of (4). Basically, if H : d = m is realized, all the eigenvalues of the sum
0 0
P
goes to 0 and Λ has a weighted chi-squabred limiting disbtribution. Otherwise, at least one
1
eigenvalue converges in probability to a positive number and for any A > 0, P(Λ > A) 1.
1
−→
The following pbroposition describes the asymptotic behaviour of Λ 5. It was stated in [8] and
1
some recent extension can be found in [7]. Our statement goes further becaubse we are also
concerned about the estimation of the asymptotic law of Λ , i.e. tbhe estimation of the weights
1
that intervenes in the weighted chi-squared asymptotic law. Besides, the proof we give in the
Appendix is quite simple6. b
Proposition 6. Under H , if (1) holds we have
0
Λ d ν W2
1 −→ k k
X
where the ν ’s are the eigenvalues of thbe matrix (Q Q )Γ(Q Q ) and the W ’s are i.i.d.
k 2 1 2 1 k
⊗ ⊗
standard Gaussian variables. If moreover (2) holds, we have
P
(ν ,...,ν ) (ν ,...,ν ),
1 pH 1 pH
−→
where the ν ’s are the eigenvalues of the matrix (Q Q )Γ(Q Q ).
k b b 2 1 2 1
⊗ ⊗
Remark 2. Unlike Theorem 1 in [8] or Theorem 1 in [7], we prefer to state this theorem with
b b b b b b
the quantities Q and Q rather than with U and V . Because we do not assume that the
1 2 0 0
kernel of M has dimension 1, the vectors that form U or V are not unique because vector
0 0
spaces with dimension larger than 2 have an infinite number of basis. As a consequence it does
not make sense to estimate either U or V . To characterize convergence of spaces, a suitable
0 0
object is their associated orthogonal projectors.
5Asimilar proposition canbestatedapplyingProposition 12. Followingthisway,theasymptoticdependson
g which is difficult to estimate for rank constraints (see Remark 1).
6We nolonger need theresults of [13] about theasymptotic behaviour of singular values.
10
