Limits of Deterministic Compressed Sensing Considering Arbitrary Orthonormal Basis for Sparsity

0.08 MB·English

by  Arash Amini

Limits of Deterministic Compressed Sensing
Considering Arbitrary Orthonormal Basis for
Sparsity
Arash Amini(1) and FarokhMarvasti(1)
(1)AdvancedCommunicationResearchInstitute(ACRI)
EEDepartment,SharifUniversityofTechnology,Iran.
9
[email protected],[email protected]
0
0 Abstract: timeprobleminsteadofanNP-completeproblem)[3,10].
2 Since in most of the applications the input signal has a
Itispreviouslyshownthatproperrandomlinearsamples
n priory known characteristics, the sparsity domain is also
ofafinitediscretesignal(vector)whichhasasparserep-
a known. Thus, it is logicalto designa deterministicsam-
J resentationinanorthonormalbasismakeitpossible(with pling matrix (Φ); in this way, we eliminate the need to
probability 1) to recover the original signal. Moreover,
2
savethesamplingmatrix. Somepossiblematricesarein-
2 the choice of the linear samples does not depend on the
troducedin[6]and[11].
sparsity domain. In this paper, we will show that the re-
] placement of random linear samples with deterministic Althoughtheanalogbranchofthecompressedsensingis
T being developedparallel to the improvementsin the dis-
functionsofthesignal(notnecessarilylinear)willnotre-
I crete domain [8, 9], we will only consider the discrete
. sultinuniquereconstructionofk-sparsesignalsexceptfor
s case in this paper. We will investigate the consequences
c k = 1. We will show that there exist deterministic non-
[ ofdeterministicsamplingwhenthesparsitydomainisnot
linear sampling functionsfor uniquereconstructionof 1-

knownatthetimeofsampling;however,wedonotrestrict
1 sparsesignalswhiledeterministiclinearsamplesfailtodo
thesamplingfunctionstobelinear. Infactwearelooking
v so.
8 fordeterministicfunctionsofxwhichuniquelyrepresent
0 all k-sparse signals irrespective of their sparsity domain
4 1. Introduction (forreconstructionthedomainshouldbeknown).
3
1. Recent results in compressed sensing [1, 5, 4, 7] show 2. Preliminaries
0 that for a class of finite discrete signals named sparse,
9 it is possible to reconstruct the signal from significantly
Beforewestartthemainargument,itisnecessarytopoint
0
: lower number of samples than predicted by Nyquist rate out a key issue: since the cardinality of C (or R) is the
v in uniform sampling theory. More precisely, if xn×1 sameasCn (orRn),thereexistsaninjectiong :Cn →C
Xi is a vector whose representation in the orthonormal ba- (or g : Rn → R) which as a sampling function, com-
sis Ψ = [ψ ] has at most k nonzero elements ,
r i,j n×n presses the whole information of a vector into a single
a it is called k-sparse. Also s = ΨH · x is the transla- sample,evenwithouttheassumptionofsparsity.Although
tion of x in the sparsity domain(since Ψ is orthonormal
the compression is amazing, the output sample can not
Ψ−1 =ΨH);i.e.,shasatmostknonzeroelements.
be easily quantized; i.e., the theoretical solution is im-
ItisprovedthatifmisofordernotlessthanO(k·log(n)),
k practical. For this reason, we are looking for sampling
m random linear combination of the samples (multipli-
functions which employ the sparsity constraint in order
cation by a random matrix Φ) of a k-sparse signal with
to reconstruct the original signal and then we will dis-
length n (k ≪ n) where the coefficients of the linear
cuss how they can be quantized. To formulate the men-
sampleshavenormali.i.d.distribution,suffice(withover-
tioned constraint: we are lookingfor sampling functions
whelmingprobability)toreconstructthesignal[7]. Other f : Cn → C, 1 ≤ i ≤ mthatresultinequivalentsam-
i
distributions have also been proposed which yield the
plingclassesdefinedby:
same result [2]. One of the features of this type of ran-
domsamplingwhichwasoriginallyofmainconcern(but Ss1,...,sm ={x∈Cn | ∀1≤i≤m: fi(x)=si} (1)
nolongerasimportantasitwas)isthatatthetimeofsam- suchthatineacharbitraryorthonormalbasis,atmostone
plingitisunimportantinwhichbasisthesignalissparse. of the elements of S has k-sparse representation. More-
Thedomainofsparsitycomesintothepictureonlyatthe over, the sparsity constraint should play a key role; i.e.,
time of the reconstruction. Therefore, this type of sam- formanycasesweshouldhave|S|>1.
pling is not matched to a specific sparsity domain. The First we show whydeterministiclinearsamplingis nota
other advantage of the method lies in its reconstruction properchoice. Assumethesamplesareproducedas:
method. The mentioned order of m is the same as the
s f (x)
orderforwhichtheminimizationusingℓ normforfind- 1 1
ingthesparsesolutioncouldbereplaced0withℓ1 without  ... = ... =Φm×n·xn×1 (2)
considerablechangeinthesolution(solvingapolynomial  sm   fm(x) 
   
If m ≥ n, the number of samples exceeds the num- 1. If v = 0, bothx andx lie in the directioniden-
1 0 1
ber of original elements in x, which defeats the under- tified by the normal vector v˜ . Similar to the ar-
0
lying purpose of compression. For the case of m < n, gument in the previous section, there exist normal
rank(Φ)≤m<nandtherefore,thenullspaceofΦwill vectors u ,...,u such that {v˜ ,u ,...,u }
1 n−1 0 1 n−1
havenon-zerodimension: formsanorthonormalbasisforCn. Obviouslyboth
x and x are 1-sparse in this basis and moreover,
0 1
∃v∈Cn,kvkℓ2 =1: Φ·v=0 (3) they produce the same set of samples using the as-
sumed sampling functions. Hence, using the men-
LetV⊥ bethesubspaceofCn formedbythevectorsper-
tioned samples, these two vectorscan notbe distin-
pendiculartov:
guishedevenwiththesparsityconstraint.
V⊥ ,{x∈Cn | vH ·x=0}, (4) 2. If v 6= 0, we define v˜ to be the normalvector in
1 1
thedirectionofv . Sincev˜ andv areorthogonal,
and let {u1,...,un−1} be an orthonormalbasis for this v˜ andv˜ arealso1orthogon0al: 1
subspace. Hence, {v,u ,...,u } forms an orthonor- 0 1
1 n−1
malbasisforCn,orequivalently, hv ,v˜ i=hx ,v˜ i−hx ,v˜ i· hv˜ ,v˜ i =0
1 0 1 0 1 0 0 0
Ψ,[v, u1, ..., un−1] (5) =kv˜0k2ℓ2=1
⇒ v ⊥v˜ ⇒ v˜ ⊥v˜ | {z } (8)
is a unitary matrix. Obviously, the input vector v is 1- 1 0 1 0
sparsewithrespecttotherepresentationinΨ;however,v Moreoverfromthedefinitionofv wehave:
1
liesinthenullspaceofΦandcannotberecoveredusing
itssamples(allzero)producedbyΦ. x1 = v1+hx1,v˜0iv˜0
Thus, for any fixed linear sampling set with less number = hx ,v˜ iv˜ +hx ,v˜ iv˜ (9)
1 1 1 1 0 0
of samples than the original signal’s length, there exists
an orthonormalbasisanda 1-sparsesignal(in thisbasis) Therefore, x0 and x1 are respectively 1-sparse and
whichcannotbeuniquelyrecoveredfromitssamples. 2-sparse in any orthonormal basis which contains
v˜ and v˜ (existence of such a basis is trivial
0 1
by considering the subspace of Cn orthogonal to
3. General Sampling fork-sparseSignals
span{v˜ ,v˜ }). Thus, if k ≥ 2, both x and x are
0 1 0 1
validsolutionstothesparsityconstraint.Thus,again
Althoughdeterministiclinearsamplingfailseitherincom-
we can not uniquely reconstruct the original signal
pression or reconstruction, there exists the possibility to
fromitssamples.
introducegeneralnonlinearsamplingfunctionswhichdo
not have these drawbacks. In this section we will show
that for k-sparse signals with k ≥ 2 no such sampling 4. Sampling for1-sparseSignals
functionsexist.
Letusassumethat{f (x)}m arethesamplingfunctions The only case that we are left is the general (nonlinear)
i i=1
and let {s }m be the respective samples obtained from sampling of 1-sparse signals. In fact we will show the
i i=1
agivenk-sparsevectorx (inanorthonormalbasis). Re- existenceofsuchsamplingfunctionsforthiscase,butfirst
0
versibilityofthesamplingprocessisequivalenttothefact lets focus on the conditions that the sampling functions
thatx istheonlyk-sparsevectorwhichresultsinsamples shouldsatisfy:
0
{si}mi=1.Aspreviouslydefined,werepresentthesampling Lemma1 The set S ⊂ Cn is uniquely 1-sparse decod-
equivalentclassoftheobtainedsamples{s }m byS:
i i=1 ableiff:
S = x∈Cn ∀1≤i≤m, fi(x)=si (6) ∀a6=b∈S : 0< ha,bi <kakℓ2 ·kbkℓ2 (10)
(cid:8) (cid:12) (cid:9) (cid:12) (cid:12)
Since x0 ∈ S we(cid:12)know S =6 ∅. Since the case m ≥ whereha,biistheinner(cid:12)produc(cid:12)tofthetwovectors(aH ·
n contradictsthe compression concept, we are assuming b), k.k is the ℓ norm of the vector (square root of the
ℓ2 2
m < n. Moreover, if for all input vectors x ∈ Cn we inner product of the vector by itself) and |.| refers to the
0
have |S| = 1, sparsity constraintis of no use; i.e., a 1-1 absolutevalueofacomplexnumber.
mappingfromCntoCmisformedwhichisundesireddue
Using the Cauchy inequality and the non-negativeprop-
to its quantization problem. Therefore, there exist input
erty of the absolute-value operator, it is straightforward
vectors for which the equivalentclass S has at least two
elements. We choose x amongthese inputvectors; i.e., that:
0
bthoetrheoefxxistsanxd1x6=axr0esthuechsatmhaet(txhe,sxam∈pleSs).oLbteatinv˜edbferothme ∀a,b∈Cn : 0≤ ha,bi ≤kakℓ2 ·kbkℓ2 (11)
0 1 0 1 0 (cid:12) (cid:12)
normalvectorinthedirectionofx andletv bedefined The only characteristic (cid:12)which(cid:12)distinguishes an arbitrary
0 1
asfollows: set from a uniquely 1-sparse decodable set is that the
equalities do not occur for the latter. Now we get back
v1 =x1−hx1,v˜0iv˜0 (7) totheproofofthelemma. AccordingtotheCauchytheo-
rem,fortherightinequalitywehave:
whereha,bistandsfortheinnerproductofthevectorsa
andb,(ha,bi=aH ·b). Wehavetwocases: ha,bi =kak ·kbk ⇔ ∃c∈C: a=cb (12)
ℓ2 ℓ2
(cid:12) (cid:12)
(cid:12) (cid:12)
IfScontainstwounequalvectorsa,bforwhichtheabove Since f (a) = f (b), we have sign(ℜ{a }) =
1 1 i
equality holds, in the orthonormal basis containing the sign(ℜ{b }) and sign(ℑ{a }) = sign(ℑ{b }) for all
i i i
normalvectorinthedirectionofa(orb)bothofthevec- 1≤i≤n. Thereforeℜ{a }·ℜ{b }+ℑ{a }·ℑ{b }≥0
i i i i
tors are 1-sparse (since they are in the same direction) andthe equalityoccursif andonlyif ℜ{a } = ℜ{b } =
i i
which means that S is not uniquely 1-sparse decodable. ℑ{a }=ℑ{b }=0. Consequentlywehave:
i i
On the other hand, if for all unequal vectors of S strict
inequalityholds,eachlineinCn passingthroughtheori- n
ℜ{a }·ℜ{b }+ℑ{a }·ℑ{b }≥0
gincannotintersectSatmorethanonepoint.Thismeans i i i i
thatforeachvectorofanorthonormalbasis,thereexistsat Xi=1
⇒ℜ{ha,bi}≥0 (16)
mostoneelementofS whichis1-sparseinthisdirection.
Theleftinequalityconsiderstheorthogonalityofthevec-
withtheequalityfora=b=0whichcontradictsthefact
torsinS:
thataandbareunequal.Thus:
ha,bi =0 ⇒ a⊥b (13)
a6=b
(cid:12) (cid:12) ⇒ ℜ{ha,bi}>0
Assume S con(cid:12)tains (cid:12)two nonzero perpendicular vectors (cid:26) f1(a)=f1(b)
a,bandleta˜,b˜ bethenormalvectorsintheirdirections,
⇒ ha,bi >0 (17)
respectively.Sincea˜andb˜ areorthogonal,thereexistsan
orthonormalbasisforCn whichcontainsbotha˜,b˜;obvi- (cid:12)(cid:12) (cid:12)(cid:12)
Now using the samples generated by f and f (in ad-
ouslyaandbare1-sparseinthisbasiswhichimpliesthat 2 3
dition to f ) we prove the right inequality. If a and
S isnotuniquely1-sparsedecodable. 1
b are such that f (a) = f (b), f (a) = f (b) and
Forthesufficiencyofthecondition,ifnotwovectorsofS 2 2 3 3
ha,bi = kak · kbk , a and b must have the same
areperpendicular,amongthevectorsofeachorthonormal ℓ2 ℓ2
(cid:12)directio(cid:12)n(Cauchyinequality)orequivalently,thereexists
basis of Cn there exists at most one which intersects S (cid:12) (cid:12)
c∈Csuchthatb=ca:
(twointersectionsrevealaperpendicularpairinS). Also
theresultoftheright-sideinequalityrestrictsthenumber n n
ofintersectionsforeachdirectiontoone;so,atmostone f2(b)=f2(ca)= c·ai = |c| |ai|
ofthevectorsinS canbe1-sparseinthisbasis(1-sparse Xi=1(cid:12) (cid:12) Xi=1
(cid:12) (cid:12)
decodablityofS)andtheproofofthelemmaiscomplete. = |c|f2(a) (18)
Our next step is to introduce sampling functions whose
equivalent sampling classes fulfill the condition of the The condition f2(a) = f2(b) restricts the amplitude of
lemma. We claim that the following three functions al- c to have the value 1 (|c| = 1); note that f2(a) = 0 re-
waysproducesuchsets: sults in a = b = 0 which again contradicts the previ-
ous assumption of distinctness of a and b. Thus, up to
f1(x)= ni=132(i−1)sign(ℜ{xi}) this pointwe shown that by havingthe first two samples
 P +32i−1sign(ℑ{xi}) (generatedbyf1andf2),theoriginalvectorwithinacon-
 f2(x)=kxkℓ1 = ni=1|xi| (14) ssthaonwtctohmatptlheextchoiredffiscaimenptleonunthiqeuuenlyitdceitrecrlmeiinneksntohwenp.hWasee
P of that coefficient which implies the overall uniqueness.
 f3(x)=Pxi6=0 ms|iℜgn{(xℜi{}x|+i}|)ℑ·{ℑxi{}x|i} Bleetfuosrecownesipdreorctheeedeftfoecretvoefaqluthanetrizoilnegotfhtehseecthoinrddssaammppllee.,
where ℜ{.} and ℑ{.} represent the real and imaginary Sincetheabsolute-valueoperatorisacontinuousfunction,
parts, respectively, and msign(.) is the modified sign thesamplingfunctionf isalso acontinuousfunctionof
2
function which generates the same output as the sign(.) the input vector. Continuity implies that small perturba-
functionexceptthatmsign(0)=1. tionsintheinputsuchasquantizationaremappedtosmall
The first sampling function, f , although produces only
1 deviations in the output. Moreover, the quantization of
onesample,uniquelydeterminesthesignofbothrealand
thissampledoesnotaffectthementionedrestrictiononc
imaginary parts of all the elements of x. Since the out-
(|c| = 1); i.e., the ambiguity is still the phase of c. In
put range of sign function is restricted to three different
simple words, although the reconstructed amplitudes do
values of −1, 0 and 1, the sample generated by f1 can notexactlymatchtheoriginalones,theuniqueness(inthe
beviewedasthebase3representationofanintegernum-
amplitude)isstillvalid.Infact,duetoquantization,there-
ber with the difference that the digit 2 is replaced with
producedamplitudesfallwithinaprecisionfromtheorig-
-1 (the same residue in division by 3 which yields the
inalvalues. If the inputelementsof the vectorare quan-
uniqueness). Moreover,thissampleisanintegerbetween tizedwithlbitspriortosampling,theoutputofthef can
−32n2−1 and 32n2−1 whichrequires⌈2nlog23⌉bitsforer- be quantized with l +⌈log2(n)⌉ bits (unsigned) wi2thout
rorlessquantization. We willshowthatthissampleguar- causinganyfurtherlossofinformation.
antees the left inequality. Let a and b be two unequal Nowwe showtheroleofthelastsampleinrevealingthe
vectorsthatf1(a)=f1(b),weshowthatha,bi=6 0: phase of the nonzero elements of the input vector in the
sparsitydomain. Sincethesummationinf istakenover
ℜ{ha,bi}=ℜ{aH ·b} 3
the nonzero elements of the vector, the denominator in
n
eachofthetermsisapositiverealnumber(thedenomina-
= ℜ{a }ℜ{b }+ℑ{a }ℑ{b } (15)
i i i i
torbecomeszeroonlywhenℜ{x } = ℑ{x } = 0which
Xi=1 i i
isexcludedbyx 6=0). Besides,wehave: sample is validhere; evenafter quantization,thereexists
i
atmostone1-sparsevectorwhichproducesthesesamples.
msign(ℜ{x }) · ℑ{x }
i i Giventhatthequantizationdoesnotchangethesignofthe
0≤
(cid:12)(cid:12) |ℜ{xi}|+|ℑ{xi}| (cid:12)(cid:12) realandimaginaryparts,thesamplingfunctionswillstill
(cid:12) |ℑ{x }| (cid:12) beintheircontinuousregionandtherefore,smallerrorsin
= (cid:12) i ≤1(cid:12) (19)
|ℜ{x }|+|ℑ{x }| the input (such as quantization)will be mapped to small
i i
errorsinthereconstructedsignal.
whichshowsthatsamplegeneratedbyf3 isboundedbe- It is easy to check that if the domain is R rather than C,
tween−nandn(boundednessisnecessaryforquantiza- the third sample is not required; i.e., the sign of the el-
tion). ements and the ℓ norm of the vector suffice for unique
1
Ifaandbaretwovectorswhichproducethesamesetof representationof1-sparsevectors.
samples (all three), we know b = c a where |c| = 1 by
useofthefirsttwosamples. Consideringthethirdsample 5. Conclusions
weshouldhavef (a)=f (b):
3 3
We have consideredthe deterministicsampling of sparse
msign(ℜ{a }) · ℑ{a }
i i signals whose sparsity domain is not known at the time
|ℜ{a }|+|ℑ{a }|
aXi6=0 i i of sampling. Although random linear sampling of such
msign(ℜ{ca }) · ℑ{ca } signalshasbeenshowntobeaproperchoice(undersome
i i
= (20) conditions),thedeterministicapproachfailsinreconstruc-
|ℜ{ca }|+|ℑ{ca }|
cXai6=0 i i tionofsomeidentifiablesubclasses.Thementioneddraw-
backisstillanissuewhennonlinearmeasurementsofthe
Owing to the fact that |c| = 1, the conditions a 6= 0
i k-sparsesignals wherek > 1 are employed. It isshown
and ca 6= 0 are equivalent; therefore, the terms used
i that class of 1-sparse signals (in an arbitrary linear spar-
in the summations are the same in both sides. In addi-
sity domain) can be uniquely identified using nonlinear
tion, the first sample uniquely determines the sign and
sampling. We have demonstrated a necessary and suffi-
therefore msign of both real and imaginary parts; thus,
cient conditionfor the sampling functionsto providethe
we should have msign(ℜ{a }) = msign(ℜ{ca }) and
i i unique1-sparsedecodabilityofthesamples,inadditionto
sign(ℑ{a })=sign(ℑ{ca }). Tosimplifythenotations,
i i presentingarealizablesetofsuchfunctions.
werepresenttherealandimaginarypartsofa andca by
i i
′ ′
R ,I and R ,I , respectively. Moreover, by sn and s˜n
i i i i References:
wemeanthesignandmsignoperators:
s˜n(R ) · I s˜n(R′) · I′ [1] R. Baraniuk. A lecture on compressive sensing.
|R |i+|I |i − |R′|i+|I′|i IEEESig.Proc.Magazine,241:118–121,July2008.
i i i i [2] R. Baraniuk, M. Davenport, R. DeVore, and M. B.
′
=s˜n(R )·sn(I ) |Ii| − |Ii| Wakin. A simple proof of the restricted isometry
i i (cid:18)|Ri|+|Ii| |Ri′|+|Ii′|(cid:19) propertyforrandommatrices. Constr.Approx.
|I |.|R′|−|I′|.|R | [3] E.CandesandT.Tao. Decodingbylinearprogram-
=s˜n(Ri)·sn(Ii) |R |i+|Ii| |Ri′|+i|I′| ming. IEEETrans.Inform.Theory,51.
i i i i [4] E. Candes and M. Wakin. An introduction to
′ ′(cid:0) (cid:1)(cid:0) (cid:1)
= Ii.Ri−Ii.Ri compressive sampling. IEEE Sig. Proc. Magazine,
|R |+|I | |R′|+|I′| 25(2):2130,March2008.
i i i i
(cid:0) −ℑ{c(cid:1)}(cid:0)·|a |2 (cid:1) [5] E.J.Candes. Compressivesampling. InProc.ofthe
i
= |R |+|I | |R′|+|I′| (21) InternationalCongressofMath.
i i i i [6] R. A. DeVore. Deterministic construction of com-
(cid:0) (cid:1)(cid:0) (cid:1)
Thus,f (a)=f (b)yields: pressed sensing matrices. Journal of Complexity,
3 3
doi:10.1016/j.jco.2007.04.002.
|a |2 [7] D. Donoho. Compressed sensing. IEEE Trans. In-
−ℑ{c} i =0
|R |+|I | |R′|+|I′| form.Theory,52(4):1289–1306,April2006.
aXi6=0 i i i i [8] Y.Eldar.Compressedsensingofanalogsignals.sub-
(cid:0) (cid:1)(cid:0) (cid:1)
>0 mittedtoIEEETrans.Inform.Theory.
| ⇒ ℑ{c}{z=0 } (22) [9] Y. Eldar. Uncertainty relations for analog signals.
submittedtoIEEETrans.Inform.Theory.
The aboveresult in conjunctionwith the previouscondi-
[10] A.C.Gilbert,M.J.Strauss,J.A.Tropp,andR.Ver-
tionthat|c| = 1remainstwopossiblechoicesforc, 1or shynin. Algorithmic linear dimension reduction in
−1;nonetheless,c =−1producesdifferentsignsforreal the l1-normfor sparse vectors. Inin Proc. Allerton
and imaginary parts in b which is not acceptable by the
Conf.onComm.
choice of the first sample. As a consequence, c = 1 is
[11] V.Saligrama. Deterministicdesignswithdetermin-
theonlychoicewhichmeansthata=bandtheprooffor
istic gaurantees: Toeplitz comprerssed sensing ma-
theclaimedunique1-sparsedecodabilityoftheaforemen-
trices, sequence design and system identification.
tionedsetofsamplingfunctionsiscomplete.Asimilarar-
arXiv:0806.4958.
gumenttotheonepresentedforquantizationofthesecond