Investigation into Matrix Factorization when Elements are Unknown

62 Pages·2005·1.56 MB·English

Investigation into Matrix Factorization
when Elements are Unknown
TechnicalReport
AeronBuchananMorgan
VisualGeometryGroup
DepartmentofEngineeringScience
UniversityofOxford
30th November2004

Abstract
Theproblemoflow-rankmatrixfactorizationhasseensignificantattentioninrecentcom-
puter vision research. Problems that use factorization to find solutions include structure
frommotion,non-rigidobjecttrackingandilluminationbasedreconstructions. Matrixde-
composition algorithms, such as singular value decomposition, can be used to obtain the
factorizationswhenalltheinputdataareknown,reliablyfindingtheglobalminimumofa
certaincostfunction. However,inpractice,missingdataleadstoincompletematricesthat
preventtheapplicationofstandardfactorizationalgorithms. Todate,manyalgorithmshave
been proposed to deal with the missing data problem. This report presents the results of
aninvestigationintothesealgorithmsanddiscussestheireffectiveness. Itisseenthatthey
rarely find the global minimum. Newton based methods, which have not been previously
appliedtothisproblem,areinvestigatedandshowntofindtheglobalminimummorereli-
ably,butdonotfulfiltheexpectationsonemayhaveofoptimizationroutines. However,itis
arguedthattheyaremoreeasilyextendedtoovercometheshortfallsofthebasicapproach
thantheotheralgorithmsreviewed. Furthermore,thesuitabilityoftheglobalminimumas
a solution is covered, creating more avenues of investigation for the improvement of fac-
torizationschemes. Futureresearchlookingintosuchextensionsisdescribedwiththeaim
ofengineeringasuccessfulalgorithm.
Acknowledgements
I would very much like to thank Andrew Fitzgibbon for being so enthusiastic and patient
andDavidCapelforhisnotesonParametricOptimization[9].
1
Contents
1 Introduction 4
1.1 ExampleProblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.2 MatrixFactorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.3 Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.5 TheErrorFunction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
2 ExistingSolutions 18
2.1 DirectMethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2 AlternationApproaches . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3 OtherLiterature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
3 ErrorSurfaceExploration 32
3.1 GradientDescent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.2 Newton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
3.3 DampedNewton . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
3.4 DampedNewtonwithLineSearch . . . . . . . . . . . . . . . . . . . . . . 34
3.5 HybridSystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
4 Results 38
4.1 SyntheticPerformanceComparison . . . . . . . . . . . . . . . . . . . . . 38
4.2 RealProblems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
4.3 Priors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
5 Conclusions 52
A AnAlternationDerivation 53
B HigherOrderTerms 55
2
Problem: forpartiallyknownM,findAandBsuchthat
M = AB(cid:62)
Inpractice:
(cid:176) (cid:179) (cid:180)(cid:176)
(cid:176) (cid:176)
min(cid:176) W(cid:175) Mˆ−AB(cid:62) (cid:176) assuming Mˆ = M+N(0,σ2)
A,B
Symbol Meaning
Mˆ The acquired measurement matrix. This is the only variable (other than
W)whoseelementsareknownatall. Itismodelledasanoisyversionof
M.
W Theweightmatrix. Amatrixwhoseelementsdescribetheconfidencein
theknowledgeofthecorrespondingelementsofMˆ.
M Therankr matrixtobefactorized.
A AcolumnspaceforM.
B ArowspaceforM.
m ThenumberofrowsinM,Mˆ,WandA.
n ThenumberofcolumnsinM,Mˆ&WandthenumberofrowsinB.
r TherankofMandthenumberofcolumnsinAandB.
m TheelementofMontheith rowandjth column.
ij
mi Theith rowofMasacolumnvector.
m Thejth columnofM.
j
M Dimensionsofvectorsandmatricesaregiveninsubscriptedparentheses.
(m×n)
δ AKroneckerdeltafunctionequallingonewheni = jandzerootherwise.
ij
(cid:175) TheHadamardelement-wisematrixproduct: R = P(cid:175)Q =⇒ r =
ij
p q
ij ij
M+ Thepseudo-inverseofMallowingM+M = Iforr = n < m.
M↓ RankTruncation: theclosestrankr matrixtoMbytheFrobeniusnorm.
r
M(:) Column-wisevectorizationofM: M(:) = (m(cid:62)m(cid:62) ... m(cid:62))(cid:62)
1 2 n
M(··) Row-wisevectorizationofM: M(··) = (m1(cid:62)m2(cid:62) ... mm(cid:62))(cid:62)
Figure1: Summaryofnotationandoperatorsymbolsusedinthisreport.
3
Chapter 1
Introduction
Matrix factorization is a powerful tool for finding solutions to several central computer
vision problems. These include structure from motion [33, 38], object illumination [5]
andnon-rigidmodeltracking[8]. Tocementthemotivationforinvestigatingfactorization,
these applications are described in detail later in this chapter. It will become clear that
a major hurdle for computer vision in all these applications is missing data. Standard
factorizationalgorithms(e.g. singularvaluedecomposition)cannotprovidesolutionswhen
data are missing. If we want to overcome real-life challenges, the ability to cope with
missing data must be part of the algorithms we intend to use. To date, there have been
many algorithms introduced to provide a solution. However, none manage to consistently
provideeithergoodperformance,satisfactorysolutionsorboth. Anactualexampleofeach
of the three applications mentioned above is introduced in the next section to illustrate
the problems in a more real sense. It is then important to review the mathematics that is
common to all the applications that are covered. Section 1.2 sets the mathematical scene
and comments on some of the finer points of the problems to ensure that the important
aspectsoftheproblemcanbedifferentiatedfromthosewhicharesecondaryandsoarenot
consideredinthisreport. Chapter2presentsthemanyalgorithmsthathavebeenproposed
intheliterature. Newtonbasedmethodsarepresentednext,beingaclassofalgorithmsthat
havenotyetbeenexploredforthisproblem. Acomparisonofalltheseschemesisgivenin
Chapter4usingextensivesynthetictestsandtherealexampleproblemsintroducedinthis
chapter. Finally,theconclusionssummarizethereportanddescribethedirectionoffurther
research.
4
1.1 Example Problems
Therestofthischapterwillprovidethemathematicaldetailsfortheseexamples,sohereis
amorequalitativedescriptionofpossibleapplicationsforafactorizationalgorithm.
Figure 1.1 shows selected frames from three real image sequences. The first is an
example of a structure from motion (SFM) problem. It is a toy dinosaur rotating on a
turntable. It is a rigid scene seen from many angles. The sequence is given to tracking
software that finds visible features in the scene and tracks the image of each feature as
it moves around in image-space. A measurement matrix is filled by the lists of the 2D
image coordinates of each feature. Impressively, both the structure of the scene (the 3D
coordinates of the tracked features on the dinosaur) and the position and orientation of all
thecamerascanbeobtainedfromthemeasurementmatrixalone. Holesinthemeasurement
matrix arise because not every feature is physically visible in every frame and because
the tracking software is not completely reliable and fails to find a visible feature in some
frames,soSFMisamissingdataprobleminpractice.
Thesecondsequenceisanexampleofanon-rigidscene. Itisnon-rigidontwolevels;at
thescenelevel(wheretherearethreeindependentlymovingobjects,namelytheeffectively
rigidsceneryandtwogiraffes)andattheobjectlevel(thegiraffesareverymuchdeforming
objects). The segmentation of the three objects (i.e. the scene level non-rigidity) is not
consideredinthisreportandisleftforfuturework(seeChapter5). Instead,thebackground
giraffeisconsideredinisolation. Again,trackingsoftwareisusedtogeneratethedatathat
is used to create the measurement matrix. The reasons for missing data in the non-rigid
reconstructionproblemarethesameasfortherigidsituation. Heretheocclusionsarevery
clear in the first frame shown where the giraffe is behind a bush and in the second frame
shownwheretheforegroundgiraffewalksinfront.
The third sequence demonstrates the setup for illumination based reconstructions. A
staticsceneislitbyadistantlightsourcefromdifferentdirections. Duringthefilming,the
movementofthelightsourcewascontrolledsothatthedistancebetweenthebulbandthe
facewasverymuchlargerthanthesizeoftheface,thusapproximatingtherequirementof
itbeinganinfinitedistance. Inthiscasethemeasurementmatrixisgenerateddirectlyfrom
the intensity values of the images. Shadows and specular highlights have to be omitted
fromthemeasurementmatrixbecausesuchvisualeventsarenotpartofthediffusemodel
(aswillbeexplainedinSection1.4.4onpage13).
5
Frame 01 Frame 13 Frame 25
Frame 026 Frame 047 Frame 099
Frame 06 Frame 10 Frame 13
Figure1.1: Threesequencesdemonstratingactualproblemsrequiringmatrixfactorization.
Top row: a toy dinosaur rotates on a turntable (a sequence of 36 image frames). Middle
row: abackgroundgiraffewalksoutfrombehindabushandismomentarilyoccludedbya
giraffeintheforegroundwalkinginfrontofit(120frames). Bottomrow: astaticsceneis
illuminatedfrommanydirections(20frames).
6
1.2 Matrix Factorization
Thegeneralproblembeingaddressedinthisreportismatrixfactorization. Givenamatrix,
M,ofknownrank,r,findtwosmallermatrices,AandB,suchthat:
M = A B(cid:62) (1.1)
(m×n) (m×r) (n×r)
Theprevioussectiondescribedexamplesofrealproblems. Theprocesscommontothe
applicationsisasfollows;animagingdevicerecordsoneormoreimagesoftherealworld
in a digital form. These images are processed and the entries of M are filled. The solution
soughtisanAandBthatsatisfyEquation1.1. Inpracticetherearefoursignificantaspects
thatmakeasolutiontoEquation1.1difficult:
1. There is not a unique answer to any such problem, as any invertible r ×r matrix, G,
mayalwaysbeintroducedasatransformonthefactoringmatrices:
M = (AG)(G−1B(cid:62)) = A(cid:48)B(cid:48)(cid:62) (1.2)
Here,Grepresentsthegaugefreedom[39]. Itturnsoutthatthisisnotactuallyasinconve-
nientasitmayseem. Inmostapplicationsitcaneitherbedisregardedoreasilycircumnav-
igated,e.g. byregistrationornormalization.
2. Misnotknownexactly. Instead,anoisyversionofMisacquired. Forconvenience,the
differences between the known noisy elements and the supposed true values are assumed
to be independently drawn from a particular distribution. Although no one distribution
maybeusedingeneral,thealgorithmsreviewedinthisreportallassumethatazero-mean
Gaussiandistributionisvalid. ThealgorithmsIproposeinChapter3arenotconstrainedto
aparticulardistribution,butusingtheNormaldistributionwehave:
Mˆ = M+N(0,σ2) (1.3)
Mˆ is the measurement matrix. It is so likely that Mˆ will have a rank greater than r that the
alternatives are ignored. Because the expectation, <Mˆ>, is M, the problem to be solved
becomesthatoffindingtherankr matrixthatisclosesttoMˆ,i.e.
(cid:176) (cid:176)
(cid:176) (cid:176)
min(cid:176) Mˆ−AB(cid:62) (cid:176) (1.4)
A,B
WhentheFrobeniusor2-normistakenasadistancemeasure,singularvaluedecomposition
givesthecorrectanswer(seeGolubandVanLoan[12],page72andReidandMurray[26]).
3. Forvariousapplication-specificreasons(seeprevioussection),notalltheelementsof
Mˆareknown. Anyminimizationschemeemployedmustbeadaptedtoignoretheelements
ofMˆthatdonothavevalues. Here,weintroduceaweightmatrix,W,whoseelementsreflect
the knowledge of the corresponding elements of Mˆ. Zeros in W mean that those positions
inMˆ donothavevaluesandonesmeantheydo. Valuesbetweenzeroandonemayalsobe
7