Structure from Motion: Theoretical Foundations of a Novel Approach Using Custom Built Invariants

15 Pages·0.15 MB·English

by  Pierre-Louis Bazin

Structure from Motion: Theoretical Foundations
of a Novel Approach Using Custom Built
2
0 Invariants
∗
0
2
n Pierre-Louis Bazin1 MireilleBoutin2
a
J
1 DepartmentofEngineering,
2
2 2 DivisionofAppliedMathematics,
BrownUniversity
]
V ProvidenceRI02912,USA.
C
.
s
c
[ Abstract

1 Werephrasetheproblemof3Dreconstructionfromimagesintermsofinter-
v sectionsofprojectionsoforbitsofcustombuiltLiegroupsactions. Wethenuse
9 analgorithmicmethodbasedonmovingframesa` laFels-Olvertoobtainafunda-
1 mentalsetofinvariantsofthesegroupsactions. Theinvariantsareusedtodefine
0 a set of equations to be solved by the points of the 3D object, providing a new
1
techniqueforrecovering3Dstructurefrommotion.
0
2
0 1 Introduction
/
s
c
Theconceptofinvarianceisofmajorimportanceinmoderngeometry. Inthefieldof
:
v computervision,invariantshavebeenusedformorethanadecadeforobjectrecogni-
i
X tionandreconstruction(seeforexample[7,8]).
One particular vision problem one could hope to solve using invariantsis that of
r
a reconstructinga3Dobjectfromasetofpicturestakenfromunknownviewpoints. For
example, given n ordered points , ,... , R3 (the object) and t sets of n
1 2 n
orderedpointsp ,... ,p R2 (fOoreaOchoftheOpic∈tures),wewouldliketodetermine
1 n
theobjectinR3fromtheim∈ages.
Onepossiblewayofsolvingthisproblemwouldbetodefineanequivalencerela-
tionbetweenall the possiblepicturesof an objectandto find functionscomputedon
the picturesthatare constanton eachequivalenceclass. Characteristicsof the object
couldbeinferredfromthesefunctions,regardlessofthecamerapositionrelativetothe
object. To define such an equivalence class, one can try to use the orbits of a group
action.Realvaluedfunctionsthatareconstantontheorbitsofagroupactionarecalled
∗ThisworkwassupportedbyNSFgrantsKDIBCS-9980091and0074276.
1
invariants. We wouldthusbeinterestedin findinginvariantsofa groupactionwhich
wouldbetransitiveonthesetofpicturesofanygivenobject,i.e.“viewinvariants”.
Unfortunately, as is commonly known in the vision community, view invariants
do not exist for 3D point sets of arbitrary size (in general position). One can still
buildinvariantsforspecificobjects(forinstance,planarsetsofpoints,pencilsoflines,
etc.) butnotforarbitraryshapes. Theproblemisthatthe setofpicturesofanobject
intersectswith the set of picturesof other objects. Observethatif a view invariantI
takesaconstantvaluecforallpicturesofanobjectO,thenI isalsoequaltoconthe
setofpicturesofanyobjectwhosesetofpicturesintersectwiththesetofpicturesof
O. Onecanactuallyshow[2]thatanyequivalencerelationbetweenallthepicturesof
eachobjectsdefinesauniqueequivalenceclassonthespaceofpicturesandthus,any
viewinvariantistrivial. Fromagrouppointofview,thismeansthatanygroupaction
thatistransitiveonthesetofpicturesofanyobjectmustbetransitiveonthesetofall
pictures.
Another way to solve this problem would be to use the reverse approach: try to
characterizeallthepossibleobjectscorrespondingtoagivenpicture. Itwouldbeuse-
fultofindfunctionsthatareconstantonequivalenceclassesthatincludealltheobjects
corresponding to a given picture. However, it is easy to see that such equivalence
classesofobjectsareinonetoonecorrespondencewiththeequivalenceclassesofpic-
turesdiscussedearlier,andthusthatonlyonesuchequivalenceclasscanexist. There
are thereforeno“objectinvariants”. Nevertheless, givena view ofan object, we can
inferinformationabouttheobject,sotheremustbeawaytoovercomethisdifficulty.
Itisthefactthatweknowhowacamerabuildsimages(usingtheperspectivepro-
jection)thatwillallowustobuildanequivalencerelationwhichcharacterizesobjects
correspondingtodifferentpictures.Thekeyistoconsideranequivalencerelationona
higherdimensionalspace,sortofliftingthesetofobjectsofdifferentpicturestodiffer-
ent“heights”intheextradimensions.Wecandothisusingthethreeextradimensions
providedbythecameracenterposition. Moreprecisely,wecanconstructaLiegroup
action on the object points ,... , and the camera center P which summarize
1 n 0
O O
whatisunknownabouttheobject-camerasystemgivenapictureofanobject. Invari-
antsofthisgroupactionsprovetobesufficientforsolvingtheproblemofrecovering
theobjectcoordinatesinR3,i.e. the”structurefrommotion”problem.
Recentadvancesinthetheoryofmovingframes[3,4]provideuswithasystematic
waytoproceedtoobtaintheinvariantsofanygiven(regular)Liegroupaction. Using
thissystematicmethod,weobtaintheinvariantsofourcustombuiltactionwithoutany
difficulty. Wecanevenventureabitfurtherbymodifyingthegroupactionandrepeat
theexercisesotoobtainevenmorenewtoolsfor3Dstructureandmotionanalysis.
We begin our exposition with a summary of some of the relevant theoretical as-
pectsofthetheoryofmovingframesanditsapplicationtothecomputationof(joint)
invariants. In Section 3, we formulate the problem of structure from motion in this
framework and obtain the corresponding invariants. We end the section by remark-
inghowthisleadstoa simple testtoidentifycameramotionsthatarepurerotations.
Section4presentssomevariationsofourapproachwheremorecameraparametersare
considered,includingageneralizationtothecaseofavariablefocaldistance.
2
2 Theoretical Foundations
LetM beanm-dimensionalsmooth(Hausdorff)manifoldandGbeanr-dimensional
Liegroupacting(smoothly)onM.
Definition2.1. AninvariantisafunctionI :M Rwhichremainsunchangedunder
→
theactionofthegroup.Inotherwords,
I(g z)=I(z), forallz M andallg G.
· ∈ ∈
Alocalinvariantis afunctionI :M RforwhichthereexistsN ,aneighborhood
e
→
oftheidentitye G,suchthat
∈
I(g z)=I(z), forallz M andallg N .
e
· ∈ ∈
Definition2.2. We say that G acts semi-regularly on M if all orbits have the same
dimension. If, in addition, any point p M is surrounded by an arbitrarily small
0
∈
neighborhoodwhoseintersectionwith theorbitthroughp isconnected,thenwesay
0
thatGactsregularly.
The following theorem, due to Frobenius [5], is of central importance to our ap-
proach.Aproofcanbefoundin[9]. Itprovidesuswithasimplewayofcharacterizing
theorbitsusinginvariants.
Theorem2.3(FrobeniusTheorem). IfGactsonanopensetO M semi-regularly
⊂
with s-dimensionalorbits, then p O there exist m s functionallyindependent
0
∀ ∈ −
localinvariantsI ,... ,I definedonaneighborhoodU ofp suchthatanyother
1 m−s 0
local invariant H defined near p is a function H = f(I ,... ,I ). If G acts
0 1 m−s
regularly on O, then we can choose I ,... ,I to be global invariants on O. In
1 m−s
that case, two points p ,p O are in the same orbit relative to G if and only if
1 2
∈
I (p )=I (p ),foralli=1,... ,m s.
i 1 i 2
−
By functional independence of the (smooth) functions I ,... ,I on an open
1 m−s
set O, we simply mean that the Jacobian matrix of I ,... ,I has maximal rank
1 m−s
m s on an open and dense subset of O. The set I ,... ,I is often called a
1 m−s
− { }
completefundamentalsetofinvariantsonO. Notethatthecompletefundamentalset
ofinvariantsisbynomeanunique.
As we shallconsideractionsonpoints, we are interestedinthe case whereM =
... (n-times)=: ×(n)istheCartesianproductofncopiesofamanifold
V×V× ×V V
.
V
Definition2.4. WesaythatGactsdiagonallyon ×(n) ifthereexistsanactionofG
on such that for any g G, for any n N anVd any z ,... ,z , the action
1 n
V ∈ ∈ ∈ V
g (z ,... ,z )canbewrittenas
1 n
·
g (z ,... ,z )=(g z ,... ,g z ).
1 n 1 n
· ∗ ∗
The groupactions we will define for our object-camerasystems are not diagonal
actions.However,wewillfindanormalsubgroupH ofGsuchthat
3
1. ThesubgroupH canbewrittenasH =H ... H(n-times)whereHisaLie
× ×
groupactingon inaconsistentmannerwiththeactionofH,i.e.(h ,... ,h )
1 n
V ·
(z ,... ,z )=(h z ,... ,h z ),forallh ,... ,h Handallz ,... ,z
1 n 1 1 n n 1 n 1 n
∗ ∗ ∈ ∈
. Thisinsurethat ×(n)/H canbewrittenas ×(n)/H =( /H)×(n).
V V V V
2. TheactionofG/H on ×(n)/H =( /H)×(n) isdiagonal.
V V
So for all practical purposes, we shall ultimately have to deal with diagonal group
actions.
Inourapproachtostructurefrommotion,invariantsareusedtoobtainedequations
thatmustbesatisfiedbytheobjectandthecamera. Themoreinvariantswehave,the
moreequationsneed to be satisfied. We need enoughequationsto completelydeter-
minetheobject. Observethatthedimensionoftheorbitisboundedbythedimension
of the group. So in the case of a diagonalaction, taking moreand more copiesof
V
(i.e. moreandmorepoints)allowsfortheexistenceofasmanyinvariantsasnecessary.
The question that remains is: how can we obtain an expression for these invariants?
Thanks to recent advances in the theory of moving frames [3, 4], this problem can
nowbesolvedinanalgorithmicfashion. Wenowsummarizesomeofthistheoryand
showhowmovingframescanbeusedasatooltoobtainacompletesetoffundamental
invariants.
Definition2.5. A(right)movingframeisamapρ:M Gwhichis(right)equivari-
→
ant,i.e.ρ(g z)=ρ(z)g−1,forallg Gandz M.
· ∈ ∈
Unfortunatelymovingframesdonotexistforallgroupactions.
Theorem2.6. Amovingframeexistsifandonlyiftheactionifthegroupactionsatis-
fies
g G z M,g z =z =e,
{ ∈ |∃ ∈ · }
whereedenotestheidentityinG. Thispropertyiscalledfreeness ofthegroupaction.
Demandingfreenessofthegroupactionisverystrong. Itappearsthat,inorderto
beabletodealwiththegenericcases,weneedtorelaxthisconditionalittlebit.
Definition2.7. A local moving frame is a map ρ : M G such that ρ(g z) =
→ ·
ρ(z)g−1,forallg N ,aneighborhoodoftheidentitye G,andallz M.
e
∈ ∈ ∈
Theorem2.8. A localmovingframe exists ifandonlyif thereexists a neighborhood
N oftheidentityine Gsuchthat
e
∈
g N z M,g z =z =e,
e
{ ∈ |∃ ∈ · }
or equivalently, if and only if for all z M, the dimensionof the orbit throughz is
∈
equal to r, the dimension of G. This property is called local freeness of the group
action.
4
Our hope is that, by acting diagonally on more and more copies of a manifold,
weshalleventuallyobtainalocallyfreeaction. Soweareinterestedindetermininga
simple criterion on the groupaction of G on to determinewhetheror notthis will
V
workout.
Definition2.9. WesaythatGactsonM effectivelyif
g G g p=p, forallp M = e .
{ ∈ | · ∈ } { }
WesaythatGactsonM locallyeffectivelyif
g G g p=p, forallp M isadiscretesubgroupofG.
{ ∈ | · ∈ }
Many groupsdo not act effectively. However, given G acting non effectively on
M,wecanconsiderG˜ = G/G ,whereG = g Gg z = z, z M ,which
M M
{ ∈ | · ∀ ∈ }
acts in essentially the same way as G except that it acts effectively. Unfortunately,
effectivenessisnotquiteenoughforachievingourgoal.
Definition2.10. WesaythatGactseffectivelyonsubsetsofM if,foranyopensubset
U M,
⊂
g G g p=p, forallp U = e .
{ ∈ | · ∈ } { }
WesaythatGactslocallyeffectivelyonsubsetsofM if,foranyopensubsetU M,
⊂
g G g p=p, forallp M isadiscretesubgroupofG.
{ ∈ | · ∈ }
Observe that effectiveness on subsets implies effectiveness. The converse is not
trueingeneral,butitholdsforallanalyticgroupactions.
Theorem2.11. IfagroupGactsonamanifold locallyeffectivelyonsubsets,then
forn Nbigenough,theinduceddiagonalactioVnofGon ×(n)islocallyfreeonon
∈ V
openanddensesubsetof ×(n). Thisisequivalenttosayingthattheorbitdimension
V
is equal to the dimension of G on this open and dense subset. We denote by n the
0
minimalintegerforwhichthisistrue.
Soatleastforallanalyticgroupactions(modingoutbyasubgroupifnecessary),
acting (diagonally)on more and more copies of a manifold will eventually lead to a
regular and locally free action on an open subset. A local moving frame will thus
exist and provideus with the tools we need to obtain a complete fundamentalset of
invariantssocompletelycharacterizetheorbits.
Wenowexplainhowtoconstructa(local)amovingframeandtoobtainacomplete
fundamentalsetofinvariants. Amoredetailedexpositioncanbefoundin[9,Chapter
8].
Letg = (g ,... ,g )belocalcoordinatesforGinaneighborhoodoftheidentity.
1 r
SupposethatGactsregularlyonM. Forsimplicity,letusassumeinadditionthatthe
orbitsofGhavethesamedimensionrasGitself.Inotherwords,weareassumingthat
the action is locally free. A simple variationallowingus to dealwith merelyregular
actionswillalsobeexplainedshortly.
5
Step1:Writedownthegrouptransformationequationsx¯=g xexplicitly.
• ·
x¯ = f (g ,... ,g ,x ,... ,x ),
1 1 1 r 1 m
.
.
 .
 x¯ = f (g ,... ,g ,x ,... ,x ).
m m 1 r 1 m
Step 2: Choose constantsc ,... ,c R and set r of the transformedcoordi-
1 r
• ∈
nates equal to those constants. For simplicity, we relabel the coordinates and
write
f (g ,... ,g ,x ,... ,x ) = c ,
1 1 r 1 m 1
.
. (1)
 .
 fr(g1,... ,gr,x1,... ,xm) = cr.
Theseequationsarecalledthenormalizationequations.
Step 3: Solve the normalizationequationsfor g = (g ,... ,g ). The solution
1 r
•
g =ρ(x)isamovingframe.
Step4: Computetheactionofthemovingframeontheremainingcoordinates.
•
Thesetofresultingfunctions
x¯ = I (x ,... ,x ),
r+1|g=ρ(x) 1 1 m
.
.
 .
 x¯ = I (x ,... ,x ).
m|g=ρ(x) m−s 1 m
isacompletefundamentalsetoflocalinvariants.
ThechoiceofconstantsinStep2issomewhatarbitrary:wearefreetochooseany
numbersforwhichasolutiontothenormalizationequationsexists,providedthatthese
constantsdefinea cross-section(i.e.providedthatthe normalizationequationsdefine
asubmanifoldwhichistransversaltotheorbits). Tosimplifythesolvingprocess,itis
usuallyagoodideatochooseasmanyconstantsaspossibletobezero.
Iftheactionisnotfreebutmerelyregular,wecanstillfindasystemoffunctionally
independentlocalinvariants. Whatwedoisthefollowing. Letsbethedimensionof
the orbitsof G (s < r). We solve the s equationsf (g,x) = c ,... ,f (g,x) = c
1 1 s s
for s of the group parameters and replace them in the remaining equations x =
s+1
f (g,x),... ,x = f (g,x)togetthem sinvariants. Theothergroupparame-
s+1 m m
−
terswillnotappearintheequations. Thisprocedureiscalledapartialmovingframe
normalizationmethod.
Equipped with these theoretical tools, the computation of invariants becomes a
simple systematic procedure. We can thusfeel free to considerany Lie groupaction
imaginable and try to obtain its invariants. As we have seen, in theory, results are
guaranteedprovidedthatthegroupactionislocallyeffectiveonsubsets,whichwecan
alwaysarrangeinthecaseofanalyticgroupactions.
6
3 Application to 3D Shape Reconstruction
Letusthinkforamomentabouttheprocessoftakingapicture.Thisprocessinvolves,
firstof all, the placementof a camerain space. Then, particlesoflightstarting from
theshape(pointsinR3)travelonastraightlineinthedirectionofthecameracenter,
leavingitstraceonafilm, i.e.ontheintersectionofthepictureplaneandthestraight
travelline.Sotothepicture-camerasystemplacedsomewhereinR3,therecorresponds
asetofn straightlinesinR3 representingpathsoflightgoingfromthe objectto the
cameracenter.
ThisprocesscanbeseenasagroupactiongeneratedbyanactionofSE(3)andan
actionofRnonthecameracentertogetherP withtheimagepointsP ,... ,P . The
0 1 n
ideaistoalloweachP tomoveindependentlyalongthelinethroughP andP ,while
i i 0
allowingthelineconfigurationtoberotatedandtranslatedinspace.Asdiscussedinthe
introduction,includingthecameracenteronthespaceactedonbythegroupwillallow
ustoobtainsignificantequivalenceclasses. Thisisthekeytoguaranteetheexistence
ofnon-trivialinvariantsandtheseinvariantscanbeobtainedbyoursystematicmethod.
So, given is a 2D image depicting n points p ,... ,p R2. We assume this
1 n
∈
picturewastakenbyacamerawithfixedinternalparameters.Theseparameterscanbe
calibratedbeforehand,sothatthefocallengthis =11andthe2Dimagecoordinates
F
matchthe 3D coordinatesasdefinedbelow. We embedthe picture-camerasystem in
R3 by setting the camera center to be p˜ = (0,0,0) and the picture points p˜’s to
0 i
be p˜ = p . This is of course, in general, not the actual position in which the
i i
×F
picturewastaken. However,thereexistsarigidtransformationg SE(3)suchthat
∈
g (p˜ ,p˜ ,... ,p˜ ) = (P ,P ,... ,P ) corresponds to the actual position of the
0 1 n 0 1 n
·
picture-camerasystematthemomentwherethepicturewastaken. Inotherwords,if
theobjectismadeofnpoints,say ,... , ,theneachtransformedimagepointP
1 n i
O O
liesonthestraightlinepassingthrough andthecameracenterP . Inordertofully
i 0
O
formulatetheproblemintermsoforbits,wewanttoconsidersmoothtransformations
thatwillmaptheimagepoints(P ,... ,P )totheobjectpoints( ,... , ). For
1 n 1 n
O O
this, we allow each pointP to moveindependentlyalong each ray of light so to go
i
backtoitssourceontheobject.
We would like to determine where P and the ’s lie. Given a picture, it is of
0 i
O
course impossible to determine the points (P , ,... , ). However the “line ar-
0 1 n
O O
rangement”definedbythepicture-camerasystemprovidesuswithimportantinforma-
tion. In particular, we know that the points (P , ,... , ) lie on the orbitof the
0 1 n
Liegroupactionon (P ,P ,... ,P ) R3 (RO3)×(n) dOefinedby:
0 1 n
{ ∈ × }
P¯ = RP +T
0 0
P¯ = R(P +λ (P P ))+T, fori=1,... ,n,
i i i i 0
−
with R SO(3) a rotation, T R3 a translation and λ R, a factor of depth,
i
fori = 1∈,... ,n. Observethatth∈e actionof Rn parameterize∈dbythe λ’s commutes
withtheactionofSE(2)generatedbytherotationRandthetranslationT.Therefore,
1thevalueisarbitrary,itsimplyfixestheoverallscaleofthe3Dreconstruction.
7
thisdefinesafinitedimensionalLiegroupaction,morepreciselytheactionofa(6+
n)-dimensionalLie groupon a (3n+3)-dimensionalmanifold. Note that, although
invariant-basedtechniqueshavealreadybeenusedinthefieldforthegeneralprojective
andaffinetransformationgroups[11,13,12],theLiegroupsweshalldefinehavelittle
todowiththesetraditionaltransformationgroups.
Assumingthatthepicturepointsaredistinct, thenthegroupactionisregularand
the orbitsare 6-dimensional,for n = 1, and(6+n)-dimensionalas soonas n 2.
≥
ThereforebyTheorem2.3,thereare2n 3fundamentalinvariantswhenevern 2.
− ≥
Wenowfollowthestepsofthemovingframenormalizationmethodtoobtainthem.
Weset
P¯ = (0,0,0)T,
0
P¯ (0,1,0) = 0,
1
·
P¯ (0,0,1) = 0,
1
·
P¯ (0,0,1) = 0,
2
·
andP¯ (1,0,0) = 1, foralli=1,... ,n.
i
·
Solvingforthegroupparameters,weobtain
T = RP ,
0
−
R = R R R ,
1 2 3
1 0 0
0 f g
R1 =  √f2+g2 √f2+g2 ,
0 g f
 −√f2+g2 √f2+g2 
 
 √x21+y12 0 z1 
√x21+y12+z12 √x21+y12+z12 (2)
R =  0 1 0 ,
2
 −z1 0 √x21+y12 
 √x21+y12+z12 √x21+y12+z12 
 x1 y1 0 
√x21+y12 √x21+y12
R = y1 x1 0 ,
3  −√x21+y12 √x21+y12 
0 0 1
 
λ =  1 1. 
i (R(Pi−P0))x −
wheref = −y1x2+x1y2, g = z2(x21+y12)−z1(x1x2+y1y2), (x ,y ,z )T = P P and
√x21+y12 √x21+y12√x21+y12+z12 1 1 1 1 − 0
(x ,y ,z )T = P P . Thesegroupparametersdefinea movingframe(MF).Re-
2 2 2 2 0
−
8
placingthemovingframeintothetransformationequations,weget:
P¯ = (0,0,0)T,
0 MF
P¯1(cid:12)(cid:12)MF = (1,0,0)T,
(cid:12) 1
P¯ (cid:12) = f√x21+y12+z12(x1y2−x2y1)+g[z2(x21+y12)−z1(x1x2+y1y2)] ,
2 MF  (x1x2+y1y2+z1z2)√x21+y12√f2+g2 
(cid:12)(cid:12)  0 
 
1
f√x21+y12+z12(x1yi−xiy1)+g[zi(x21+y12)−z1(x1xi+y1yi)]
P¯i MF =  (x1xi+y1yi+z1zi)√x21+y12√f2+g2 .
(cid:12)(cid:12)  g√x21+y12+(xz121x(xi+iyy11−yxi+1yzi1)z+i)f√[zix(21x+21+y12y√12)f−2z+1(gx21xi+y1yi)] 
 
foralli=3,... ,n,where(x ,y ,z )=P P . Eachcomponentofthesevectorsis
i i i i 0
−
aninvariantofthegroupaction.
Letustrytounderstandthegeometricmeaningoftheseexpression. Observethat
f2+g2 = k(x1,y1,z1)×(x2,y2,z2)k. After a few manipulations, we can rewrite the
√x21+y12+z12
pabovesystemas:
P¯ = (0,0,0)T,
0 MF
P¯1(cid:12)(cid:12)MF = (1,0,0)T
(cid:12) 1
P¯ (cid:12) = k(x1,y1,z1)×(x2,y2,z2)k ,
2 MF  (x1,y1,z1)·(x2,y2,z2) 
0
(cid:12)
(cid:12)  
1
P¯ = [(x1,y1,z1)×(xi,yi,zi)]·[(x1,y1,z1)×(x2,y2,z2)] .
i MF  [(x1,y1,z1)·(xi,yi,zi)]k(x1,y1,z1)×(x2,y2,z2)k 
(xi,yi,zi)·[(x2,y2,z2)×(x1,y1,z1)]k(x1,y1,z1)k
(cid:12)(cid:12)  [(x1,y1,z1)·(xi,yi,zi)]k(x1,y1,z1)×(x2,y2,z2)k 
 
Itnowbecomesclearerthatthecomponentsof P¯ and P¯ aresineorco-
2 MF i MF
sineofanglesbetweenthedirectionsspannedbyP P ,P P ,P P andthedirections
1 0(cid:12) 2 0 i (cid:12)0
orthogonal to them. These are clearly invariant by tr(cid:12)anslation, ro(cid:12)tation, and motion
along the projection lines. As a fundamental set, we simply pick the only 2n 3
−
non-constantinvariants:
(x ,y ,z ) (x ,y ,z )
1 1 1 2 2 2
I = k × k
2
(x ,y ,z ) (x ,y ,z )
1 1 1 2 2 2
·
[(x ,y ,z ) (x ,y ,z )] [(x ,y ,z ) (x ,y ,z )]
1 1 1 i i i 1 1 1 2 2 2
I = × · ×
i
[(x ,y ,z ) (x ,y ,z )] (x ,y ,z ) (x ,y ,z )
1 1 1 i i i 1 1 1 2 2 2
· k × k
(x ,y ,z ) [(x ,y ,z ) (x ,y ,z )] (x ,y ,z )
i i i 2 2 2 1 1 1 1 1 1
J = · × k k
i
[(x ,y ,z ) (x ,y ,z )] (x ,y ,z ) (x ,y ,z )
1 1 1 i i i 1 1 1 2 2 2
· k × k
9
fori=3,... ,n.
Each picturetaken definesa pointin R3 (R3)×(n) andthereforedeterminesan
×
orbitofourgroupaction.Eachorbitischaracterizedbythesetof2n 3equationsgiven
−
by the invariants. More precisely, indexing the pictures with the discrete parameter
τ =1,... ,t,wehave
I (Pτ,P ,... ,P ) = ατ, fori=2,... ,n,
i 0 1 n i
J (Pτ,P ,... ,P ) = βτ, forj =3,... ,n.
j 0 1 n j
forappropriateconstantsατ’sandβτ’s. Theseconstantsareprescribedbythepictures:
i j
sincethepicture-camerasystemitselfbelongstotheorbits,wehave
ατ = I (p˜τ,p˜τ,... ,p˜τ)
i i 0 1 n
βτ = J (p˜τ,p˜τ,... ,p˜τ)
j j 0 1 n
Weareinterestedinsolvingtheequations
I (Pτ, ,... , ) = ατ, fori=2,... ,n
i 0 O1 On i
Jτ(Pτ, ,... , ) = βτ, forj =3,... ,n.
j 0 O1 On j
forτ =1,... ,t. For
Wehave(2n 3)t(non-linear)equationswith3n+3tunknowns,thesolutionof
−
whichisdetermineduptoarotationandtranslationofthe3Dcamera-objectsystemas
awhole,whichcancanfixarbitrarily,thuseliminatingsixvariables2. Forn > 3and
t 3n−6, the numberof equationisgreaterthanthenumberofunknownso we can
≥ 2n−6
trytosolvethem.
Experiments with real video images have been performed (see Fig.1 using a se-
quential non-linear optimization technique based on the Levenberg-Marquardtalgo-
rithm[10].Thefeaturepointsusedinthepicturesareendpointsoflinesandrectangles
associatedfromoneimagetothenextwithatrackingprocedure[1].Thereconstructed
3Dobjectisvalidinanyview,evenifthebottomandleftsideelementsarenotperfectly
replaced,duetothetrackingnoise. Thecomputationstakeonlyafewminutes.
Observethatourcamera-systemdoesnottakeintoaccounttheangleofthecamera;
theorientationoftheimageplanewasnotusedinourdescriptionofthecamera-picture
system, Our invariants are invariant under a rotation of the image plane and so we
cannot use them to recover the camera orientation. But this has its advantages, as
illustratedbythefollowinglemma.
Lemma3.1. Themotionofthecamerabetweentwopicturesisapurerotation(i.e.a
rotationaroundthecenterofprojectionP )ifandonlyifthevaluesoftheinvariants
0
I ,J i=2,... ,n,j =3,... ,n evaluatedonanycorrespondingpointsinthetwo
i j
{ | }
viewsareequal.
2However, we should keep in mind that the choice of these variables will affect the numerical
resolution[6].
10