Structure from Motion: Theoretical Foundations of a Novel Approach Using Custom Built Invariants PDF
Preview Structure from Motion: Theoretical Foundations of a Novel Approach Using Custom Built Invariants
Structure from Motion: Theoretical Foundations of a Novel Approach Using Custom Built 2 0 Invariants ∗ 0 2 n Pierre-Louis Bazin1 MireilleBoutin2 a J 1 DepartmentofEngineering, 2 2 2 DivisionofAppliedMathematics, BrownUniversity ] V ProvidenceRI02912,USA. C . s c [ Abstract 1 Werephrasetheproblemof3Dreconstructionfromimagesintermsofinter- v sectionsofprojectionsoforbitsofcustombuiltLiegroupsactions. Wethenuse 9 analgorithmicmethodbasedonmovingframesa` laFels-Olvertoobtainafunda- 1 mentalsetofinvariantsofthesegroupsactions. Theinvariantsareusedtodefine 0 a set of equations to be solved by the points of the 3D object, providing a new 1 techniqueforrecovering3Dstructurefrommotion. 0 2 0 1 Introduction / s c Theconceptofinvarianceisofmajorimportanceinmoderngeometry. Inthefieldof : v computervision,invariantshavebeenusedformorethanadecadeforobjectrecogni- i X tionandreconstruction(seeforexample[7,8]). One particular vision problem one could hope to solve using invariantsis that of r a reconstructinga3Dobjectfromasetofpicturestakenfromunknownviewpoints. For example, given n ordered points , ,... , R3 (the object) and t sets of n 1 2 n orderedpointsp ,... ,p R2 (fOoreaOchoftheOpic∈tures),wewouldliketodetermine 1 n theobjectinR3fromtheim∈ages. Onepossiblewayofsolvingthisproblemwouldbetodefineanequivalencerela- tionbetweenall the possiblepicturesof an objectandto find functionscomputedon the picturesthatare constanton eachequivalenceclass. Characteristicsof the object couldbeinferredfromthesefunctions,regardlessofthecamerapositionrelativetothe object. To define such an equivalence class, one can try to use the orbits of a group action.Realvaluedfunctionsthatareconstantontheorbitsofagroupactionarecalled ∗ThisworkwassupportedbyNSFgrantsKDIBCS-9980091and0074276. 1 invariants. We wouldthusbeinterestedin findinginvariantsofa groupactionwhich wouldbetransitiveonthesetofpicturesofanygivenobject,i.e.“viewinvariants”. Unfortunately, as is commonly known in the vision community, view invariants do not exist for 3D point sets of arbitrary size (in general position). One can still buildinvariantsforspecificobjects(forinstance,planarsetsofpoints,pencilsoflines, etc.) butnotforarbitraryshapes. Theproblemisthatthe setofpicturesofanobject intersectswith the set of picturesof other objects. Observethatif a view invariantI takesaconstantvaluecforallpicturesofanobjectO,thenI isalsoequaltoconthe setofpicturesofanyobjectwhosesetofpicturesintersectwiththesetofpicturesof O. Onecanactuallyshow[2]thatanyequivalencerelationbetweenallthepicturesof eachobjectsdefinesauniqueequivalenceclassonthespaceofpicturesandthus,any viewinvariantistrivial. Fromagrouppointofview,thismeansthatanygroupaction thatistransitiveonthesetofpicturesofanyobjectmustbetransitiveonthesetofall pictures. Another way to solve this problem would be to use the reverse approach: try to characterizeallthepossibleobjectscorrespondingtoagivenpicture. Itwouldbeuse- fultofindfunctionsthatareconstantonequivalenceclassesthatincludealltheobjects corresponding to a given picture. However, it is easy to see that such equivalence classesofobjectsareinonetoonecorrespondencewiththeequivalenceclassesofpic- turesdiscussedearlier,andthusthatonlyonesuchequivalenceclasscanexist. There are thereforeno“objectinvariants”. Nevertheless, givena view ofan object, we can inferinformationabouttheobject,sotheremustbeawaytoovercomethisdifficulty. Itisthefactthatweknowhowacamerabuildsimages(usingtheperspectivepro- jection)thatwillallowustobuildanequivalencerelationwhichcharacterizesobjects correspondingtodifferentpictures.Thekeyistoconsideranequivalencerelationona higherdimensionalspace,sortofliftingthesetofobjectsofdifferentpicturestodiffer- ent“heights”intheextradimensions.Wecandothisusingthethreeextradimensions providedbythecameracenterposition. Moreprecisely,wecanconstructaLiegroup action on the object points ,... , and the camera center P which summarize 1 n 0 O O whatisunknownabouttheobject-camerasystemgivenapictureofanobject. Invari- antsofthisgroupactionsprovetobesufficientforsolvingtheproblemofrecovering theobjectcoordinatesinR3,i.e. the”structurefrommotion”problem. Recentadvancesinthetheoryofmovingframes[3,4]provideuswithasystematic waytoproceedtoobtaintheinvariantsofanygiven(regular)Liegroupaction. Using thissystematicmethod,weobtaintheinvariantsofourcustombuiltactionwithoutany difficulty. Wecanevenventureabitfurtherbymodifyingthegroupactionandrepeat theexercisesotoobtainevenmorenewtoolsfor3Dstructureandmotionanalysis. We begin our exposition with a summary of some of the relevant theoretical as- pectsofthetheoryofmovingframesanditsapplicationtothecomputationof(joint) invariants. In Section 3, we formulate the problem of structure from motion in this framework and obtain the corresponding invariants. We end the section by remark- inghowthisleadstoa simple testtoidentifycameramotionsthatarepurerotations. Section4presentssomevariationsofourapproachwheremorecameraparametersare considered,includingageneralizationtothecaseofavariablefocaldistance. 2 2 Theoretical Foundations LetM beanm-dimensionalsmooth(Hausdorff)manifoldandGbeanr-dimensional Liegroupacting(smoothly)onM. Definition2.1. AninvariantisafunctionI :M Rwhichremainsunchangedunder → theactionofthegroup.Inotherwords, I(g z)=I(z), forallz M andallg G. · ∈ ∈ Alocalinvariantis afunctionI :M RforwhichthereexistsN ,aneighborhood e → oftheidentitye G,suchthat ∈ I(g z)=I(z), forallz M andallg N . e · ∈ ∈ Definition2.2. We say that G acts semi-regularly on M if all orbits have the same dimension. If, in addition, any point p M is surrounded by an arbitrarily small 0 ∈ neighborhoodwhoseintersectionwith theorbitthroughp isconnected,thenwesay 0 thatGactsregularly. The following theorem, due to Frobenius [5], is of central importance to our ap- proach.Aproofcanbefoundin[9]. Itprovidesuswithasimplewayofcharacterizing theorbitsusinginvariants. Theorem2.3(FrobeniusTheorem). IfGactsonanopensetO M semi-regularly ⊂ with s-dimensionalorbits, then p O there exist m s functionallyindependent 0 ∀ ∈ − localinvariantsI ,... ,I definedonaneighborhoodU ofp suchthatanyother 1 m−s 0 local invariant H defined near p is a function H = f(I ,... ,I ). If G acts 0 1 m−s regularly on O, then we can choose I ,... ,I to be global invariants on O. In 1 m−s that case, two points p ,p O are in the same orbit relative to G if and only if 1 2 ∈ I (p )=I (p ),foralli=1,... ,m s. i 1 i 2 − By functional independence of the (smooth) functions I ,... ,I on an open 1 m−s set O, we simply mean that the Jacobian matrix of I ,... ,I has maximal rank 1 m−s m s on an open and dense subset of O. The set I ,... ,I is often called a 1 m−s − { } completefundamentalsetofinvariantsonO. Notethatthecompletefundamentalset ofinvariantsisbynomeanunique. As we shallconsideractionsonpoints, we are interestedinthe case whereM = ... (n-times)=: ×(n)istheCartesianproductofncopiesofamanifold V×V× ×V V . V Definition2.4. WesaythatGactsdiagonallyon ×(n) ifthereexistsanactionofG on such that for any g G, for any n N anVd any z ,... ,z , the action 1 n V ∈ ∈ ∈ V g (z ,... ,z )canbewrittenas 1 n · g (z ,... ,z )=(g z ,... ,g z ). 1 n 1 n · ∗ ∗ The groupactions we will define for our object-camerasystems are not diagonal actions.However,wewillfindanormalsubgroupH ofGsuchthat 3 1. ThesubgroupH canbewrittenasH =H ... H(n-times)whereHisaLie × × groupactingon inaconsistentmannerwiththeactionofH,i.e.(h ,... ,h ) 1 n V · (z ,... ,z )=(h z ,... ,h z ),forallh ,... ,h Handallz ,... ,z 1 n 1 1 n n 1 n 1 n ∗ ∗ ∈ ∈ . Thisinsurethat ×(n)/H canbewrittenas ×(n)/H =( /H)×(n). V V V V 2. TheactionofG/H on ×(n)/H =( /H)×(n) isdiagonal. V V So for all practical purposes, we shall ultimately have to deal with diagonal group actions. Inourapproachtostructurefrommotion,invariantsareusedtoobtainedequations thatmustbesatisfiedbytheobjectandthecamera. Themoreinvariantswehave,the moreequationsneed to be satisfied. We need enoughequationsto completelydeter- minetheobject. Observethatthedimensionoftheorbitisboundedbythedimension of the group. So in the case of a diagonalaction, taking moreand more copiesof V (i.e. moreandmorepoints)allowsfortheexistenceofasmanyinvariantsasnecessary. The question that remains is: how can we obtain an expression for these invariants? Thanks to recent advances in the theory of moving frames [3, 4], this problem can nowbesolvedinanalgorithmicfashion. Wenowsummarizesomeofthistheoryand showhowmovingframescanbeusedasatooltoobtainacompletesetoffundamental invariants. Definition2.5. A(right)movingframeisamapρ:M Gwhichis(right)equivari- → ant,i.e.ρ(g z)=ρ(z)g−1,forallg Gandz M. · ∈ ∈ Unfortunatelymovingframesdonotexistforallgroupactions. Theorem2.6. Amovingframeexistsifandonlyiftheactionifthegroupactionsatis- fies g G z M,g z =z =e, { ∈ |∃ ∈ · } whereedenotestheidentityinG. Thispropertyiscalledfreeness ofthegroupaction. Demandingfreenessofthegroupactionisverystrong. Itappearsthat,inorderto beabletodealwiththegenericcases,weneedtorelaxthisconditionalittlebit. Definition2.7. A local moving frame is a map ρ : M G such that ρ(g z) = → · ρ(z)g−1,forallg N ,aneighborhoodoftheidentitye G,andallz M. e ∈ ∈ ∈ Theorem2.8. A localmovingframe exists ifandonlyif thereexists a neighborhood N oftheidentityine Gsuchthat e ∈ g N z M,g z =z =e, e { ∈ |∃ ∈ · } or equivalently, if and only if for all z M, the dimensionof the orbit throughz is ∈ equal to r, the dimension of G. This property is called local freeness of the group action. 4 Our hope is that, by acting diagonally on more and more copies of a manifold, weshalleventuallyobtainalocallyfreeaction. Soweareinterestedindetermininga simple criterion on the groupaction of G on to determinewhetheror notthis will V workout. Definition2.9. WesaythatGactsonM effectivelyif g G g p=p, forallp M = e . { ∈ | · ∈ } { } WesaythatGactsonM locallyeffectivelyif g G g p=p, forallp M isadiscretesubgroupofG. { ∈ | · ∈ } Many groupsdo not act effectively. However, given G acting non effectively on M,wecanconsiderG˜ = G/G ,whereG = g Gg z = z, z M ,which M M { ∈ | · ∀ ∈ } acts in essentially the same way as G except that it acts effectively. Unfortunately, effectivenessisnotquiteenoughforachievingourgoal. Definition2.10. WesaythatGactseffectivelyonsubsetsofM if,foranyopensubset U M, ⊂ g G g p=p, forallp U = e . { ∈ | · ∈ } { } WesaythatGactslocallyeffectivelyonsubsetsofM if,foranyopensubsetU M, ⊂ g G g p=p, forallp M isadiscretesubgroupofG. { ∈ | · ∈ } Observe that effectiveness on subsets implies effectiveness. The converse is not trueingeneral,butitholdsforallanalyticgroupactions. Theorem2.11. IfagroupGactsonamanifold locallyeffectivelyonsubsets,then forn Nbigenough,theinduceddiagonalactioVnofGon ×(n)islocallyfreeonon ∈ V openanddensesubsetof ×(n). Thisisequivalenttosayingthattheorbitdimension V is equal to the dimension of G on this open and dense subset. We denote by n the 0 minimalintegerforwhichthisistrue. Soatleastforallanalyticgroupactions(modingoutbyasubgroupifnecessary), acting (diagonally)on more and more copies of a manifold will eventually lead to a regular and locally free action on an open subset. A local moving frame will thus exist and provideus with the tools we need to obtain a complete fundamentalset of invariantssocompletelycharacterizetheorbits. Wenowexplainhowtoconstructa(local)amovingframeandtoobtainacomplete fundamentalsetofinvariants. Amoredetailedexpositioncanbefoundin[9,Chapter 8]. Letg = (g ,... ,g )belocalcoordinatesforGinaneighborhoodoftheidentity. 1 r SupposethatGactsregularlyonM. Forsimplicity,letusassumeinadditionthatthe orbitsofGhavethesamedimensionrasGitself.Inotherwords,weareassumingthat the action is locally free. A simple variationallowingus to dealwith merelyregular actionswillalsobeexplainedshortly. 5 Step1:Writedownthegrouptransformationequationsx¯=g xexplicitly. • · x¯ = f (g ,... ,g ,x ,... ,x ), 1 1 1 r 1 m . . . x¯ = f (g ,... ,g ,x ,... ,x ). m m 1 r 1 m Step 2: Choose constantsc ,... ,c R and set r of the transformedcoordi- 1 r • ∈ nates equal to those constants. For simplicity, we relabel the coordinates and write f (g ,... ,g ,x ,... ,x ) = c , 1 1 r 1 m 1 . . (1) . fr(g1,... ,gr,x1,... ,xm) = cr. Theseequationsarecalledthenormalizationequations. Step 3: Solve the normalizationequationsfor g = (g ,... ,g ). The solution 1 r • g =ρ(x)isamovingframe. Step4: Computetheactionofthemovingframeontheremainingcoordinates. • Thesetofresultingfunctions x¯ = I (x ,... ,x ), r+1|g=ρ(x) 1 1 m . . . x¯ = I (x ,... ,x ). m|g=ρ(x) m−s 1 m isacompletefundamentalsetoflocalinvariants. ThechoiceofconstantsinStep2issomewhatarbitrary:wearefreetochooseany numbersforwhichasolutiontothenormalizationequationsexists,providedthatthese constantsdefinea cross-section(i.e.providedthatthe normalizationequationsdefine asubmanifoldwhichistransversaltotheorbits). Tosimplifythesolvingprocess,itis usuallyagoodideatochooseasmanyconstantsaspossibletobezero. Iftheactionisnotfreebutmerelyregular,wecanstillfindasystemoffunctionally independentlocalinvariants. Whatwedoisthefollowing. Letsbethedimensionof the orbitsof G (s < r). We solve the s equationsf (g,x) = c ,... ,f (g,x) = c 1 1 s s for s of the group parameters and replace them in the remaining equations x = s+1 f (g,x),... ,x = f (g,x)togetthem sinvariants. Theothergroupparame- s+1 m m − terswillnotappearintheequations. Thisprocedureiscalledapartialmovingframe normalizationmethod. Equipped with these theoretical tools, the computation of invariants becomes a simple systematic procedure. We can thusfeel free to considerany Lie groupaction imaginable and try to obtain its invariants. As we have seen, in theory, results are guaranteedprovidedthatthegroupactionislocallyeffectiveonsubsets,whichwecan alwaysarrangeinthecaseofanalyticgroupactions. 6 3 Application to 3D Shape Reconstruction Letusthinkforamomentabouttheprocessoftakingapicture.Thisprocessinvolves, firstof all, the placementof a camerain space. Then, particlesoflightstarting from theshape(pointsinR3)travelonastraightlineinthedirectionofthecameracenter, leavingitstraceonafilm, i.e.ontheintersectionofthepictureplaneandthestraight travelline.Sotothepicture-camerasystemplacedsomewhereinR3,therecorresponds asetofn straightlinesinR3 representingpathsoflightgoingfromthe objectto the cameracenter. ThisprocesscanbeseenasagroupactiongeneratedbyanactionofSE(3)andan actionofRnonthecameracentertogetherP withtheimagepointsP ,... ,P . The 0 1 n ideaistoalloweachP tomoveindependentlyalongthelinethroughP andP ,while i i 0 allowingthelineconfigurationtoberotatedandtranslatedinspace.Asdiscussedinthe introduction,includingthecameracenteronthespaceactedonbythegroupwillallow ustoobtainsignificantequivalenceclasses. Thisisthekeytoguaranteetheexistence ofnon-trivialinvariantsandtheseinvariantscanbeobtainedbyoursystematicmethod. So, given is a 2D image depicting n points p ,... ,p R2. We assume this 1 n ∈ picturewastakenbyacamerawithfixedinternalparameters.Theseparameterscanbe calibratedbeforehand,sothatthefocallengthis =11andthe2Dimagecoordinates F matchthe 3D coordinatesasdefinedbelow. We embedthe picture-camerasystem in R3 by setting the camera center to be p˜ = (0,0,0) and the picture points p˜’s to 0 i be p˜ = p . This is of course, in general, not the actual position in which the i i ×F picturewastaken. However,thereexistsarigidtransformationg SE(3)suchthat ∈ g (p˜ ,p˜ ,... ,p˜ ) = (P ,P ,... ,P ) corresponds to the actual position of the 0 1 n 0 1 n · picture-camerasystematthemomentwherethepicturewastaken. Inotherwords,if theobjectismadeofnpoints,say ,... , ,theneachtransformedimagepointP 1 n i O O liesonthestraightlinepassingthrough andthecameracenterP . Inordertofully i 0 O formulatetheproblemintermsoforbits,wewanttoconsidersmoothtransformations thatwillmaptheimagepoints(P ,... ,P )totheobjectpoints( ,... , ). For 1 n 1 n O O this, we allow each pointP to moveindependentlyalong each ray of light so to go i backtoitssourceontheobject. We would like to determine where P and the ’s lie. Given a picture, it is of 0 i O course impossible to determine the points (P , ,... , ). However the “line ar- 0 1 n O O rangement”definedbythepicture-camerasystemprovidesuswithimportantinforma- tion. In particular, we know that the points (P , ,... , ) lie on the orbitof the 0 1 n Liegroupactionon (P ,P ,... ,P ) R3 (RO3)×(n) dOefinedby: 0 1 n { ∈ × } P¯ = RP +T 0 0 P¯ = R(P +λ (P P ))+T, fori=1,... ,n, i i i i 0 − with R SO(3) a rotation, T R3 a translation and λ R, a factor of depth, i fori = 1∈,... ,n. Observethatth∈e actionof Rn parameterize∈dbythe λ’s commutes withtheactionofSE(2)generatedbytherotationRandthetranslationT.Therefore, 1thevalueisarbitrary,itsimplyfixestheoverallscaleofthe3Dreconstruction. 7 thisdefinesafinitedimensionalLiegroupaction,morepreciselytheactionofa(6+ n)-dimensionalLie groupon a (3n+3)-dimensionalmanifold. Note that, although invariant-basedtechniqueshavealreadybeenusedinthefieldforthegeneralprojective andaffinetransformationgroups[11,13,12],theLiegroupsweshalldefinehavelittle todowiththesetraditionaltransformationgroups. Assumingthatthepicturepointsaredistinct, thenthegroupactionisregularand the orbitsare 6-dimensional,for n = 1, and(6+n)-dimensionalas soonas n 2. ≥ ThereforebyTheorem2.3,thereare2n 3fundamentalinvariantswhenevern 2. − ≥ Wenowfollowthestepsofthemovingframenormalizationmethodtoobtainthem. Weset P¯ = (0,0,0)T, 0 P¯ (0,1,0) = 0, 1 · P¯ (0,0,1) = 0, 1 · P¯ (0,0,1) = 0, 2 · andP¯ (1,0,0) = 1, foralli=1,... ,n. i · Solvingforthegroupparameters,weobtain T = RP , 0 − R = R R R , 1 2 3 1 0 0 0 f g R1 = √f2+g2 √f2+g2 , 0 g f −√f2+g2 √f2+g2 √x21+y12 0 z1 √x21+y12+z12 √x21+y12+z12 (2) R = 0 1 0 , 2 −z1 0 √x21+y12 √x21+y12+z12 √x21+y12+z12 x1 y1 0 √x21+y12 √x21+y12 R = y1 x1 0 , 3 −√x21+y12 √x21+y12 0 0 1 λ = 1 1. i (R(Pi−P0))x − wheref = −y1x2+x1y2, g = z2(x21+y12)−z1(x1x2+y1y2), (x ,y ,z )T = P P and √x21+y12 √x21+y12√x21+y12+z12 1 1 1 1 − 0 (x ,y ,z )T = P P . Thesegroupparametersdefinea movingframe(MF).Re- 2 2 2 2 0 − 8 placingthemovingframeintothetransformationequations,weget: P¯ = (0,0,0)T, 0 MF P¯1(cid:12)(cid:12)MF = (1,0,0)T, (cid:12) 1 P¯ (cid:12) = f√x21+y12+z12(x1y2−x2y1)+g[z2(x21+y12)−z1(x1x2+y1y2)] , 2 MF (x1x2+y1y2+z1z2)√x21+y12√f2+g2 (cid:12)(cid:12) 0 1 f√x21+y12+z12(x1yi−xiy1)+g[zi(x21+y12)−z1(x1xi+y1yi)] P¯i MF = (x1xi+y1yi+z1zi)√x21+y12√f2+g2 . (cid:12)(cid:12) g√x21+y12+(xz121x(xi+iyy11−yxi+1yzi1)z+i)f√[zix(21x+21+y12y√12)f−2z+1(gx21xi+y1yi)] foralli=3,... ,n,where(x ,y ,z )=P P . Eachcomponentofthesevectorsis i i i i 0 − aninvariantofthegroupaction. Letustrytounderstandthegeometricmeaningoftheseexpression. Observethat f2+g2 = k(x1,y1,z1)×(x2,y2,z2)k. After a few manipulations, we can rewrite the √x21+y12+z12 pabovesystemas: P¯ = (0,0,0)T, 0 MF P¯1(cid:12)(cid:12)MF = (1,0,0)T (cid:12) 1 P¯ (cid:12) = k(x1,y1,z1)×(x2,y2,z2)k , 2 MF (x1,y1,z1)·(x2,y2,z2) 0 (cid:12) (cid:12) 1 P¯ = [(x1,y1,z1)×(xi,yi,zi)]·[(x1,y1,z1)×(x2,y2,z2)] . i MF [(x1,y1,z1)·(xi,yi,zi)]k(x1,y1,z1)×(x2,y2,z2)k (xi,yi,zi)·[(x2,y2,z2)×(x1,y1,z1)]k(x1,y1,z1)k (cid:12)(cid:12) [(x1,y1,z1)·(xi,yi,zi)]k(x1,y1,z1)×(x2,y2,z2)k Itnowbecomesclearerthatthecomponentsof P¯ and P¯ aresineorco- 2 MF i MF sineofanglesbetweenthedirectionsspannedbyP P ,P P ,P P andthedirections 1 0(cid:12) 2 0 i (cid:12)0 orthogonal to them. These are clearly invariant by tr(cid:12)anslation, ro(cid:12)tation, and motion along the projection lines. As a fundamental set, we simply pick the only 2n 3 − non-constantinvariants: (x ,y ,z ) (x ,y ,z ) 1 1 1 2 2 2 I = k × k 2 (x ,y ,z ) (x ,y ,z ) 1 1 1 2 2 2 · [(x ,y ,z ) (x ,y ,z )] [(x ,y ,z ) (x ,y ,z )] 1 1 1 i i i 1 1 1 2 2 2 I = × · × i [(x ,y ,z ) (x ,y ,z )] (x ,y ,z ) (x ,y ,z ) 1 1 1 i i i 1 1 1 2 2 2 · k × k (x ,y ,z ) [(x ,y ,z ) (x ,y ,z )] (x ,y ,z ) i i i 2 2 2 1 1 1 1 1 1 J = · × k k i [(x ,y ,z ) (x ,y ,z )] (x ,y ,z ) (x ,y ,z ) 1 1 1 i i i 1 1 1 2 2 2 · k × k 9 fori=3,... ,n. Each picturetaken definesa pointin R3 (R3)×(n) andthereforedeterminesan × orbitofourgroupaction.Eachorbitischaracterizedbythesetof2n 3equationsgiven − by the invariants. More precisely, indexing the pictures with the discrete parameter τ =1,... ,t,wehave I (Pτ,P ,... ,P ) = ατ, fori=2,... ,n, i 0 1 n i J (Pτ,P ,... ,P ) = βτ, forj =3,... ,n. j 0 1 n j forappropriateconstantsατ’sandβτ’s. Theseconstantsareprescribedbythepictures: i j sincethepicture-camerasystemitselfbelongstotheorbits,wehave ατ = I (p˜τ,p˜τ,... ,p˜τ) i i 0 1 n βτ = J (p˜τ,p˜τ,... ,p˜τ) j j 0 1 n Weareinterestedinsolvingtheequations I (Pτ, ,... , ) = ατ, fori=2,... ,n i 0 O1 On i Jτ(Pτ, ,... , ) = βτ, forj =3,... ,n. j 0 O1 On j forτ =1,... ,t. For Wehave(2n 3)t(non-linear)equationswith3n+3tunknowns,thesolutionof − whichisdetermineduptoarotationandtranslationofthe3Dcamera-objectsystemas awhole,whichcancanfixarbitrarily,thuseliminatingsixvariables2. Forn > 3and t 3n−6, the numberof equationisgreaterthanthenumberofunknownso we can ≥ 2n−6 trytosolvethem. Experiments with real video images have been performed (see Fig.1 using a se- quential non-linear optimization technique based on the Levenberg-Marquardtalgo- rithm[10].Thefeaturepointsusedinthepicturesareendpointsoflinesandrectangles associatedfromoneimagetothenextwithatrackingprocedure[1].Thereconstructed 3Dobjectisvalidinanyview,evenifthebottomandleftsideelementsarenotperfectly replaced,duetothetrackingnoise. Thecomputationstakeonlyafewminutes. Observethatourcamera-systemdoesnottakeintoaccounttheangleofthecamera; theorientationoftheimageplanewasnotusedinourdescriptionofthecamera-picture system, Our invariants are invariant under a rotation of the image plane and so we cannot use them to recover the camera orientation. But this has its advantages, as illustratedbythefollowinglemma. Lemma3.1. Themotionofthecamerabetweentwopicturesisapurerotation(i.e.a rotationaroundthecenterofprojectionP )ifandonlyifthevaluesoftheinvariants 0 I ,J i=2,... ,n,j =3,... ,n evaluatedonanycorrespondingpointsinthetwo i j { | } viewsareequal. 2However, we should keep in mind that the choice of these variables will affect the numerical resolution[6]. 10
The list of books you might like
