Geometric, Kinematic and Radiometric Aspects of Image-Based Measurements

30 Pages·2002·1.7 MB·English

by  Liu, Tiansh

AIAA 2002-3239
Geometric, Kinematic and Radiometric
Aspects of Image-Based Measurements
Tianshu Liu
NASA Langley Research Center
Hampton, VA 23681-2199
22nd AIAA Aerodynamic
Measurement Technology and
Ground Testing Conference
24-26 June 2002/St. Louis, Missouri
I
For permission to cop'_' or reput;iist_, contact t_e American Institute ol Aeronautics and Astronautics
t801 Aie×_nder Bell Drive. SuiteS00 Reston, VA 20191

AIAA 2002-3239
Geometric, Kinematic and Radiometric Aspects
of Image-Based Measurements
Tianshu Liu _
NASA Langley Research Center, Hampton, VA 23681-0001
computer vision, and other scientific and engineering
Abstract
disciplincs have developed various methods that are best
This paper discusses theoretical foundations of
suitable to particular applications in their fields. In
quantitative image-based measurements for extracting and
particular, both photogrammetrists and computer vision
reconstructing geometric, kinematic and dynamic
scientists have studied image-based techniques for many
properties of observed objects. New results are obtained
years to obtain metric and geometric inlormation. The
by using a combination of methods in perspective
approaches developed by photogrammetrists arc more
geometry, differential geometry, radiometry, kinematics
mature and quantitative, which are recently extended to
and dynamics. Specific topics include perspective
non-topographic applications [1]. By contrast, in order to
prqiection transformation, perspective developable conical
deal with more complicated vision problems relatcd to
surface, perspective projection under surface constraint,
artificial intelligence, computer scientists tend to adopt
perspective invariants, the point correspondence problem,
morc versatile mathematical approaches in perspective
motion fields of curves and surfaces, and motion equations
geometry, differential geometry and image algebra [2-5].
of image intensity. The methods given in this paper arc
However, the approaches used by computer vision
useful for determining morphology and motion fields of
scientists are of qualitative nature in many cases and
detormable bodies such as elastic bodies, viscoelastic
generally less accurate than those used in photogrammetry
mediums and fluids.
in metric measurements. Because the objectives of
Table of Contents different disciplines are very different, there is a lack of
sufficient interaction among specialists in various technical
1.Introduction
communities. Perhaps due to different notations, jargons
2. Perspective Prqiection Transformation from 3D Space
and methodologies in these communities, it is difficult to
to 2D Image
transcend the different technical domains and see a unified
3. Projective Developable Conical Surlace Containing 3D
scope of various image techniques.
Curve
From a methodological standpoint, the approaches in
4. Perspective Projection under Surthce Constraint
photogrammetry and computer vision should be integrated
5. Perspective Projection of Motion Field Constrained on
into a universal theoretical framework. Furthermore,
Surface
unlike computer vision scientists who mainly study rigid
6. The Correspondence Problem
bodics, aerospace engineers and scientists often deal with
7. Composite Image Space and Object Space
complex morphology and motion fields of deformable
8. Perspective Invariants of 3D Curve
bodies such as elastic bodies, viscoelastic mediums and
9. Modeling of Imaging System
fluids. It is highly desirable to formulate universal
10. Typical Radiation Processes
theoretical foundations lor quantitative image-based
I1.Reflection and Shape Recovery
measurements of morphology and motion fields of
12.Motion Equations of Image Intensity
deformable bodies. In this paper, we will focus on the
13.Conclusions
geometric, kinematic and radiometric aspects of image-
based measurements. First, we will provide a unified
1.Introduction
treatment of the perspective projection transtbrmation
Image-based measurement techniques play an
from the 3D object space to the 2D image plane and
increasingly important role in virtually all natural sciences
illustrate geometric connections among different
and engineering disciplines since they can provide
formulations of the perspective proiection translormation.
tremendous information and knowledge about observed
Then, we will discusses some specific problems for
objects in a global, non-contact way with high temporal
recovering geometry and motion, such as projective
and spatial resolution. Specialists in photogrammetry,
developable conical surface, projection under surfacc
constraint, reconstruction of motion field on a surface and
motion field of a 3D curve, the correspondencc problem,
Research Scientist. Model Systems Branch. MS 238, Member AIAA
Copyright © 2002 by the American Institute of Aeronautics and and projective invariants. This is an area tor combined
Astrorlautics, Inc. No copyright is asserted in the United States under application of approaches in perspective geometry,
Title 17. U.S. Code. The U,S. Govemment has a royalty-free licensc to
differential geometry, kinematics and dynamics. In the
exercise all rights under the copyright claimed herein for Governmental
purposes. All other rights are reserved by the copyright owner.
radiometricaspectw, e will discussthe fundamental
ax e are modeled and characterized by a number of the
relationshbipetweetnheimageintensitayndradiancferom
anobject.Basedonthisrelationandimposedphysical lens distortion parameters.
constrainttsh,emotion The image and object space coordinates of the points
equations of image intensity will be
are related by the collinearity condition in which the image
derived for typical physical processes such as moving
vector is aligned with the vector from the perspective
Lamertian surface, emitting passive scalar transport, and
center to the object space point
transmitting passive scalar transport. These equations
provide a rational way for reconstructing the geometric
and kinematic properties of deformable bodies like fluids.
-x,:+Sx: =2. M X:-X: (2.1)
In general, the geometric, kinematic and radiometric
approaches are closely coupled. - c X _- X !
where M = [m,j ] is the rotation matrix, )l is a scaling
2. Perspective Projection Transformation from 3D
factor. Algebraic manipulation of Eq. (2.1) yields the
Space to 2D Image
well-known collinearity equations (with the distortion
Image-based measurement techniques extract data
from 2D images and then map them into the 3D object terms 8x s and ax e) relating the point in the 3D object
spacc. There is aperspective relationship between the 3D space to the corresponding point on the image plane,
coordinates in the object space and the corresponding 2D --1
m/( X - X_ ) X
coordinates in the image plane [I, 6-8]. Here, we discuss x' -x_,+gx' =-c =-c--
several tbrmulations of the perspective projection m3r( X - X_ ) -X-3
transformation. Although these formulations are --2 ' (2.2)
re,r( X - X_ ) X
equivalent, one may be more convenient to use than others X2--X_+t_X: =--c =--C._-- i-
for a specific problem. The fundamental geometric m._7( X-X_ ) X
problem in image-based measurements is to determine the where the vectors m t = (m_ ,m_, ,mj¢ )r and
object space coordinates X = ( X i, X "-.X 3)r given the m e = (m:_.me:,me. _)r are the directional cosine vectors
corresponding image (retinal) coordinates x = (x_,x-_)7-. along the x_-axis, xe-axis in the image plane, respectively.
Figure I illustrates the camera imaging process. The lens The vector m_ =(m,j.m,:,m,, )r is normal to the image
of the camera is modeled by a single point known as the
plane, directing from the principal point to the optical
perspective center (or tile optical center), the location of
center on the optical axis. As shown in Fig. 1, the unit
which in the object space is Xc=(X(,X:,X_) r. orthogonal vectors mt, m2,and m_ constitute an object
Likewise, the orientation of the camera is characterized by
space coordinate frame at the optical center X_ and
three Euler orientation angles. The orientation angles and
location of the perspective center are referred to as the _- = (-_, ,_-:,_-3 )r are the projections of the object space
exterior orientation parameters. The object space point. position vector X - X_ in this frame. The elements of the
perspective center and image point lie along a straight line
rotation matrix m_ (i.j = 1.2. 3) are functions of the Euler
for a "'perfect'" camera. This relationship is described by
orientation angles tco,¢._ ),
the collinearity equations, the fundamental equations of
photogrammetry. On the other hand, the relationship m. =cosO coslc, m_: = sinm sinO costc + costa sinlc,
between the perspective center and the image coordinate m_, = - cosm sin¢ cos i¢+sin cosin Ic, m:j = -cos 0 sin lc,
system is defined by the camera interior orientation
m2e = --sin09sin0 sin _ +cos 09cos/c,
parameters, namely, the camera principal distance c and
the photogrammetric principal-point location m2_= cos COsin 0sin IC+ sinCOcos _C,
xt, = (x_,, y_ )r. The principal distance c, which equals ms_ = sin O, m.,, =-sincocosO, m_
= COS(1) COS_).
the camera local length for a camera focused at infinity, is (2.3)
the perpendicular distance from the perspective center to The orientation angles (co.0.t¢) are essentially the pitch,
the image plane, whereas the photogrammetric principal- yaw, and roll angles of the camera in the established
point is where a perpendicular line from the perspective coordinate system in the object space. The rotational
center intersects the image plane. Due to the lens matrix M is an orthogonal matrix having the property of
distortion, however, perturbations to the imaging process
M -t =M r or m[Cms =b'_,. The scaling factor
lead to departures from collinearity that can be represented
by the shifts 8x _ and 6x: of the image point from its 2=-c/m3r(X - X ) is a ratio between the principal
C
"'ideal" position on the image plane. The shifts _x _ and
distance and the projected component of the object space
position vector X-X_ on the optical axis in -m_
directionW. henanobjectspacpeointX is on the focal where X h=(X;,X2,X¢,I)r is the homogenous
coordinates in the object space, and Ph= A-tMh and
plane mj7(X-X )=0, the scaling tactor becomes
C
infinite, i.e., 2 = _, which corresponds to the points at M h=(M -MX_ ) are 3x4 matrices. Although Eqs.
(2.6) and (2.8) arc formally written as a linear relation
infinity on the image (retinal) plane.
between xh and X or X h, they are essentially non-
The terms ax; and ax 2 in Eq. (2.2) are the image
linear because not only the lens distortion is a non-linear
coordinate shifts induced by the lens distortion. The lens
function of x, but also the scaling factor
distortion terms can be modeled by the sum of the radial
distortion and decentering distortion [9- I0] 2=-c/mJ(X-X ) is not a constant in general.
• C
a.r'=ax[ + ax,5 and ax e=a.r_ + a.r_, (2.4)
Nevertheless, because the lens distortion is usually small,
where, assuming that the optical axis of the lens is its effect can be corrected by using an iterative scheme.
perpendicular to the image plane, we have Hence, Eqs. (2.6) and (2.8) can be treated as a quasi-linear
a.'hI = K;( ";_' --"r;I' )re + K,( x1,-x;,; )i.4 , system at each iteration. Without the lens distortion, the
a.r =K,(,,-:'-.q,)r"+X:(.r:'-.q,J," collinearity equations describe the ideal perspective
projection. Eq. (2.8) is particularly suitable for utilizing
ax5 = e,[ ,.2+ 2(x;,_xl ' )2]+ 2¢( x;,_xl ' )(x2,_.r_, ), useful results of classical perspective geometry to
construct projective geometric invariants.
a,r_ = _ / r2+2( xe'-x_, )2]+ 2P,( x"-x_, )(x2'-.r_ )
Furthermore, Eq. (2.2) can be re-written as a form
,.: =( x"-xl, )"+( .r:'-x_, ):. (2.5) suitable to least-squares estimation for the object space
Here, K; and K2are the radial distortion parameters, P; and coordinates X,
P2 arc the decentering distortion parameters• and .r_' and W;; ( X-X_ )=0
(2.9)
x-" arc the undistorted coordinates in image. When the W2r(X-X_ )=0
lens distortion is small, the unknown undistorted
where W_ and We are defined as
coordinates can be approximated by the known distorted
W_ =( x_-.ri,+a.r _)m3 +cm_
coordinates, i.e., x;'= x; and x:'= x2. For large lens (2.10)
distortion, an iterative procedure is employed to determine W, =( x2-x_,+ax 2)ra, +cm 2
the appropriate undistorted coordinates to improve the As shown in Fig. 2. the vector W_ is on the plane spanned
accuracy of the estimate. The following iterative relations
by the orthogonal unit vectors m_ and m,, while W2 is
are used: (x;')° = x; and (x2, )o= x2,
on a plane spanned by m e and m 3. Geometrically
(x_,)_+;=x; +ax; l( x;' )k,( x2' )k ] and
speaking, W/( X-X_ )=O and W2r(X-X_ )=O
(x2, )k+; = x2+ax e[( x;, )k,( x2, )*] , where the
describe two planes normal to Wt and We through the
superscripted iteration index k is k--0, 1,2..-.
optical center. Thus, Eq. (2.9) defines an intersection of
The collincarity equations Eq. (2.2) can bc re-written
these two planes, which is a line through the optical center
in the homogenous coordinates in the image plane
X_. For a given image point x =(x;,x: )r Eq. (2.9) is
xh= (.rj,x: ,x: )r = (xI,.r2 ,1)r
not sufficient to determine a point in the object space with
Ax h =2MfX-X_) or Xh=2P(X--X:), (2.6) the three unknown coordinates X=(X;.Xe,X')r.
where P=[pii]=A-IM and A=[a,j] isdcfined as Hence, extra equations associated with additional cameras
and other geometrical constraints should be added for
'1 0 -x;,
seeking a unique least squares solution of X . In contrast
A= 0 1 -x_+ax 2 (2.7) to Eq. (2.8), Eq. (2.9) does not include the scaling factor
I ";+ ax;
0 0 -c /t.
The collinearity equations Eq. (2.2) contain the
The terse tensor form of Eq. (2.6) is
camera parameters to be determined by geometric camera
aiixil = 2m,/X j -Xf ), where the Einstein convention calibration. The parameter sets (oo,O,_',X/,X,e,X;!),
for summation is used. The matrix-form and tensor-torm
(_v,x, ;,xT_ ), and (K;, K,,_PI,P_)_ in Eq. (2.2) are the
of the collinearity equations are sometimes convenient for
exterior orientation, interior orientation, and lens distortion
mathematical manipulation. Another alternative form of
the collinearity equations in the homogenous coordinates is parameters of a camera, respectively. Geometric camera
calibration is a key problem in quantitative image-based
(2.8)
Xh = /_Ph Xh'
measurements and a specific topic in both photogrammetry
andcomputevrision.Hereweonlybrieflyaddrestshis
curve lies can be reconstructed. When two calibrated
issueandreaderscanfind the technicadletailsof
cameras are used. the 3D curve can be uniquely
geometriccameracalibrationfromreferencesI.n this
determined as an intersection of two different projective
paperw, egenerallayssumtehatthecameraiscalibrated
conical developable surfaces. Furthermore, a 3D surface
andacompletseetoftheorientatiopnarametearnsdlens
can be reconstructed as an envelope of a family of the
distortion parameters of the camera
projective developable conical surfaces obtained from
(ogO,,I¢,X,(,X,:,Xi_,c,x_,,x_,K,,K2,P_,P: ) is known. images taken at different viewing angles. The motion field
Analytical camera calibration techniques utilize the of the 3D curve can be obtained from a time sequence of
the curve.
collinearity equations and distortion terms to determine
Generating Proiective Developable Conical Surface
these camera parameters [6-8]. Since Eq. (2.2) is non-
linear, iterative methods of least squares estimation have Consider a 3D simple curve C in the object space, and
its proiection to the image plane and a plane P normal to
been used as a standard technique for the solution of the
the optical axis (parallel to the image plane), as shown Fig.
collinearity equations in photogrammetry. However.
3. The coilinearity equations Eq. (2.6) are written as
direct recovery of the interior orientation parameters could
be problematic and unstable since the normal-equation- X- Xc = 2-J-fi Xh, (3.1)
matrix of the least squares problem is nearly singular. The
where -fi=p-t =[Pi/I=M-tA=MrA" When the
singularity of the normal-equation-matrix mainly results
from strong correlation between the exterior and interior camera parameters and the scaling factor are constant and
orientation parameters. In order to reduce the correlation the lens distortion is fixed, differentiating Eq. (3.1) yields
between these parameters and enhance the determinability dX = 2-IP._2 dr . (3.2)
of (c,xp,yp). Fraser 19, 1I] suggested thc use of multiple where dX =( dX I,dX :.dX -_)r. dr =(dr z dr: )r .and
camera stations, varying image scales, different camera
roll angles and a well-distributed target field in three I-PH -PJ:
dimensions. Nevertheless, the multiple-station, multiple-
image method Ibr camera calibration is not easy to use in
L:
many engineering and scientific applications like wind
tunnel testing where optical access tbr cameras is limited
A constraint imposed on Eq. (3.2) is mJdX=O,
and the positions of cameras are fixed. AbdeI-Aziz and
Karara 1121 proposed a simple linear method for camera indicating that Eq. (3.2) actually describes the projection
calibration, Direct Linear Translbrmation (DLT). Cp of the 3D curve C on the plane P orthogonal to the
Scientists in computer vision and robotics have developed optical axis direction or m._. This constraint is equivalent
various camera calibration schemes to achieve a fast to the constancy condition of the scaling factor
calibration with an acceptable accuracy (a lower accuracy
2 =-c/m_T( X - X ) since the differential
for a photogrammetric application). Tsai's two-step C
method [13] is representative in computer vision, which dA = cm_rdX (cid:0)(cid:0)m J( X - X, )1: shows
uses a radial alignment constraint to obtain a linear least
m_rdX = 0 ¢:_d2 = 0. In fact, the constraint
squares solution tor a subset of the calibration parameters,
whereas the rest of the parameters including the radial 2=-c/m_rfX-X_ ): const, defines the plane
distortion parameter are estimated by an iterative scheme.
orthogonal to the optical axis direction or m3. As shown
By circumventing the singularity problem. Liu et al. [141
in Fig. 3, the projected curve Cp on the plane P can be
developed a robust optimization method for single-image,
automatic camera calibration to determine the interior and reconstructed from the image and then the developable
exterior orientation parameters and lens distortion conical surface D containing the 3D curve C can bc
generated.
parameters plus the pixel spacing ratio.
The arc length element of the projected curvc Cp on
the plane P is
3. Projective Developable Conical Surface Containing
3D Curve dSc_ = IdX I= 2-I I-fi._:t Ids, (3.3)
In this section, we introduce the concept of proiective
where t =dr/ds and ds =ldxl are the unit tangent vector
developable conical surface and show how to reconstruct
and arc length element of the image of the 3D curve C in
this surface containing a 3D curve from asingle image. In
the image plane, respectively. Thus, the unit tangent
principle, a 3D curve in the object space cannot bc
vector of the projected curve Cp on the plane P is
completely recovered from a single image since
information in one dimension is lost in the imaging dX P._:t
Tc, - - -- (3.4)
process. Nevertheless, using a calibrated camera, a " dS,-k IP_: t I
projective conical developable surface on which a 3D
tangent plane on the developable conical surface D is
Note that the unit tangent vector To,' is independent of the
given by
scaling factor 2'. The curvature vector of the projected (X-X_)oNo(s)=O, (3.1 I)
curve Cp on the plane P can be obtained by differentiating
where ND(s)=Tcp×(Xc, ,-X_)/ITcP×(Xc e-X_)l
Eq. (3.4) with respect to the arc length Sc+'
is the unit normal vector to the tangent plane on the
dT¢,, _ 2 -- d t-P3,_t I developable surface, which is independent of the scaling
(p,:k-Tce --), (3.5)
Key - dScv [P.¢2t 12 ds factor. Eq. (3.11) describes a single-parameter family of
the tangent planes where the parameter is the arc length s
where k =dt/ds = d:x/ds 2 is the curvature vector of the
of the curve in the image plane. The projective conical
curve image in the image plane. The curvature vector k developable surface, the envelope generated by the family
can be expressed as k=_n, where Ic and n=k/Ikl of the tangent planes, is given by a system of Eq. (3.11)
are the curvature and the unit normal vector of the curve and Eq. (3.12)115]
image in the image plane, respectively. Furthermore, we ( X- X_ ).dNo( s)/ds =O. (3.12)
prove Thus, the projective developable conical surface and
d IPc: t l_ (P._: ) (3.6) associated geometric quantities such as the curvature,
k )T(-P32 t
ds IPs: t I tangent vector and normal vector in the 3D object space
can be obtained by using measured image quantities given
Hence, Eq. (3.5) becomes
the camera parameters.
2' -- (P_: n )r( P__2t ) ] Reconstructing 3D cuta,e and Surface
Kc,, =__I_P .¢2t I2 [p_:n-Tc '' t-P._:t I . (3.7) From a single image, we arc able to reconstruct the
proiectivc conical developable surface containing the 3D
The curvature of the projected curve Cp on the plane P is
curve C rather than the 3D curve itself. Nevertheless,
_cp =Kcv'Nc,," where Nee =Kcr/IKc, ' I is the when two calibrated cameras are used, as shown in Fig. 4,
principal normal vector of the projected curve Cp. Thus. the 3D curve Ccan be uniquely determined by intersecting
the ratio between the curvatures _'c,, on the plane P and the two projective developable conical surfaces associated
with the different cameras. Interestingly, the developable
_" on the image plane is
conical surface intersection method for determining the 3D
K'c,__=L"2 curve only requires knowing the correspondence of one
r_ Ip._:tl 2 [P_2n-Tc" (-P_:n)r(P_:tt)]l'Nc''lPe-- distinguished point such as an end point of the curve.
Furthermore, the developable conical surfaces can be
(3.8)
used to reconstruct a 3D surface in the object space. As
Clearly, Eq. (3.8) indicates that ,vc,'/t¢ is proportional to
shown in Fig. 5, the developable conical surface
the scaling factor 2,. containing the contour of the 3D surface can be
After the unit tangent vector To," is obtained from the constructed. Here the contour is a set of points on the 3D
surface at which the surface normal is also the normal of
image, the projected curve Cp on the plane P is readily
the developable conical surface. When the camera is
reconstructed by
moved to a number of known positions through a
rotational and translational transformation (rigid-body
Xc_, = Xcpo + I ,S(p Top(Sop)dSc_ . (3.9)
motion), a family of the developable conical surfaces can
The initial position Xo, . on the projected curve Cp, in the be obtained. The 3D surface is generated as an envelope
of the family of the conical surfaces. Instead of moving
object space is often chosen at the end point of the curve.
the camera, the 3D surface can be rotated around a fixed
Eq. (3.1) gives Xce,, - X _= 2,-JP xho , where axis such that a family of the conical surfaces can be
Xho =(X/,.X_,I) r is the homogenous coordinates of the obtained using a camera at a fixed position and viewing
angle. From a computational viewpoint, this method may
corresponding image point to Xcr .. Substituting Eqs. not be the most efficient since the intersection and
(3.3) and (3.4) into Eq. (3.9) yields a ray vector directing envelope of the developable conical surfaces has to be
from the optical center X_ to a point Xc, , on the determined. However, this method is to great extent
immune from the ambiguous correspondence problem in
projected curve Cp
stereovision.
Xc_, -X_ =2,-I(Pxho P¢,_tds). (3.10) Recovering Motion Field o[3D Curve
-- .+. Jl'*_ --
0 Alter two or more 3D curves in the object space at
A family of the projective rays through the optical center successive instants are reconstructed, we can estimate the
X c given by Eq. (3.10) generates a projective developable motion field U( X ) of the 3D curve that is defined as
conical surface D that contains the 3D curve C. The
dX the curvature K of the filament along the binormal
U( X )=-- (3.13)
dt direction vector B [17]
The curve is given by X = X[S(t),t], where t is time U( X )o¢ _B. (3.19)
and S( t ) is the arc length of the curve in the object space. Overall, the physical constraints tbr a specific application
are necessary for recovering the correct motion field and
Measurements give the temporal and spatial difference
associated physical properties of the 3D curve.
between two curves at two successive instants t¢ and t,
(the time interval At = t: -t_ is small) 4. Perspective Projection under Surface Constraint
As,X =XIS(t: ),t:l-X[S(t I ),t_]. (3.14) In general, mapping between a point in the 3D object
Reconstruction of the motion tield of the 3D curve from space and the corresponding image point is not one-to-one.
Nevertheless, as shown in Fig. 6, under a given surface
AstX is a non-trivial problem since the point
constraint, a point on the surface has the one-to-one
correspondence between two sequential images is not
correspondence to the image point. In this section, wc
known without using distinct targets on the curve
discuss the geometric relationship between the surface in
especially for an elastic curve experiencing large and
the object space and the image plane. This topic is closcly
complicated deformation.
related to some applications in experimental fluid
The motion field of the curve is constrained by the
mechanics and aerodynamics such as reconstruction of
underlying physical mechanisms behind the motion and
complex Ilow topology from images of surface oil
deformation of the curve. In general, reconstructing the visualization and laser-sheet-induced fluorescence
motion field is fornmlated as an optimization problem of
visualization. Consider a surface in the object space given
the functional
by
J[U( X )]=IIA._,X-U( X )AtlI---_ min (3.15)
X _= F( X I, X2 ). (4.1)
subject to relevant physical and geometric constraints
When Eq. (4.1) is imposed on Eq. (2.9) as a surface
G,[U(X)]=O, (i=1.2,...) (3.16)
constraint, the perspective projection transformation Eq.
and the suitable boundary conditions. Without the (2.9) isreduced to
sufficient constraints, the solution to the optimization
(wHw:_-- wl._wel )Xt +(w_:w:_ - wl._w,_,_)X "
problem may not be unique. Also, the imposed physical
constraints serve as a bridge connecting image-based = w,._W/r X, - n'l._W2zX,
measurements with the physical quantities in a specific
w/iX_ +wl:X -"+wI._F(XI.X: )= W/rX . (4.2)
problem being studied.
where wj (i = 1,2 and j = 1,2,3 ) arc the elements of the
In the simplest case in which the curve is rigid, the
rigid-body motion field is expresscd as vectors Wt =( w_.wj:,wj, )7 and W2=(w:j,w::,w:_)r.
U( X )=U o+ _,x( X-X o), (3.17)
For the given surface equation X _= F( X _,X'- ) and the
where Uo and .(2o are the constant translation velocity
known camera parameters, the coordinates (X _,X: )r
and angular velocity, respectively, and Xo is the rotational
can bc obtained from the image coordinates x =(x _,x: )7
center of the curve. Because Uo and .(2, together contain
by numerically solving Eq. (4.2). Thus. the coordinates
only six unknown constants, it is easier to solve the
X=(X_,X-',X -')7- in the object space can be
optimization problem. A slightly complicated case is that
the curve is stretched in three fixed directions in addition symbolically expressed as a function of the image
to the constant translation and rotation. In this case, three coordinates x = (xj,x-' )7, that is,
stretching constants are added, and thus the total number X =fs(x). (4.3)
of the unknowns in the optimization problem is nine.
In fact, Eq. (4.3) is a parametric representation of the
Next, we consider a highly deformable material line
surface using the imagc coordinatcs x =(x_,x-' )r as thc
convected in an incompressible and irrotational flow. In
this case, the physical constraints arc the solenoidal and parameters. Generally, the function fs(x) cannot be
irrotational conditions [16] written as a closed-lorm solution except in some special
V.U( X )=O and VxU( X )=O. (3.18) cases such as a plane and a cylindrical surface.
A vortex-filament in an incompressible and irrotational Differentiating Eq. (2.9), we have
flow isan interesting example since the filament driven by dW/X + W/dX = dW/X_
not only mean flow, but also self induction is no longer
dW 2rX + W27dX = dW 2rX, . (4.4)
passive and the motion field is directly related to the
geometric features of the filament. In this case, the When the lens distortion is fixed, dWtT= dx_ms r and
induced motion velocity of the filament is proportional to
dW2 r = dx2m_ r hold. Then, substitution of
curve on the image plane given by a parametric form
dX ._=(OF/OX I)dX 1+(OF/OX 2 )dX 2 into Eq. (4.4t
x(t)=(xJ(t),x:(t)) T and the corresponding 3D curve on
yields
the surface X(t)=X(x(t))=fs(x(t)), where t is a
parameter (e.g. time). The length of an arc bounded the
dX '-
points corresponding to the parametric values t= t,, and
where
t=t I is
o=(Wll +w_._3F/OXI wI, +wI_OF/OX 2
_w,i + w_,._3F/OX I w,_:+ w2_3F/OX: (4.6) S= ["[g,_/3(dx"/dt)(dx_/dt)] _/2dt. (4.15)
ql h
Furthermore, the differential dX J can be expressed as a The angle of two 3D curves at the intersecting point on the
function of the image coordinatcs dx = (dx I,dr"_)T surface can be calculated based on the image quantities.
dX -_=(dF/dx I )dx I+(dF/dr'- )dx:, (4.7) Consider two image curves x(t)=(x_(t),x2(t)) r and
where x( t) = (xJ+(t), x"+(t))r • The tangential vectors of the two
dF OF OXJ OF OX: 3D curves on the surface are
- + .(ct = 1,2 ) (4.8)
dx a OXI Ox" OX : Ox'_ dX (.rJ(t),x: (t))/dt = OX/&r_dx" / dt and
Combining Eqs. (4.5) and (4.?), we have
dX(x_*(t),x-'*(t))/dt = OX/Ox_+d__+ / dt. Thus, the angle
dX = m.,r( X_ - f s )Qdx (4.9)
yof intersection is
where
g_/j(dr"/dt)(dr/_+/dt)
cos 7= _]g_/_(dr_,/dt )(dx, /dt )_/g,tfldx,,+/dt )(dx /_./dt )
-Q= ( ddtF.i ' ddxF2)/Qm-_Ir(.X¢-fs) _. (4.10) (4.16)
The area of a domain H on the surface can be expressed in
Eq. (4.9) provides a fundamental relation between the
the image coordinates
differentials dX on the surface and dr on the image
plane. The matrix Q is a lunction of the image A(H) = [[,_-ff dx Idx:, (4.17)
coordinates, the camera parameters and the geometric U
where U is the domain in the image (x_,x :) plane
properties of the given surface.
On the other hand, we notice corresponding to the domain H on the surface in the object
dX =(OX/_x I )dx j+(OX/Ox 2)dx2" (4.11) space and g is the determinant g =1ga/J I.
From Eqs. (4.9) and (4.11), we obtain the following Example I: Plane
equality The plane constraint is a simple, but very useful case
(OX/OxI.OX/Ox "-) =mjr(x_ -fs )Q" (4.12) in which the vector function fs(x ), the matrices Q and
The element dS of the arc length of a curve on the surface can be explicitly expressed as a function of the known
can be determined from Eqs. (4.11) and (4.12) from the camera parameters and the measured image coordinates.
image coordinates. We know Many aerodynamic flow structures are observed on a plane
dS: = IdX I= g,,l_dr"d_ "_, (4.13) or a near-planar surface. Planar laser sheet Iqow
visualization is just a typical case of the plane constraint.
where
OX OX In addition, a polyhedron consists of a number of the
g_ = Ox,------O7.x_ (c_,fl = 1,2 ) (4.14) planar faces. Consider a plane in the object space
is the so-called metric tensor in classical differential X _=aiX I+a,x:+a_. (4.18)
geometry [I81. The summation convention is used in Eqs. This plane is defined by the vector a =(a I,az,a_)r
(4.13) and (4.14). The quadratic differential form Eq. related to the normal vector of the plane. In this case. the
(4.13) is the first fundamental form of the surface in which matrix Q in Eq. (4.6) is
the image coordinates are the parametric variables. In the
case of the perspective projection transformation, gaff
Q=I w''+w'-_a' w'-'+w'-_a" )-. (4.19)
1,1'21 + W2d a l w22 + W23 a2
may be properly named as the perspective metric tensor
The function fs(X) in Eq. (4.3) has a closed-form
that is a function of the image coordinates, the camera
solution
parameters, and the properties of the given surface.
The first fundamental form Eq. (4.13) allows us to
measure the basic geometric quantities on the surface in
the 3D object space from the image quantities. Consider a
where
Q-s! /
//Q,,,)j
(4.20)
= . (4.27)
a T Q (w,2P,,c°sq_-ws,p,,sinq) ws3 1
(%:p,, cosec- I%p,, sin _o w2_
where Another differential is dp =0. Note that the expressions
( T of fs(X), Q and Q lbr a spherical surface can be also
i= Ws Xc - Irr_a_
(4.21 ) analytically derived, but they are so tedious that we do not
T
W2 X c - w:j ax
present them here.
Now the matrix Q in Eq. (4.10)is
5. Perspective Projection of Motion Field Constrained
on Surface
= (4.22)
O-'
(aj. a_,)Q-S After discussing the geometric relationship between a
surface in the object space and the image planc, we study
Example 2."Cylindrical Surface
kinematics under the surface constraint, that is, the
A cylindrical surface isanother case where fs (x), Q
perspective projection of a motion field on a surface.
and Q can be cxplicitly expressed. For the sake of Consider a dynamical system
convenience, a transformation from the Cartesian dX
i=U( X ), (5.1)
coordinate system to the cylindrical coordinate system is dt
used, i.e..
where U( X ) =(Us.Ue,U +)r is a motion field in the 3D
X=( X_,X".x_ )r=(pcos_o, psinq),z) T, (4.23)
object space and t is time. A surface constraint imposed
where p is the radial coordinate, (p is the polar angle, and on the motion field Eq. (5.1) is
Z is the axial coordinate. The differential dX is X 3= F( XI.X 2). (5.2)
Under this surface constraint, U( X ) should be parallel to
(,'os_ - p sin _ Oy dp ]
the surface, which obeys the orthogonality condition
dX =lsin _ pcos_ 0 Ida1. (4.24)
N,. .U( X )=O. (5.3)
t0 0 S X<I: )
where N =(3F/OXS,OF/aXe,-I) r is the normal
For a cylindrical surface constraint P CO#1SI.,
= PC =
vector of the surface. Under the surface constraint Eq.
solving Eq. (2.9) lot ¢,0 and z, we have fs(x) as a
(5.2), Eq. (5. I)is effectively reduced to a 2D system
function of the image coordinates and camera parameters
f s(x )= (p,, cos _0,p, sin q),z)r , (4.25)
where X 2 U, IX_ X: FfXS.X2)]
b/,, +xlt, +b:- b;b; In fact, Eq. (5.4) describes an orthographic projection of
COS _ =- I_f+b7 the motion field Eq. (5. I) onto the plane (X s,X : ). From
Eq. (4.5), the dynamical system in the image plane, which
+_4t,7t, +b/- t,,<
is corresponding to Eq. (5.4). is
sin _o=
(5.5)
"= w_(( wnp,, cos_+ %z p,, sin _-W, rX, ), u=--_t( x-' m.,r(X,-fs) U:lfs (x)] o
bs = Po( wHwex - ll':s%._ ), We call u =dr/dt =d/dt(xS,x: )r the optic flow in the
b, = p (%_.W_v- w_,_u,'/._). image plane. The oplic lqow, a term first used in computer
vision, is defined as the velocity field in the image plane
b_=u' vl_lz TX_-%+W, TX,.
that translorms one image into the next image in a
There are two solutions for fs(x ), which are sequence. If Eq. (4.2) gives a one-to-one topological
corresponding to two intersecting points between a mapping (homeomorphism): fxs,x_)_ (X _,X : ), the
perspective ray and the cylinder. For a non-transparent topological structure of the dynamical system Eq. (5.5) in
solid surface, a camera only sees one intersecting point at
the image plane is equivalent to that of Eq. (5.4) on the
thc surface lacing the camera and hence fs(X ) is one-to- surface in the object space when Q has the lull rank of 2
one. The differentials in the cylindrical coordinate system and m3r(x< -fs ) is not zero. Figure 6 illustrates this
are related to the image coordinate differentials by the
point. The problem is to recover two components of the
|ollowing relation
motion field (U_. U2)r using Eq. (5.5) from the measured
d( (0,z)r = m iT( X c - f s )Q-J dr , (4.26)