DTIC ADA463028: Stochastic Constraints for Fast Image Correspondence Search with Uncertain Terrain Model

0.3 MB·English

by  Defense Technical Information Center

1
Stochastic Constraints for Fast Image
Correspondence Search with Uncertain Terrain
Model
Michael Veth, Student Member, IEEE, John Raquet, Member, IEEE, Meir Pachter, Fellow, IEEE,
Air Force Institute of Technology
Abstract—Thenavigationstate(position,velocity,andattitude) always been in the interpretation of the image, a difficulty
can be determined using optical measurements from an imaging shared with Automatic Target Recognition (ATR). Indeed,
sensor pointed toward the ground. Extracting navigation infor-
when celestial observations are used, the ATR problem in this
mationfromanimagesequencedependsontrackingthelocation
structuredenvironmentistractableandautomaticstartrackers
of stationary objects in multiple images, which is generally
termed the correspondence problem. This is an active area of are widely used for space navigation and ICBM guidance.
research and many algorithms exist which attempt to solve this Whengroundimagesaretobeused,thedifficultiesassociated
problem by identifying a unique feature in one image and then withimageinterpretationareparamount.Atthesametime,the
searching subsequent images for a feature match. In general,
problems associated with the use of optical measurements for
the correspondence problem is plagued by feature ambiguity,
navigation are somewhat easier than ATR. Moreover, recent
temporalfeaturechanges,andocclusionswhicharedifficultfora
computertoaddress.Constrainingthecorrespondencesearchto developments in feature tracking algorithms, miniaturization,
asubsetoftheimageplanehasthedualadvantageofincreasing and reduction in cost of inertial sensors and optical imagers,
robustnessbylimitingfalsematchesandimprovingsearchspeed. aided by the continuing improvement in microprocessor tech-
A number of ad-hoc methods to constrain the correspondence
nology, motivates us to consider using inertial measurements
search have been proposed in the literature.
to aid the task of feature tracking in image sequences.
In this paper, a rigorous stochastic projection method is
developed which constrains the correspondence search space The methods are typically classified as either feature-based
by incorporating a priori knowledge of the aircraft navigation or optic flow-based, depending on how the image correspon-
stateusinginertialmeasurementsandastatisticalterrainmodel. denceproblemisaddressed.Feature-basedmethodsdetermine
The stochastic projection algorithm is verified using Monte
correspondence for “landmarks” in the scene over multiple
Carlosimulationandflightdata.Theconstrainedcorrespondence
frames, while optic flow-based methods typically determine
search area is shown to accurately predict the pixel location of
a feature with an arbitrary level of confidence, thus promising correspondence for a whole portion of the image between
improved speed and robustness of conventional algorithms. frames. A good reference on image correspondence is [7].
Opticflowmethodshavebeenproposedintheliteraturegener-
allyforelementarymotiondetection,focusingondetermining
I. INTRODUCTION
relative velocity, angular rates, or for obstacle avoidance [4].
IT is well-known that optical measurements provide ex- Feature tracking-based navigation methods have been pro-
cellent navigation information, when interpreted properly. posed both for fixed-mount imaging sensors or gimbal
Optical navigation is not new. Pilotage is the oldest and most mounted detectors which “stare” at the target of interest,
natively familiar form of navigation to humans and other similartothegimballedinfraredseekeronheat-seeking,air-to-
animals. For centuries, navigators have utilized mechanical air missiles. Many feature tracking-based navigation methods
instruments such as astrolabes, sextants, and driftmeters [12] exploit knowledge (either a priori, through binocular stereop-
tomakeprecisionobservationsoftheskyandgroundinorder sis,orbyexploitingterrainhomography)ofthetargetlocation
to determine their position, velocity, and attitude. and solve the inverse trajectory projection problem [1], [10].
The difficulty in using optical measurements for au- If no a priori knowledge of the scene is provided, egomotion
tonomous navigation, that is, without human intervention, has estimation is completely correlated with estimating the scene.
This is referred as the structure from motion (SFM) problem.
Theviewsexpressedinthisarticlearethoseoftheauthoranddonotreflect
A theoretical development of the geometry of fixed-target
theofficialpolicyorpositionoftheUnitedStatesAirForce,Departmentof
Defense,ortheU.S.Government. tracking, with no a priori knowledge is provided in [11]. An
M. Veth, Major, U.S. Air Force, Air Force Institute of Technology, online(ExtendedKalmanFilter-based)methodforcalculating
2950 Hobson Way, Wright-Patterson Air Force Base, OH 45433, USA
a trajectory by tracking features at an unknown location on
[e-mail]
J. Raquet, Associate Professor, Department of Electrical and Computer the Earth’s surface, provided the topography is known is
Engineering, Air Force Institute of Technology, 2950 Hobson Way, Wright- given in [3]. Finally, navigation-grade inertial sensors and
PattersonAirForceBase,OH45433,[e-mail]
terrain images collected on a T-38 “Talon” are processed
M.Pachter,Professor,DepartmentofElectricalandComputerEngineering,
Air Force Institute of Technology, 2950 Hobson Way, Wright-Patterson Air and the potential benefits of optical-aided inertial sensors are
ForceBase,OH45433,[e-mail] experimentally demonstrated in [14].
Manymethodsforsolvingthecorrespondenceproblemhave
Report Documentation Page Form Approved
OMB No. 0704-0188
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington
VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it

does not display a currently valid OMB control number.
1. REPORT DATE 3. DATES COVERED
2007 2. REPORT TYPE 00-00-2007 to 00-00-2007
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
Stochastic Constraints for Fast Image Correspondence Search with
5b. GRANT NUMBER
Uncertain Terrain Model
5c. PROGRAM ELEMENT NUMBER
6. AUTHOR(S) 5d. PROJECT NUMBER
5e. TASK NUMBER
5f. WORK UNIT NUMBER
7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION
Air Force Institute of Technology,2950 Hobson Way,Wright Patterson REPORT NUMBER
AFB,OH,45433-7765
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)
11. SPONSOR/MONITOR’S REPORT
NUMBER(S)
12. DISTRIBUTION/AVAILABILITY STATEMENT
Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES
The original document contains color images.
14. ABSTRACT
15. SUBJECT TERMS
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF
ABSTRACT OF PAGES RESPONSIBLE PERSON

a. REPORT b. ABSTRACT c. THIS PAGE 8
unclassified unclassified unclassified
Standard Form 298 (Rev. 8-98)
Prescribed by ANSI Std Z39-18
2
been proposed in the computer vision literature. A popular
arbmipanelfelgfilitatineownneniresemesi)teiohitonzminmeresoittm.bthhiiiToseealinghsnpteeuehrmsmaee.armolIogdLtitfoesruauelsr,cniqstaoesuhbsfsfamu-orKttraehmadcteytnaaedpnatimiiindfocvfbpeaneaellsrarlfe.yiteeaenxaOncattccesuetoehssnrureed(romrSeefStldefraDtesathtiaco)oeatkunbeoirelrneipatntwtle[eigcn6meaoos]errir,inzirt(tyeehxwismmpfioh−aoveicgnlteydhoe-r ETAPRILPGIONELETA R1 TARGET 1 CORRTESAPROGNEDTE N2C CE OSERARRECSH PZOONNEDENCE SEARCH ZONE
dence algorithms have been proposed which are invariant
to rotations, scaling or both. (e.g., [5]) More robust feature
tracking algorithms are typically computationally expensive EPIPOLE
and a designer must trade tracking robustness and accuracy
TARGET 2
EPIPOLAR
for real-time performance. LINE IMAGE PLANE
This paper proposes an approach to optimize the feature
trackingproblembyexploitingnavigationinformation,derived Fig. 1. Correspondence search constraint using epipolar lines. Given a
projectionofanarbitrarypointinaninitialimage,combinedwithknowledge
from six degree-of-freedom inertial measurements, and prior ofthetranslationandrotationtoasecondimage,thecorrespondencesearch
terrain information, to constrain the correspondence search canbeconstrainedtoanareaneartheepipolarline.Notetheepipolecanbe
spaceandaidtheattendantoptimizationalgorithm.Thetheory locatedoutsideoftheimageplane,asshowninthisexample.
is developed for a kinematic motion model with inertial
sensors.
Strelowalsoincorporatesinertialmeasurementstoconstrain
The paper is organized as follows. Section II explores
the correspondence search between image frames [15]. This
current approaches for constraining correspondence searches
constraint on the image search space is a similar concept to
anddiscussesthestrengthsandweaknessesofsuch.SectionIII
the field of expansion method proposed by Bhanu; however,
poses the statistical projection problem in the most general
Strelow generalizes the approach by exploiting epipolar ge-
terms. Reasonable assumptions are proposed which make the
ometry.
general problem tractable for use with an Extended Kalman
Theprojectionofanarbitrarypointinanimageisdescribed
Filteralgorithm.InSectionIV,themathematicalmodelusedto
by an epipolar line in a second image. All epipolar lines
describe the navigation state and navigation state uncertainty
in an image converge at the projection of the focus of the
is presented. This includes definition of reference frames,
complimentary image. Combining knowledge of the transla-
navigation dynamics, perturbation model, and defines the
tion and rotation between images and the pixel location of a
initial conditions. Section V builds upon the mathematical
candidatetargetinthefirstimage,acorrespondencesearchcan
modeltoderivethestochasticprojectionmethod.Theresulting
then be constrained to an area “near” the epipolar line. This
equations allow the user to predict the pixel location and
approach is illustrated in Fig. 1. Strelow’s method of using
uncertainty of a feature between two images. The stochastic
inertial measurements to constrain the correspondence search
projection method is validated in Section VI using Monte
along an epipolar line is ad-hoc, since the search space is not
Carlo simulation and flight data. Finally, conclusions are
defined statistically.
drawnregardingtheperformanceofthemethodinSectionVII.
InthenextSection,thecorrespondenceproblemisdescribed
usingastochasticmodel.Thismodelisthenusedtodetermine
II. CURRENTCORRESPONDENCECONSTRAINT a statistically-rigorous correspondence search area.
APPROACHES
Exploiting inertial measurements to constrain the corre- III. GENERALPROBLEMFORMULATION
spondence search has been proposed in the literature. In this
The general problem is described as follows. Given a
section, two methods which exploit inertial measurements are
pixel location of a specified landmark at time t , predict the
discussed. i
probability density function of the pixel location of the same
Bhanu and Roberts [2] utilize inertial measurements to
landmarkattimet .Priorinformationregardingthevehicle
compensate for rotation between images and to predict the i+1
navigation state, terrain statistics, and the dynamics of the
focus of expansion in the second image. Once the second
vehicle and landmark are exploited.
image is derotated and the focus of expansion is established,
Mathematically, the pixel location, z(t ), corresponding to
the correspondence between points of interest is calculated i
a landmark at location y(t ) in the scene, is governed by the
using goodness-of-fit metrics. One relevant metric is the i
nonlinear projection function
correspondence search constraint placed on each point. This
constraint ensures each interest point lies in a cone-shaped z(t )=h[x(t ),y(t ),t ] (1)
i i i i
region, with apex at the focus of expansion, bisected by the
line joining the focus of expansion and the interest point in where x(t ) represents the navigation state at the time of the
i
thecameraframeatthefirstimagetime.Whilethisconstraint measurement.
is not statistically rigorous, it does show the value of using The vehicle and landmark dynamics are modeled by the
inertial measurements to aid the correspondence problem. following non-linear Itoˆ stochastic differential equations in
3
white noise notation, x
c
x˙=f[x(t),u(t),t]+G [x(t),t]w (t) (2)
x x
y
y˙=r[y(t),t]+G [y(t),t]w (t) (3)
y y c
where u(t) is a known input function, and w (t) and w (t)
x y
are white noise processes.
A theoretical formulation exists for this general problem,
however two issues make this solution intractable. First, the
measurement observation function is ill-posed (i.e., a unique
inverse does not exist). Second, propagating the conditional
probability density function in time requires solving the for-
ward Kolmogorov (i.e., Fokker-Planck) second-order partial
differential equation for an infinite number of moments [9].
To make the problem tractable, the following reasonable
z
assumptions are made: c
• The prior knowledge of the navigation and target state
can be adequately described as a multivariate Gaussian
distribution. Fig. 2. Camera frame illustration. The camera reference frame
• Additivemeasurementnoiseiszero-mean,Gaussian,and originatesatthecenterofthefocalplane.
white.
• Stochastic process noise is zero-mean, Gaussian, and
B. Vehicle State and Dynamics
white.
• Thenonlinearstatedynamicsandmeasurementequations The vehicle state of interest consists of position (pe),
canbeadequatelymodeledusingperturbationtechniques. velocity(ve),anddirectioncosinematrixofthebodytoECEF
Although not required for tractability, additional assumptions frame (Ceb). From [16], the vehicle state kinematics are
aremadetosimplifythedevelopmentandclarifytheunderly-
p˙e = ve (4)
ing concepts. First, the landmark is assumed to be stationary
v˙e = Cefb−2Ωe ve+ge (5)
with respect to the surface of the Earth (i.e., r[y(t),t] = b ie
0.) Second, the camera is rigidly mounted to the vehicle C˙e = CeΩb −Ωe Ce (6)
b b ib ie b
with known alignment and calibration. Third, the terrain is
where fb is the specific force vector measured by the ac-
described by a statistical elevation model.
celerometers, Ωe is the Earth’s sidereal angular rate vector
In the next section, the relevant reference frames and the ie
in skew-symmetric form, ge is the gravitational acceleration
vehicle dynamics are defined.
vector, and Ωb is the angular rate of the vehicle relative to
ib
the inertial frame in skew-symmetric form and measured by
IV. MATHEMATICALMODEL
the gyroscopes.
A. Reference Frames
In this paper, three reference frames are used. Variables
C. Perturbation Model
expressed in a specific reference frame are indicated using
superscript notation. The Earth-Centered Earth-Fixed (ECEF, The navigation errors are defined as differences from a
or e frame) is a Cartesian system with the origin at the nominal trajectory and are represented as a position error
Earth’s center, the xˆe axis pointing toward the intersection (δpe), a velocity error (δve), and an attitude error ((cid:178)) vector,
of the equator and the prime (Greenwich) meridian, the zˆe defined as:
axis extending through the North pole, and the yˆe axis is the
p˜e(t) = pe(t)+δpe(t) (7)
orthogonalcompliment(inthispaper,acaratsymbol,ˆ,denotes
aunitvector).Thenavigationstateisexpressedintheeframe. v˜e(t) = ve(t)+δve(t) (8)
The vehicle body frame (or b frame) is a Cartesian system C˜e(t) = [I −((cid:178)(t)×)]Ce(t) (9)
b 3 b
withoriginatthevehiclecenterofgravity,thexˆb axisextend-
ing through the vehicle’s nose, the yˆb axis extending through wherethetilderepresentsanominalparameter.Theerrorstate
the vehicle’s right side, and the zˆb axis points orthogonally is modeled as a zero-mean Gaussian random vector
 
out the bottom of the vehicle. The inertial measurements are δpe(t)
expressed in the b frame. δx(t)= δve(t)  (10)
The camera frame (or c frame) is a Cartesian system with (cid:178)(t)
origin at the center of the camera image plane, the xˆc axis
with covariance defined as
is parallel to the camera image plane and defined as “camera
up”, the yˆc axis is parallel to the camera image plane and E[δx(t)δxT(t)]=P (t) (11)
defined as “camera right”, and the zˆc axis points out of the xx
camera aperture, orthogonal to the image plane. where E[·] is the expectation operator.
4
Using perturbation techniques, the dynamics of the naviga- Applying perturbation techniques to the landmark position
tion error states are modeled as a linear stochastic differential function,thelandmarkerror,δye,canbeexpressedasalinear
equation [8] function of the errors of the navigation state, terrain model,
and pixel measurement model
δx˙(t)=F(t)δx(t)+G [x˜e(t),t]w (t) (12)
x x
δye =G δx+G δh+G v(t ) (19)
yx yh yz i
where w (t) is a zero-mean, white Gaussian noise process
x
with covariance kernel where the influence coefficients
(cid:175)
E[wx(t)wxT(t+τ)]=Qx(t)δ(τ). (13) Gyx = ∂∂xg(cid:175)(cid:175)(cid:175) (20)
(cid:175)x˜,h˜,z˜(ti),Cbc,Π
∂g(cid:175)
D. Initial Conditions G = (cid:175) (21)
yh ∂h(cid:175)
At the time of the first image, ti, the navigation error state (cid:175)x˜,h˜,z˜(ti),Cbc,Π
is a zero-mean Gaussian random variable with covariance, ∂g(cid:175)
G = (cid:175) (22)
P (t ). The terrain elevation, h, is a random variable with yz ∂z(cid:175)
mxexan,ih˜, and variance, σ2. The terrain elevation errors are x˜,h˜,z˜(ti),Cbc,Π
h and
assumed to be independent of the navigation errors.
δh=h˜−h (23)
V. STOCHASTICPROJECTIONTHEORY Using the linearized position measurement, the landmark
The theory is divided into three sections: estimating the error is a zero-mean, Gaussian random vector. The land-
initial landmark position and covariance based on the pixel markerrorcovariance,Pyy(ti),andcross-correlationmatrices,
locationofthefeatureselectedinthefirstimage,usinginertial Pyx(ti), are defined as
measurements to propagate this augmented state to the time
P (t ) = E[δyδyT] (24)
yy i
of the second image, and projecting this landmark position
P (t ) = E[δyδxT] (25)
state onto the second image as a probability density function yx i
in pixel coordinates. In simpler terms, these equations allow
Substituting (19) into (24), and noting the independence be-
us to “predict” where a stationary feature should appear in
tween navigation state, terrain, and pixel measurement errors
subsequent images, thus providing a statistical measure to
yields:
constrain our search space within the image.
P (t ) = G E[δxδxT]GT
A. Landmark Error Statistics yy i yx yx
+G E[δh2]GT
The landmark position corresponding to a pixel location is yh yh
a non-linear function of the navigation state, pixel location, +GyzE[v(ti)vT(ti)]GTyz (26)
z(t ), terrain elevation, h, camera to body direction cosine
i Substitutingthepreviouslydefinedcovariancematricesforthe
matrix, Cb, and homogeneous camera projection matrix, Π
c navigation errors, terrain, and pixel measurement yields the
(see [7] for a description):
final form of the landmark position error covariance.
(cid:163) (cid:164)
ye =g pe(t ),Ce(t ),z(t ),h,Cb,Π (14)
i b i i c P (t ) = G P (t )GT +G σ2GT
yy i yx xx i yx yh h yh
The pixel location measurement at time ti is a non-linear +GyzRGTyz (27)
functionofthenavigationstate,landmarkposition,andcamera
parameters: The cross-correlation matrices are calculated in a similar
manner and are expressed as:
z˜(t ) = h[pe(t ),Ce(t ),ye(t ),Cb,Π]
i i b i i c
P (t ) = P (t )GT (28)
+v(t ) (15) xy i xx i yx
i
P (t ) = G P (t ) (29)
yx i yx xx i
where v(t ) is a zero-mean, additive white Gaussian noise
i
process with:
(cid:189) B. State Propagation
R(t ) t =t
E[v(t )v(t )]= i i j (16) Inthissection,thenominalnavigationstate,navigationerror
i j 0 t (cid:54)=t
i j state, and landmark error states are propagated from time t
i
Similarly to the navigation state, the calculated landmark to t .
i+1
position, y˜e, is also modeled as a perturbation about the true The nominal aircraft navigation state is propagated forward
position: based on the non-linear dynamics model given in Equations
y˜e =ye+δye (17) (4-6), typically using a non-linear differential equation solver
(e.g., Runge-Kutta) [13].
and is a function of the calculated trajectory
Thelandmarkerrordynamicsaredefinedasarandomwalk:
(cid:104) (cid:105)
y˜e =g p˜e(ti),C˜eb(ti),z˜(ti),Cbc,Π (18) δy˙ =Gywy(t) (30)
5
where w (t) is a zero-mean, white Gaussian noise process where
y
(cid:175)
with covariance kernel ∂h(cid:175)
H = (cid:175) (43)
E[wy(t)wyT(t+τ)]=Qyδ(τ) (31) zx ∂x(cid:175)(cid:175)x˜,y˜,Cbc,Π
∂h(cid:175)
H = (cid:175) (44)
Thenavigationerrorstochasticdifferentialequationisdefined zy ∂y(cid:175)
x˜,y˜,Cb,Π
in Equation (12) as c
The pixel error covariance, P (t ), is defined as
zz i+1
δx˙(t)=F(t)δx(t)+G [x˜e(t),t]w (t) (32)
x x
P (t )=E[δzδzT] (45)
zz i+1
The navigation and landmark error covariance propagation
Substituting (42) into (45), and eliminating independent error
dynamics are derived using the linearized dynamics mod-
sources yields the pixel location covariance:
els (12),(13),(30),(31) [9]:
P˙xx(t) = F(t)Pxx(t)+Pxx(t)FT(t) P (t ) = H P (t )HT
zz i+1 zx xx i+1 zx
+Gx(t)Qx(t)GTx(t) (33) +H P (t )HT
zx xy i+1 zy
P˙ (t) = F(t)P (t) (34)
xy xy +H PT (t )HT
P˙ (t) = G Q GT (35) zy xy i+1 zx
yy y y y +H P (t )HT (46)
zy yy i+1 zy
An equivalent expression for the time propagation is rep-
Finally, the covariance of the pixel location errors can
resented by the state transition matrix, Φ(t ,t ), which
i+1 i be summarized by combining the equations presented in the
projects the navigation and landmark error covariance from
previous sections:
timet tot [8].Theresultingexpressionforthenavigation
i i+1
and landmark error covariance is Pzz(ti+1)=HzxΦ((cid:90)ti+1,ti)Pxx(ti)ΦT(ti+1,ti)HTzx
ti+1
Pxx(ti+1) = Φ(ti+1,ti)Pxx(ti)ΦT(ti+1,ti) +Hzx Φ(ti+1,τ)GxQxGTx ·
(cid:90) ti+1 ti
+ Φ(t ,τ)G Q GT · ΦT(t ,τ)dτHT
i+1 x x x i+1 zx
ti +H Φ(t ,t )P (t )GT HT
ΦT(t ,τ)dτ (36) zx i+1 i xx i yx zy
i+1 +H G P (t )ΦT(t ,t )HT
P (t ) = Φ(t ,t )P (t ) (37) zy yx xx i i+1 i zx
xy i+1 i+1 i xy i +H G P (t )GT HT
P (t ) = P (t ) zy yx xx i yx zy
yy i+1 yy i
+H G σ2GT HT
+(t −t )G Q GT (38) zy yh h yh zy
i+1 i y y y +H G RGT HT
zy yz yz zy
+(t −t )H G Q GTHT (47)
C. Projection of Uncertainty Statistics onto Image i+1 i zy y y y zy
This equation shows how an initial covariance, P (t ),
The pixel projection function is used to project the naviga- xx i
height uncertainty, σ2, measurement noise (characterized by
tion state and landmark location into the image plane at time h
R), and process noise (characterized by Q and Q ) can be
t . The pixel projection is x y
i+1 projectedtotheimageplaneatalatertime,t ,asexpressed
i+1
z(t )=h[pe(t ),Ce(t ),ye(t ), by Pzz(ti+1).
i+1 i+1 b i+1 i+1
In summary, given the pixel coordinates of a stationary
Cb,Π] (39)
c ground landmark at time t , the predicted pixel coordinates
i
of the same landmark at time t can be described by the
The estimated pixel location error, δz(t ), is modeled as a i+1
i+1
bivariate Gaussian probability density function given in Equa-
perturbation about the nominal pixel location
tion (47). Thus, the correspondence search for the landmark
δz(t )=z˜(t )−z(t ) (40) can be constrained using a statistical confidence threshold.
i+1 i+1 i+1
In the following section, the stochastic projection method is
where the nominal pixel location, z˜(ti+1), is calculated using used to predict the location (and uncertainty) of a stationary
the nominal navigation state and landmark position landmark in an image.
z˜(t ) = h[p˜e(t ),C˜e(t ),y˜e(t ),
i+1 i+1 b i+1 i+1 VI. EXPERIMENT
Cb,Π] (41)
c The experiment validates the stochastic projection method
Perturbing the pixel projection function, the pixel location using both simulated and real data collected from an airborne
error can be expressed as a linear function of the errors of the system. In this experiment, a Northrop T-38 “Talon” aircraft
navigation state and landmark position: wasequippedwithaday-nightmonochromedigitalvideocam-
era synchronized to a Honeywell H-764G Inertial Navigation
δz(t )=H δx(t )+H δy(t ) (42) System.Thecamerawasmountedinthe cockpit,pointingout
i+1 zx i+1 zy i+1
6
AREA OF INTEREST
Fig.3. NorthropT-38instrumentedwithsynchronizeddigitalvideocamera
andinertialnavigationsystem.
the right wing. Flight data were collected in Fall of 2002 at
Edwards Air Force Base, California.
Fig. 4. Simulated flight path. In order to generate a good observation
A Monte Carlo simulation of the test flight is performed
geometry,thecircularorbitwaschosensuchthatafixedterrainpatchremained
to verify the stochastic projection model with respect to a inthecamerafieldofviewthroughouttheflight.
statistically significant sampling of random error contributors.
While this provides an indication of the adequacy of the sys- 30
Estimated 1-σ error bound
tem model, flight test data are used to verify the performance True error sample
of the algorithm in a real-world environment. 20
10
A. Monte Carlo Simulation
senTthedeipnerSfeocrtmioanncVewofasthveersifitoecdhuasstiincgparostjaetcitsitoicnalmlyetrheopdresperne-- Y (pixels)error 0
tative ensemble of sample functions (300 per run). The data
collection system used on the T-38 flights was simulated in -10
software, based on a reference trajectory chosen to generate
aninterestingobservationgeometry.Thisconstant-altitudecir- -20
cularflightpathwasconstructedsuchthatafixedterrainpatch
remainedinthecamerafieldofviewthroughouttheflight.The
-30
simulated aircraft speed was 150 meters-per-second, altitude -2 -1.5 -1 -0.5 0 0.5 1 1.5
X (pixels)
was 2296 meters, and bank angle was 27 degrees which error
described a circular flight path with 4592 meter radius. The Fig.5. Landmarkpixellocationerrorandpredicted2-σboundfor25meter
resulting slant range to the landmark was 5134 meters. The terrainelevationuncertainty.Notetheactualpixellocationerrorsaresimilar
tothepredictederrorbound.Note:XandYaxeshavedifferingscalestoshow
terrain elevation was simulated as a zero-mean random vari-
detail.
able. Simulations were accomplished using a terrain elevation
errorstandarddeviationof25meters,representingamoderate
accuracyterrainmodel.Allsimulationsuseda10secondinter- area developed using a statistical model. This results in faster
val between the first and second image, which was equivalent and more robust correspondence searches.
to 18.7 degrees of arc in the horizontal plane. The simulation
geometry is shown in Fig. 4. B. Flight Data
TheresultsareshowninFig.5.Inthisfigure,thepredicted In this section, the stochastic projection method is imple-
pixel location errors for each Monte Carlo sample function mented using image and inertial flight data collected on the
are represented by a “plus” symbol. The predicted 2-σ pixel T-38 aircraft. The aircraft state dynamics are a function of the
location error bound is indicated by a line. Note the inclined measurements from the strapdown inertial sensors. All states
ellipticalnatureofthe2-σboundisafunctionofthetrajectory areestimatedintheEarth-centeredEarth-fixedreferenceframe
and measurement geometry. previously defined. The error equations were developed based
The same predicted pixel location errors are shown refer- on[16],[17].Forthisexample,athreeimagesequencefroma
enced to a 256×256 pixel image in Fig. 6. The stochastic rightturningprofileisshowninFig.7.Theresultsoftheabove
constraint method shows a small correspondence search area methodforpredictingthefuturetargetlocationanduncertainty
which gives the highest probability of the landmark location. are shown in Fig. 8.
The stochastic constraint method is an improvement over the Thetargetselectedwasthewestcornerofabuildingshown
epipolar line search method as it provides a smaller search in Fig. 7. The estimated target location and 2-σ variance
7
Fig.7. ThreeimagesequenceofanindustrialarearecordedduringaT-38flight,withasamplestationarygroundlandmarkidentified.Image(b)wastaken
1secondafterimage(a).Image(c)wastaken7secondsafterimage(a).Theaircraftisinarightturnapproximately3.8kilometersfromthelandmark.
Base Image (2X Zoom)
a)
(
Image 1 (2X Zoom)
b)
(
Image 2 (2X Zoom)
c)
(
Fig. 8. Predicted landmark location uncertainty using stochastic projection method. The landmark selected was the west corner of a building in the base
image(a),representedbythecrosshair.Usingthestochasticprojectionmethod,thelandmarkmeanand2-σvarianceisprojectedintotwosubsequentimages
todemonstratetheconcept.Theestimatedlandmarklocationandpredicted2-σ varianceforimage(b)showsanellipsoidaluncertaintyafteronesecondof
flight.Image(c)showsafurtherincreaseintheuncertaintyaftersevensecondsofflight.Ineachsubsequentimage,constrainingthecorrespondencesearch
forthelandmarktotheellipsoidalregionreducestherequiredsearchareaandwouldeliminatefalsematcheswithotherfeatureswithasimilarappearance
(e.g.,otherbuildingcorners).
8
128
[8] Peter S. Maybeck. Stochastic Models Estimation and Control, Vol I.
AcademicPress,Inc.,Orlando,Florida32887,1979.
96
[9] Peter S. Maybeck. Stochastic Models Estimation and Control, Vol II.
AcademicPress,Inc.,Orlando,Florida32887,1979.
64 [10] Clark F. Olson, Larry H. Matthies, Marcel Schoppers, and Mark W.
Maimone. Robust stereo ego-motion for long distance navigation. In
32 ProceedingsoftheIEEEConferenceonAdvancedRobotics,volume2,
Y (pixels)error 0 2-σ Stochastic Constraint Ellipse [11] pMaiadegiienrsgP:4a5Tc3hh–ete4rt5h8ar,eneJdu-dnAiemle2ec0n0sPi0oo.nrtaelr.casBee.arIinngPsr-oocnelyedminegassuorfetmheen2ts00f3orAIINAAS
-32 Guidance, Navigation and Control Conference, 2003. AIAA paper
number2003-5354.
-64 Sample Epipolar-based Search Area [12] Meir Pachter and Alec Porter. INS aiding by tracking an unknown
ground object - theory. In Proceedings of the American Control
Conference,volume2,pages1260–1265,2003.
-96 [13] WilliamH.Press,SaulA.Teukolsky,WilliamT.Vetterling,andBrianP.
Flannery. Numerical Recipies in C++ Second Edition. Cambridge
-12-8128 -96 -64 -32 Xerr0or(pixels) 32 64 96 128 UKninivgedrosmity,2P0re0s2s.,TheEdinburghBuilding,CambridgeCB22RU,United
[14] John F. Raquet and Michael Giebner. Navigation using optical mea-
Fig.6. Landmarkpixellocationerrorandpredicted2-σboundfor25meter
surementsofobjectsatunknownlocations. InProceedingsofthe59th
terrainelevationuncertaintyreferencedtoa256×256pixelimage.Notethe
Annual Meeting of the Institute of Navigation, pages 282–290, June
stochastic constraint can limit the correspondence search area significantly
2003.
comparedtoasearchneartheepipolarline.
[15] Dennis W. Strelow. Motion Estimation from Image and Inertial Mea-
surements. PhDthesis, Schoolof ComputerScience, CarnegieMellon
University,Pittsburgh,PA15213,November2004.
shown in Fig. 8 shows an predicted ellipsoidal uncertainty [16] D.H.TittertonandJ.L.Weston.StrapdownInertialNavigationTechnol-
after one second and seven seconds of flight. Note the uncer- ogy. PeterPeregrinusLtd.,Lavenham,UnitedKingdom,1997.
[17] M. Wei and K. P. Schwarz. A strapdown inertial algorithm using an
tainty ellipse increases with flight time, as expected. In each
Earth-fixed cartesian frame. Journal of the Institute of Navigation,
case, incorporating camera motion information can constrain 37(2):153–167,Summer1990.
the correspondence search space significantly. Note the true
landmark location remains consistent with the predicted 2-σ
uncertainty ellipse in the presence of real measurement noise
and terrain model errors.
VII. CONCLUSIONS
In this paper, a stochastic projection method to incorporate
the statistics of navigation dynamics and target motion mod-
els is developed to project the estimated pixel location and
uncertainty of a landmark between two images. The theory is
statistically rigorous. Thus, results derived from simulations
and actual flight data validate the accuracy of the approach
for a number of realistic scenarios.
REFERENCES
[1] HeneleAdams,SanjivSingh,andDennisStrelow.Anempiricalcompar-
isonofmethodsforimage-basedmotionestimation. InProceedingsof
the2002IEEE/RSJIntl.ConferenceonIntelligentRobotsandSystems,
volume1,pages123–128,September2002.
[2] BirBhanu,B.Roberts,andJ.Ming.Inertialnavigationsensorintegrated
motion analysis for obstacle detection. In Proceedings IEEE Interna-
tional Conference on Robotics and Automation, pages 954–959, May
1990.
[3] Espen Hagen and Eilert Heyerdahl. Navigation by optical flow. In
Proceeding of the 11th IAPR International Conference on Pattern
Recognition,volume1,pages700–703,1992.
[4] StefanHrabarandGauravS.Sukhatme. Acomparisonoftwocamera
configurationsforoptic-flowbasednavigationofaUAVthroughurban
canyons. In Proceedings of the 2004 IEEE/RSJ International Confer-
ence on Intelligent Robots and Systems, volume 3, pages 2673–2680,
September2004.
[5] DavidG.Lowe. Objectrecognitionfromlocalscale-invariantfeatures.
InProc.oftheInternationalConferenceonComputerVision,volume2,
pages1150–1157,September1999. Corfu,Greece.
[6] B. D. Lucas and T. Kanade. An iterative image registration technique
withanapplicationtostereovision.Proc.DARPAImageUnderstanding
Workshop,pages121–130,1981.
[7] Yi Ma, Stefano Soatto, Jana Kosecka, and S. Shankar Sastry. An
Invitation to 3-D Vision. Springer-Verlag, Inc., New York, New York,
2004.