Analysis and Design of Panoramic Stereo Vision Using Equi-Angular Pixel Cameras

44 Pages·1999·2.59 MB·English

Analysis and Design of Panoramic Stereo
Vision Using Equi-Angular Pixel Cameras
Mark Ollis, Herman Herman and Sanjiv Singh
CMU-RI-TR-99-04
The Robotics Institute
Carnegie Mellon University
5000 Forbes Avenue
Pittsburgh, PA 15213
January 1999
© 1999 Carnegie Mellon University
This work was sponsored by the Defence Research Establishment, Suffield, Canada.
Abstract
Inthisreportwediscussmethodstoperformstereovisionusinganovelconfigura-
tionofdevicesthatallowimagingofa verywidefieldofview(full360degreesin
the azimuth and up to 120 degrees in elevation). Since the wide field of view pro-
ducessignficantdistortionthatvarieswithviewpoint,wehavedevelopedamethod
to do correlation matching and triangulation for stereo vision that incorporates
mirror shape. In this report we evaluate various configurations of cameras and
mirrors that could be used to produce stereo imagery, including one that can use a
single camera to produce stereo images. Range from panoramic stereo imagery
can be used in many applications that require the three dimensional shape of the
world. The chief advantages of this method is that it can made cheaply with no
moving parts and can provide dense range data. Specifically, this kind of a sensor
could be used for telepresence, autonomous navigation by robots, automatic map-
ping of environments and attitude estimation.
Table of Contents
1. Introduction 3
1.1 Panoramic Stereo Vision 5
1.2 Outline of the report 6
2. Panospheric Mirrors 7
3. Stereo Configuration Analysis 10
3.1 Configuration 1 11
3.1.1 Computing Range 11
3.1.2 Depth map results 14
3.1.3 Discussion 14
3.2 Configuration 2 17
3.2.1 Discussion 17
3.3 Configuration 3 19
3.3.1 Depth map results 20
3.3.2 Discussion 20
3.4 Configuration 4 22
3.4.1 Discussion 22
3.5 Configuration 5 25
3.5.1 Depth map results 25
3.5.2 Discussion 25
4. Stereo Algorithms 29
4.1 Correlation 29
4.2 Triangulation 32
4.3 Stereo Algorithms: Caveats 35
5. Other uses of panoramic imaging 36
5.1 Attitude Determination 36
5.2 User Assisted Mapping 38
6. Conclusions 42
1. Introduction
Recent research in tele-operated and autonomous systems has shown the utility of imaging that is able to
span a very wide field of view. If instead of providing a small conic section of view, a camera could cap-
ture an entire half-sphere at once, several advantages can be accrued. Specifically, If the entire environ-
mentisvisibleatthesametime,itisnotnecessarytomovethecameratofixateonanobjectofinterestor
to perform exploratory camera movements. This also means that it is not necessary to actively counteract
the torques resulting from actuator motion. Processing global images of the environment is less likely to
be affected by regions of the image that contain poor information. Generally, the wider the field of view,
the more robust the image processing will be. For autonomous robots, panoramic imagery allows the use
of regions of the image that correspond to the features transverse to the motion of the vehicle for map-
ping,whileforwardandrearpointingregionscanbeusedforpurposessuchastele-presenceandobstacle
avoidance.
Today, the alternative method to capture the entire scene in a single image is to use a ultra-wide-angle
lensatthecostofweight,expenseandopticaldistortion.Atypical180degreelenscanweighoverakilo-
gramandcostthousandsofdollars.Reflectiveopticsofferasolutiontotheproblemofglobalimaging.A
cameraplacedbelowaconvexreflectivesurfacecanproducealargefieldofviewprovidedanappropriate
mirror shape. Such a configuration (Figure 1) is eminently suited to miniaturization and can be produced
inexpensively.
curved mirror
light ray
Figure 1 Configuration of a panospheric camera.
3
Withappropriatelyshapedmirrorsthecameracanimageafull360degreesinazimuth,andinsomecases
upto120degsinelevation.Ansampleimagetakenfromacolorcameramatchedwithacurvedmirroris
shown in Figure 2. For purposes of tele-presence, one common application of this imagery is to unwarp
the raw spherical image into a cylindrical one as shown in Figure 3.
Figure 2 Raw image from panospheric camera
Figure 3 Raw image from Figure 2 displayed as a rectangular image using a projection onto a
sphere.
4
The use of such imagery has distinct advantages: First it is a passive sensor, so power requirements are
minimal. Second, it has the potential to be extremely robust, since the sensor is purely solid state and has
no moving parts. Third, curved mirrors can be made free of optical distortion that is typically seen in
lenses. Third, the large field of view available offers substantial advantages for target tracking, obstacle
detection, localization, and tele-navigation.
1.1. Panoramic Stereo Vision
Typically stereo vision has been conducted with small field of view cameras because of lens distortion.
The largest field of view that has been used to do stereo is roughly around 110 degrees. Panoramic stereo
visionispossibleif mirrorsareusedratherthanlensesbecauseoflowopticaldistortion.Notonlyispan-
oramic stereo more favorable than narrower field of stereo systems employed for telepresence or auton-
omy, it offers several advantages for applications that might otherwise use inertial sensor or active
ranging.Sincesuchsystemscanbebuiltinexpensively,itispossibletosubstituteonepanosphericsystem
formanymonocularones.Asanexample,motionestimationfromopticalflowbecomeswellconditioned
with panoramic stereo imaging.
In this report we report on the analysis and design of a panospheric stereo vision system. Such a system
could provide complete appearance, that is, both the shape and the reflectance properties of the environ-
ment. While reflectance information is simple to produce as shown above, depth perception is typically a
difficult problem.
We have evaluated a number of configurations of cameras and mirrors to produce dense panoramic range
maps. We have done the following:
• Developed a mirror shape that is truly equi-angular. The implication of this is that each pixel in
the image spans exactly the same angle. Only approximately equi-angular mirror shapes have
been reported in the literature.
• Developedanovelmethodofcorrelatingfeaturesinpanoramicimages.Correlationistheprocess
of matching features between images.
• Developed a novel method of triangulation that accounts for mirror distortion to calculate range.
Triangulation is the process of determining range from an image disparity
• Developed a prototype stereo vision engine that produces range maps from panoramic images.
• Examined 5 configurations or mirrors and cameras and evaluated them on the basis of field of
view, range precision and experimental validation with sample simulated (ray-traced) imagery.
5
1.2. Outline of the report
In Section 2 we discuss different mirror shapes that have been used by other groups and present a mirror
shape that has the property that every pixel images an equal angle. In Section 3 we evaluate 5 configura-
tions of panoramic stereo systems. In Section 4 we discuss stereo algorithms and in Section 5 we discuss
other uses of panoramic imaging.
6
2. Panospheric Mirrors
At least three families of mirrors have been used to create panoramic images:
• Spherical: These mirrors have constant curvatures and are easy to manufacture but have no com-
pellingopticaladvantages.ThesemirrorswereusedtoprovidetelepresencefortheAtacamamis-
sion[2] and the Antarctic meteorite search mission [4].
• Parabolic: Columbia University has developed a panoramic imaging system that uses a parabolic
mirror and an orthographic lens to produce a perspective panoramic image. The disadvantage of
this system is that it requires orthographic lenses which can be expensive [3].
• Equiangular: Chahl and Srinivasan have proposed a family of equi-angular mirrors that are
parameterized by a few parameters [1]. The idea with these mirrors is that each pixel spans an
equal angle irrespective of its distance from the center of the image.
Given the geometry shown in Figure 4, the general form of these mirrors is given in (1)
curved mirror
r
r
0
Figure 4 Mirror geometry from Srinivasan and Chahl paper.
cos(cid:230) q 1-----+-----a-(cid:246)-- = (r ⁄r )–(1+ a ) ⁄2 (1)
Ł 2 ł 0
For different values of a, different mirror shapes produced. Examples are shown in Figure 5. The advan-
tageofthesemirrorsisthatresolutionisunchangedwhenthecameraispitchedoryawed.However,since
the combination of the mirror/camera is no longer a projective device, stereo algorithms must be modi-
fied.
7
55 a = 7
50
45
a = 5
40
35
30 a = 3
25
20
-30 -20 -10 10 20 30 mm
Figure 5 Mirror shape fora = 3, 5 and 7. Arbitrary angular magnification allowed by
controlling thea.
Chahl and Srinivasan make a small angle approximation when they claim that each pixel spans an equal
angle. They use the following approximation to derive mirror shape:
dr (cid:230) p(cid:246)
= rcot kq + ---
dq Ł 2ł
(2)
k = (–1–a ) ⁄2
Sincethecameraisstillaprojectivedevicethisonlyworksforverysmallfieldsofview.Surfacesofmir-
rorsinwhicheachpixeltrulycorrespondstoanequalangleareshapeswhichsatisfythepolarcoordinate
equation
dr = rcot(cid:230) ktanq + p-(cid:246)-- (3)
dq Ł 2ł
instead of the approximation used in their paper. The advantage of using (2) is that the surfaces produced
have a close-form solution where as (3) must be solved numerically. The difference in the mirror shapes
is shown in Figure 6.
Forthisreportwehaveusedtheapproximatedmirrorshapessincetheyallowanalyticsolutionsofseveral
problems which would otherwise need to be estimated numerically. In a real system, however, it is likely
thatallthecalculationswillbeimplementedinlookuptablesforspeed;thus,forarealsystem,thereisno
advantage to using an approximation; it is merely a useful tool for analysis.
8
50
exactly
40
equiangulara = 3
30
approximately
equiangulara = 3
20
10
mm
-30 -20 -10 10 20 30
Figure 6 Mirrors (a = 3) that provide approximately equal angles for each pixel and exactly
equal angles for each pixel.
9