3D tracking of water hazards with polarized stereo cameras

3.3 MB·

by  Chuong V. Nguyen

3D tracking of water hazards with polarized stereo cameras
Chuong V. Nguyen1, Michael Milford2 and Robert Mahony1
Abstract—Current self-driving car systems operate well in
sunny weather but struggle in adverse conditions. One of the
most commonly encountered adverse conditions involves water
on the road caused by rain, sleet, melting snow or flooding.
While some advances have been made in using conventional
RGB camera and LIDAR technology for detecting water
7 hazards,othersourcesofinformationsuchaspolarizationoffer
1 a promising and potentially superior approach to this problem
0 in terms of performance and cost. In this paper, we present
2 a novel stereo-polarization system for detecting and tracking
water hazards based on polarization and color variation of
b
reflected light, with consideration of the effect of polarized
e
light from sky as function of reflection and azimuth angles. To
F evaluate this system, we present a new large ‘water on road’ Fig. 1. Model of polarized sky light interaction with water surface for
detectionofwaterhazards.
6 datasets spanning approximately 2 km of driving in various
2 on-road and off-road conditions and demonstrate for the first
timereliablewaterdetectionandtrackingoverawiderangeof
] realistic car driving water conditions using polarized vision as ground plane depth fitting and then employed a threshold
O the primary sensing modality. Our system successfully detects technique on depth information to detect puddles.
R waterhazardsuptomorethan100m.Finally,wediscussseveral Detection of water hazards from a single camera is also
interesting challenges and propose future research directions
. possibleusingonlycolorandtextureinformation.Zhaoetal
s for further improving robust autonomous car perception in
c hazardous wet conditions using polarization sensors. [8] developed SVM-based classification of color and texture
[ information from a single image. Rankin and Matthies [9]

I. INTRODUCTION proposed a technique based on variation in color from sky
2
v Polarization of light in outdoor environments is very reflectionsasacameramovesclosertoapuddle.LaterRank
5 common. Scattered light from the sky is polarized [1], etal[10]proposedanothertechniquethatsearchesforwater
7 especially close to the horizon. Light reflected from water reflection matching with the sky. Rankin’s techniques work
1
surfaces [2] is also polarized, particularly when the angle well for wide open space, well defined water bodies and
4
of reflection is low. Some vegetation and many man-made distance greater than 7m.
0
. objects, especially those with glass, polarize light [3], [4]. In this paper we use polarization effect as the primary
1
Water on the road poses a significant hazard when driving cue to identify water in driving conditions. To improve on
0
a vehicle, so the utility of polarized light for detecting it past research, we take a novel approach using the change
7
1 has obvious appeal, especially in the age of self-driving in the color saturation and polarization as functions of
: cars where perception systems must match current human reflection and azimuth angles to account for effect from sky
v
i capability. polarization. The method is illustrated in Fig. 1.
X
Several authors have considered using polarization as cue
r to identify water surfaces. Xie et al [5] used three cameras II. LIGHTPOLARIZATIONANDREFLECTIONONWATER
a
with polarizers to estimate polarizing angle and detect water SURFACES
surfaces. Stereo cameras equipped with horizontal and verti- In this section, we provide a treatment of the theoretical
calpolarizershavebeenusedtodetectstillandrunningwater background behind the use of polarization to detect water
bodies[6]orwetground[7].WhileYan[6]usedhand-tuned surfaces. We review the Rayleigh sky model of polarization
parameters for classification, Kim et al [7] used Gaussian as described in [1] and a reflection model proposed by [9].
Mixture Model (GMM) for hypothesis generation and a Toprovidethetheoreticalbackgroundforthenovelapproach
Support Vector Machine (SVM) for hypothesis verification. in this paper, we describe how incident light polarization,
Forpuddledetection,Kimetal[7]usedRANSACtoperform scattering and refraction can be used to model the color and
the polarization of light received from water surfaces.
1Chuong Nguyen and Robert Mahony are with the Australian
Centre for Robotic Vision, Research School of Engineering, A. Review of existing work
Australian National University, Canberra ACT 2601, Australia
[email protected] Ambient light is often treated as non-polarized; however
2Michael Milford is with Australian Centre for Robotic this is not true for light coming from sky. Illguth [1] shows
Vision, School of Electrical Engineering and Computer Science,
that sky polarization variations during the day can affect the
Queensland University of Technology, Brisbane QLD 4000, Australia
[email protected] appearanceofbuildingglassfac¸ades,whichpresumablyalso
applies to water surfaces. Polarized light is generated by
Rayleighscatteringeffect.FollowingtheRayleigh-skymodel
[1], the degree or intensity of the polarized light component
is a function of scattering angle γ:
η sin2γ
max
η = (1)
1+cos2γ
cosγ =sinθ sinθ cosψ+cosθ cosθ (2)
sun view sun view
where θview is viewing angle between a viewed point on sky Fig.3. Lightreflectionandrefractioninawaterpuddlewithdirtparticles
and zenith, θ angle between the sun and zenith, and ψ andgroundbottom.
sun
azimuth angle between the sun and the viewed point on the
sky.
R is a sum of two polarization components R
reflect reflect,⊥
and R , perpendicular and parallel respectively to the
reflect,(cid:107)
plane formed by the incident and reflected rays:
 (cid:113) 2
n cosθ−n 1−(n /n )2sin2θ
1 2 1 2
Rreflect,⊥(n1,n2,θ)= (cid:113) 
n cosθ+n 1−(n /n )2sin2θ
1 2 1 2
(4)
 (cid:113) 2
n 1−(n /n )2sin2θ−n cosθ
1 1 2 2
Rreflect,(cid:107)(n1,n2,θ)= (cid:113) 
n 1−(n /n )2sin2θ+n cosθ
1 1 2 2
(5)
where n and n are refractive indices of first and second
1 2
media (here air and water, respectively) and θ is reflection
angle at the medium interface.
The higher the coefficient R , the stronger the
Fig.2. Degreeorintensityofskylightpolarizationasfunctionofviewing reflect,⊥
angle θview (between a viewed point on sky and zenith) and angle of the intensity and apparent color of the sky as it appears in the
sunθsun (betweenthesunandzenith). reflection on water surface. The difference between the two
coefficients R and R gives the polarization in
reflect,⊥ reflect,(cid:107)
Fig.2visualizesequations1and2.ThetopofFig.2shows the reflected light. However, Equations 4 and 5 alone do not
that when observing the horizon (θview=90), the degree of explain the influence of sky polarization on the polarization,
polarization is maximal at azimuth angles of 90 and 270 the color and intensity of reflection on water surfaces in
degreesfromthesunwhenitisnearthehorizon(θsun =900), reality.
and maximal across the entire horizon when the sun is at its
zenith (θ =00). The bottom of Fig. 2 shows that when B. Effect of polarized sky light on polarization of reflection
sun
the sun is at zenith (θ =0), the degree of polarization is To take into account the effect of polarization of sky light
sun
maximum at the ground horizon. In reality, the maximum on water reflection, we propose a model of light interaction
degree of polarization can reach up to 90% of the total light withwater.Fig.3illustratesthe4componentsofequation3.
energy. Thescatteringreflectioncomponentduetosurfaceroughness
Althoughskylightisgenerallypolarized,existingresearch at O is very small compared to specular reflection for
1
often avoids dealing with this effect [9], [6], [7]. With an water, unless there is a large concentration of particles or
assumption of unpolarized incident light, Rankin et al. [9] it is windy. In addition, the light component exiting the
modeled total water reflection as a summation of specular water from particles R and ground bottom R
particles bottom
reflection from the water surface R , scattering reflection are in fact refractions, not reflections. As a result, the term
ref
of water surface R , scattering reflection of particles in ”water reflection” as sensed by eyes or cameras should be
scatter
water R , and scattering reflection from the ground understood as a sum of reflection and refraction.
particles
bottom of the water R : Internal reflection coefficients within water at O and O
bottom 2 3
are also given by equations 4 and 5 with n =n and
1 water
R =R +R +R +R (3) n =n . Refraction coefficients (from air to water or from
total reflect scatter particles bottom 2 air
water to air) can be obtained from reflection coefficients:
Specular reflection on water is known to polarizes light
[2], [5], [9], [7]. Specular reflection from the water surface R (n ,n ,θ(cid:48))=1−R (n ,n ,θ) (6)
refract,⊥ 1 2 reflect,⊥ 1 2
R (n ,n ,θ(cid:48))=1−R (n ,n ,θ) (7)
refract,(cid:107) 1 2 reflect,(cid:107) 1 2
where Snell’s law gives:
n
sinθ(cid:48)= 1sinθ (8)
n
2
Suppose that polarized light from the sky comprises en-
ergy components ES(θ,ψ) and ES(θ,ψ) for perpendicular
⊥ (cid:107)
and parallel components respectively as functions of reflec-
tionangleandazimuthangle.Lightthatenterswatersurface
illuminatesparticlesandthegroundbottom.Thetotalenergy
entering the water is therefore:
FS=ES(θ,ψ)[1−R (n ,n ,θ)]
⊥ reflect,⊥ 1 2
+ES(θ,ψ)[1−R (n ,n ,θ)] (9)
(cid:107) reflect,(cid:107) 1 2
Fig. 4. Reflection and refraction from water as function of degree
Part of the energy FS is scattered by suspended particles and direction of polarization. Top: unpolarized light source. Middle: light
source that is 80% polarized in perpendicular direction with reflection
and ground bottom while the rest is absorbed:
plane.Bottom:lightsourcethatis80%polarizedinparalleldirectionwith
reflectionplane.
µ +µ +µ =1 (10)
particles bottom absorption
whereµ andµ arescatteringcoefficientsforpar-
particles bottom Equations 11 and 12 are significant as they explain
ticlesandthegroundbottom,andµ istheabsorption
absorption the mechanism of the color mixing process for the sky,
coefficient for both particles and the ground.
suspended particles and water bottom. As θ and ψ vary
Due to random scattering and internal reflection process,
with viewpoint, the polarized components of light coming
light within water can be considered highly unpolarized
from different sources vary accordingly, leading to color
(the perpendicular component is approximately the same as
variation on the water surface. Furthermore, when capturing
parallel component). Part of the scattered light comes out of
animageofthewatersurfacewithorthogonalpolarizers,the
the water via refraction. The total light energy component
orthogonal components of polarized light can be captured
coming out of water is the summation of reflection and
separately and show up as changes in the image color and
refraction (viewed at the same point and angle) for each
intensity dependent on the direction of the filters. We use
polarization component:
these phenomena as the basis for our water hazard detection
process
E⊥R(θ,ψ)≈E⊥S(θ,ψ)Rreflect,⊥(n1,n2,θ)+ III. APPROACH
0.5FS[µparticles+µbottom]Rrefract,⊥(n1,n2,θ(cid:48)=θ) (11) With the theoretical foundation provided in the previous
section,wenowdescribeourapproachtovisualdetectionof
water hazards. The overall algorithm is as follows:
ER(θ,ψ)≈ES(θ,ψ)R (n ,n ,θ)+ 1) Calculate disparity map from a stereo image pair.
(cid:107) (cid:107) reflect,(cid:107) 1 2
0.5FS[µ +µ ]R (n ,n ,θ(cid:48)=θ) (12) 2) Estimate ground plane disparity by 3D plane fitting
particles bottom refract,(cid:107) 1 2
using robust estimation within a triangular region in
Equations11and12arevisualizedinFig4.Forillustrative front of the car. Obtain the equation of the horizon
purpose, the absorption coefficient µabsorption is set to 60% line.
here. Depending on the position of the sun and the viewed 3) Use ground plane disparity to warp the right-hand
point on the sky, a different ratio of ES(θ,ψ) and ES(θ,ψ) image to the left-hand image, producing point corre-
⊥ (cid:107)
is obtained. Fig 4 shows that for unpolarized light (top) and spondence for feature extraction.
polarized light perpendicular to reflection plane (middle), 4) Compute the reflection and azimuth angles for all
ER(θ,ψ) is higher than ER(θ,ψ). This agrees with the pixels below horizon line.
⊥ (cid:107)
conventionalassumption[6],[7].Howeverforpolarizedlight 5) Extraction image color features from point correspon-
of 80% in parallel direction to the reflection plane (bottom dences as function of reflection and azimuth angle.
of Fig 4), ER(θ,ψ) is lower than ER(θ,ψ). This is however 6) Training: manually created ground truth masks are
⊥ (cid:107)
different from conventional assumption. Fig. 4 also shows used to train Gaussian Mixture Models for water and
that at reflection angles above 70 degrees (or at a large not-water.
distance), the polarization difference between perpendicular 7) Testing: trained Gaussian Mixture Models are used
and parallel components is large and thus provides a strong to compute likelihoods of water and not-water pixels,
detection cue for water hazards at a large distance. producing a water mask for each stereo image pair.
Fig.6. Top:schematicsideviewofreferencecamera(left)andthescene
when the horizon is completely horizontal to the camera image. Bottom:
geometryofcameraandreflectionwhenthehorizonlineisnothorizontal.O
iscameraorigin,Ccameraopticalcenter,IandI0toI5pointsonhorizonline
image,Rimageofreflectionpointonwaterorground,θ incident/reflection
angle,ψ azimuthangle,β andω relativeanglesofhorizonlineimageto
opticalaxisandimagehorizontaldirection.
Fig.5. Planefittingtostereodisparitymap.Datapointswithintheblack
triangle are selected for fitting. Red points are inliers and blue points are
outliers when the fitted plane extends to the whole image. The red line
representsthehorizonlinewheredisparityiszero. where:
RI =RI cosω =(v −v )cosω (14)
4 3 R I3
(cid:112) (cid:113)
A. Stereo disparity and ground-plane disparity estimation OR= OC2+CR2= f2+(u −u )2+(v −v )2
C R C R
(15)
Given a pair of stereo images, a variant of semi-global (cid:113) (cid:113)
OI = OC2+CI2= f2+(u −u )2+(v −v )2
block matching algorithm [11], [12] is employed to obtain a 4 4 C I4 C I4
disparity map. (16)
Plane fitting by robust estimation with Cauchy loss func- uI4 =uR+RI4sinω =uR+(vR−vI3)cosωsinω (17)
tion is applied to extract ground disparity shown as black (cid:18)v −v (cid:19)
ω =arctan I5 I0 (18)
rectangle in Fig. 5. Horizon line is the zero disparity line on u −u
I5 I0
thefittedgrounddisparityplaneshowasredlineinFig.5.If
and f is focal length.
the 3D plane equation of ground disparity is au+bv+c=δ
To account for sky polarization effect, azimuth angle ψ
where δ is disparity and u, v are pixel coordinates, the
is also computed at each pixel position. For simplicity, the
horizon line equation is au+bv+c=0.
azimuth angle is obtained with respect to camera forward
For simplicity, assuming there are no obstacles on the
direction for the tests in this paper (without taking account
groundnearthecar,therightimagecanbewarpedtotheleft
camera orientation in the earth coordinate system), as the
image using the fitted ground disparity. Direct pixel-to-pixel
camera mostly points along a single orientation and sky
comparison between the left image and warped right image
polarization in opposite directions are similar enough:
is now possible.
 
(cid:18) (cid:19)
I I CR cos(η−ω)
ψ =arctan 2 4 =arctan (cid:113)  (19)
OI
B. Reflection and azimuth angles 2 OC2+CI2
2
u =u +CI sinω =u +(v −v )cosωsinω (20)
I2 C 2 C C I1
AsdiscussedinsectionII,thereflectionangleisoneofthe
main factors affecting the color and polarization of refection where η is angle of CR formed with pixel horizontal
from water. Here, we propose an algorithm to compute direction in the image.
reflection angle from the pixel position and horizon line as
C. Gaussian Mixture Model for water-ground classification
illustrated in Fig. 6. Here we assume that water surface is
As discussed in section II, color and polarization (i.e.
parallel to horizon line, and that camera optical axis OC is
intensity)varywithincidentandazimuthangles.AGaussian
perpendicular to the image plane.
Mixture Model (GMM) is used to capture these effects:
(cid:91)
Reflection angle θ =(π/2−α), where α =ROI (RI is
4 4
perpendiculartohorizonline)canbeobtainedbycosinerule:
p(x;a ;S ;π )=Σm π p (x),π >0,Σm π =1 (21)
k k k k=1 k k k k=1 k
α =arccos(cid:18)OR22∗+OORI∗42O−IRI42(cid:19) (13) p(x)= (cid:112)(2π1)d|S |exp(cid:18)−12(x−ak)(cid:124)Sk−1(x−ak)(cid:19) (22)
4 k
TABLEI
where π is the weight of the k-th Gaussian cluster, p is a
k k
VALIDATIONOFTHEWATERDETECTIONALGORITHMWITHOUTAND
normal distribution with mean a and covariance S .
k k
WITHAZIMUTHANGLEFEATURE.
Images are converted from RGB to HSV color space
and features at pixels below horizon line were collected Seq. Train Test Accuracy Recall Precision
for training and classification of the GMM models. After On-road 54 65 0.956/0.970 0.906/0.860 0.247/0.378
Off-road 71 80 0.933/0.931 0.824/0.837 0.223/0.220
manyvalidations,wefoundthatstrongfeaturesfordetection
are saturations (left and right cameras), value or brightness
(left camera), reflection and azimuth angles (left camera).
Fig. 8 show examples of prediction masks, and ground truth
Saturations and value are normalized to vary from 0 to 1.
masks.Theblackregionsofpredictionmasksindicatewater
IV. EXPERIMENTALSETUP pixels where the ratio is larger than 1. These examples were
Images were captured using a laptop-connected Stereo- produced by the GMM models with azimuth angle.
Labs ZED stereo camera [13] mounted on top of a car at Truepositives(realwater),truenegatives(realnot-water),
the height of 1.77m as shown in Fig. 7. Each stereo video falsepositives(wrongwater),andfalsenegatives(wrongnot-
frame is a side-by-side left-right image of (2×1280)×720 water)wereobtainedbycomparingpixelsbetweenpredicted
pixel resolution. The nominal camera focal length is 720 masks and ground-truth masks. Accuracy, recall and preci-
pixels with a stereo baseline of 120mm. Linear polarizing sion performance were then computed. Table I provides the
films [14] with max transmittance of 42% and polarizing details of training, testing and the prediction results using
efficiencyof99.9%areattachedtothefrontofcameralenses GMM models with left-right saturations, left brightness,
in the horizontal direction (left view) and vertical direction reflection angle, and without and with azimuth angle. This
(rightview).APythonscriptacquiredimagesfromthestereo is to test the effectiveness of taking account the effect of
camera at 30 frames per second, added them to a buffer polarization from the sky.
queue which was streamed to the solid-state drive of the Test accuracy and recall are excellent, above 90% and
laptop to avoid frame dropping. Stereo calibration was done 80%respectively,forbothsequencesandbothGMMmodels.
using StereoLabs calibration software. Stereo reconstruction The test precision is however relatively low, at 25% and
was performed using semi-global block matching algorithm 22% for on-road and off road respectively, using our GMM
implemented by the OpenCV library [15]. models without azimuth angle therefore ignoring the effect
Two video sequences of real driving conditions (Fig. 7) ofskypolarization.WithourGMMmodelincludingazimuth
were selected to test our algorithm: angle, the precision of on-road sequence increases to 38%
• Theon-roadsequencecontains5357videoframescom- while that of off-road sequence stays the same. This shows
prisinga1360mroundtripofcountryroad(locationbe- that our theory of sky polarization affecting the appearance
tween 35◦13’19.6”S 149◦09’09.9”E and 35◦13’40.9”S of water applies well to the on-road sequence where most
149◦09’12.1”E). water puddles reflect light directly from the sky. However,
• The off-road sequence contains 6098 frames compris- this theory does not apply successfully for the off-road case
ing a 780m round trip in a car park (location at wheretherearetreesblockinglightfromtheskyandaltering
35◦17’13.1”S 149◦08’09.0”E) next to a roadwork site. the relationship.
For both sequences, there are several source of error.
Foreachsequence,ground-truthmaskswerecreatedman-
Objects protruding above the ideal ground plane lead to
ually from images containing a number of water puddles
somefalsedetections.Suchfalsedetectionscanbemitigated
of various sizes and distances. Training images were se-
by excluding above-ground trees and obstacles. Wet ground
lected such that they always contained water puddles of
also reflects and polarizes light. As a result, wet areas
various sizes and distances. Test images were selected at
are often classified as water puddles, although the ground-
approximatelyevenintervalsthroughoutthevideosequences,
truthmasksdon’tincludethem,leadingtoreducedprecision
includingframesthatcontainnopuddles.Thesedatasetsand
performance. For hazard detection, it is however useful to
ground-truth masks are made freely available at [16].
detect wet ground. Furthermore to some extend the ground-
V. RESULTS truth masks have errors as human can miss out some small
For each sequence, masks and corresponding stereo im- waterregionswhereGMMmodelscanpickupcorrectly.The
ages were used to train two GMM models of 5 Gaussian precision is also affected in this case. Finally, images cap-
clusters, one for water and one for not-water. Numbers of turedbytheZEDcamerainthisworkoftenhasnoisystripes
ground-truth water masks for training and testing the GMM in dark image areas. These noisy stripes slightly change
models are given in Table I. image brightness and perceived polarization characteristics
For testing, the trained GMM models were used to com- therefore adversely affecting the detection algorithm.
pute the likelihood of pixels being water and not-water from Fig. 9 shows effective detection range using our GMM
features extracted from individual stereo image pair. The model with azimuth angle to account for sky polarization
ratio of the likelihood of water over that of not-water shown effect. Vertical axis is ratio between water true positives
in the second rows of Fig. 8 is used to determined if a and water ground truth positives. For both on-road and
pixel belongs to water or not. The third and fourth row of off-road sequences, the true detection rate is around 90%
Fig. 7. Experimental setup. Top left: ZED stereo camera on top of car with polarizing films on camera lenses. Top middle and right: trajectories of
on-roadandoff-roadvideosequences.Bottomrow:examplesofstereoimagepairsfromon-roadandoff-roadvideosequences.
Fig.8. Examplesnapshotsofwaterdetectionfortheonroad(leftcolumns)andoffroad(rightcolumns)sequences.Fromtoptobottomrowsarestereo
leftimages,predictedratioofwaterandnot-waterlikelihoods,watermasks(wheretheratioishigherthan1),andgroundtruthwatermasks.Waterareas
showstrongratiovaluewhilesomewetareasalsoshowhighratiovalue.Thecolorbarrangesfrom0to255.
reflected light to detect water. Effect of sky polarization is
accounted by the use of azimuth angle which significantly
improvesdetectionprecisionforon-roadsequence.Fromthis
project, we have developed two new datasets for on-road
and off-road driving in wet conditions which we provide to
the community, and demonstrated water detection on these
datasets using the proposed approach. We hope that the
research presented here provides a new stimulus for further
investigations of the utility of using conventional camera
technology equipped with polarizing filters for improving
robot and autonomous vehicle perception in the myriad of
environmental conditions where water poses a hazard.
ACKNOWLEDGEMENT
This research was conducted by the Australian Re-
search Council Centre of Excellence for Robotic Vi-
Fig.9. Waterdetectionrangeforon-roadandoff-roadsequencestogether sion (CE140100016) http://www.roboticvision.
withresultsbyYan[6]andRankinetal[10]. org. The authors would like to thank Prof Richard Hartley
of Research School of Engineering, Australian National
University, for his kind support to this project, and Mr
for distance from 3 to 10m and then decreases gradually Alexander Martin of the Research School of Engineering
for distance between 10 to 35m and then stays almost the who helped record early water-on-road videos.
same for distance from 35m to about 60m (considering the
size of the water puddles are quite small at this distance REFERENCES
range). For off-road sequence, the true detection is lower [1] M. Illguth, C. Schuler, and O¨. Bucak, “The effect of optical
than for on-road sequence, presumably due to reflection of anisotropiesonbuildingglassfac¸adesanditsmeasurementmethods,”
FrontiersofArchitecturalResearch,vol.4,no.2,pp.119–126,2015.
trees. True detection drops to 20% at distance of about
[2] R.Wehner,“Polarizationvision–auniformsensorycapacity?”Journal
35m. As a crude comparison, results achieved in previous ofExperimentalBiology,vol.204,no.14,pp.2589–2596,2001.
studies by [9] and [6] are also included, although these [3] V. C. Vanderbilt, L. Grant, and C. Daughtry, “Polarization of light
scatteredbyvegetation,”ProceedingsoftheIEEE,vol.73,no.6,pp.
results were obtained from different datasets (not publicly
1012–1024,1985.
available) with slightly different conditions. Yan [6] used a [4] R. Walraven, “Polarization imagery,” Optical Engineering, vol. 20,
similar polarized stereo camera system and achieved about no.1,pp.200114–200114,1981.
[5] B.Xie,H.Pan,Z.Xiang,andJ.Liu,“Polarization-basedwaterhazards
90% true detection rate for images captured in rain forest
detectionforautonomousoff-roadnavigation,”in2007International
up to 8m distance then drop down sharply to 20% at 10m ConferenceonMechatronicsandAutomation. IEEE,2007,pp.1666–
distanceandabove.RankinandMatthies[10]didnotprovide 1670.
[6] S.H.Yan,“Waterbodydetectionusingtwocamerapolarizedstereo
statistics of detection at different distances but max and min
vision,” International Journal of Research in Computer Engineering
distanceswhenwaterwasfirstlydetectedincloudy,overcast &Electronics,vol.3,no.3,2014.
and clear sky conditions with a single large water body. [7] J.Kim,J.Baek,H.Choi,andE.Kim,“Wetareaandpuddledetection
foradvanceddriverassistancesystems(adas)usingastereocamera,”
Technically the system is correctly detecting (small) International Journal of Control, Automation and Systems, vol. 14,
amounts of water on the ground at a wide range of distance, no.1,pp.263–271,2016.
[8] Y. Zhao, Y. Deng, C. Pan, and L. Guo, “Research of water hazard
but for practical applications it is likely we would need
detectionbasedoncolorandtexturefeatures,”Sensors&Transducers,
to find a way to discount non-significant water coverage, vol.157,no.10,p.428,2013.
perhaps by attempting to calculate the depth of the water [9] A.RankinandL.Matthies,“Daytimewaterdetectionbasedoncolor
variation,”inIntelligentRobotsandSystems(IROS),2010IEEE/RSJ
hazardfromthechangesofthecolorofthewaterasfunction
InternationalConferenceon. IEEE,2010,pp.215–221.
of viewing angle. Noisy water masks could be cleaned up [10] A.L.Rankin,L.H.Matthies,andP.Bellutta,“Daytimewaterdetection
by filtering across multiple image frames. based on sky reflections,” in Robotics and Automation (ICRA), 2011
IEEEInternationalConferenceon. IEEE,2011,pp.5329–5336.
[11] H. Hirschmuller, “Stereo processing by semiglobal matching and
VI. CONCLUSIONS
mutual information,” IEEE Transactions on pattern analysis and
machineintelligence,vol.30,no.2,pp.328–341,2008.
Water on the ground poses a fundamental hazard to
[12] S. Birchfield and C. Tomasi, “A pixel dissimilarity measure that is
robots and manned or autonomous vehicles. While there insensitivetoimagesampling,”IEEETransactionsonPatternAnalysis
has been significant research on this problem to date, most andMachineIntelligence,vol.20,no.4,pp.401–406,1998.
[13] Stereolabs, “Zed - depth sensing and camera tracking,” 2016.
has involved using conventional sensing modalities or have
[Online].Available:https://www.stereolabs.com/zed/specs/
been limited studies using light polarization. In this paper, [14] D.Corporation,“Polarizer,linearpolarizer-3dlens.com,”2016.
we have attempted to advance the concept of using light [15] G.BradskiandA.Kaehler,LearningOpenCV:Computervisionwith
theOpenCVlibrary. ”O’ReillyMedia,Inc.”,2008.
polarization to detect water hazards on the ground in a self-
[16] C. Nguyen, “Acrv’s videos of water puddles on roads.”
driving car context. In particular we have developed a novel [Online]. Available: https://cloudstor.aarnet.edu.au/plus/index.php/s/
approach that uses both polarization and color variation of O4Zpl4rWkcjsIUO