Camera distortion self-calibration using the plumb-line constraint and minimal Hough entropy

0.64 MB·English

by  Edward Rosten

NonamemanuscriptNo.
(willbeinsertedbytheeditor)
Camera distortion self-calibration using the plumb-line constraint and
minimal Hough entropy.
EdwardRosten · RohanLoveland
9 thedateofreceiptandacceptanceshouldbeinsertedlater
0
0
2 Abstract Inthispaperwepresentasimpleandrobustmethod 1 Introduction

n forself-correctionofcameradistortionusingsingleimages
a ofsceneswhichcontainstraightlines.Sincethemostcom- 1.1 CameraCalibrationandDistortionCorrection
J
mon distortion can be modelled as radial distortion, we il-
4
lustratethemethodusingtheHarrisradialdistortionmodel, Cameracalibrationaddressestheproblemoffindingthepa-

] but the method is applicable to any distortion model. The rameters necessary to describe the mapping between 3-D
V methodisbasedontransformingtheedgelsofthedistorted world coordinates and 2-D image coordinates. This can be
C imagetoa1-DangularHoughspace,andoptimizingthedis- dividedintothedeterminationofextrinsicandintrinsicpa-
. tortion correction parameters which minimize the entropy rameters. The extrinsic parameters are necessary to relate
s
c of the corresponding normalized histogram. Properly cor- an arbitrary 3-D world coordinate system to the internal 3-
[ rected imagery will have fewer curved lines, and therefore D camera coordinate system, where the z-axis is typically
2 lessspreadinHoughspace.Sincethemethoddoesnotrely taken to be along the optical axis. The intrinsic parameters
v onanyimagestructurebeyondtheexistenceofedgelsshar- are independent of the camera position, and typically in-
6 ingsomecommonorientationsanddoesnotuseedgefitting, cludeafocallength,anoffsetrelatingpixelcoordinatescen-
2
4 itisapplicabletoawidevarietyofimagetypes.Forinstance, ter to the optical center, a scale factor describing the pixel
4 itcanbeappliedequallywelltoimagesoftexturewithweak aspect ratio, and one or more parameters describing non-
0. butdominantorientations,orimageswithstrongvanishing linear radial and tangential distortions. Detailed examples
1 points. Finally, the method is performed on both synthetic ofthisprocesscanbefoundin[13,16],amongothers.
8 andrealdatarevealingthatitisparticularlyrobusttonoise. In this work we address the determination of the non-
0 linear distortion parameters, with the assumption that the
:
v otherintrinsicparameters,usuallyaddressedassumingapin-
i holecameramodel,canbefoundaftertheinitialnon-linear
X Keywords Radialdistortion·Cameradistortion·Plumb-
distortion correction. In order to do this a model must be
r lineconstraint
a selected, as well as a method of estimating the model pa-
rameters.
EdwardRosten·RohanLoveland
LosAlamosNationalLaboratory 1.2 SummaryofRelatedWork
LosAlamos,NewMexico,USA
E-mail:[e-mail]·[e-mail]
1.2.1 DistortionCorrectionModels
EdwardRostem
DepartmentofEngineering
Ageneralequationfordistortioncorrectionisgivenby
UniversityofCambridge,Cambridge,UK
E-mail:[e-mail]
RohanLoveland I1(x(cid:48))=I0(x), x(cid:48)≡D(x), (1)
DepartmentofEngineeringScience
UniversityofOxford,Oxford,UK whereI1istheoutput(corrected)image,I0istheinput(dis-
E-mail:[e-mail] torted) image, and x is a pixel location in the input image
2
thatismappedtolocationx(cid:48) intheoutputimagebydistor- frommultiplepositionsorviewsofaparticularscene.This
tioncorrectionfunctionD(x):R2⇒R2. isproblematicininstanceswhereaccessisnolongeravail-
If the distortion is assumed to be isotropic, and strictly abletothecamera,butonlytotheresultingimagery.
radial,thenD(x)canbewrittenas As a result, a number of methods have been developed
which can operate on single views but which make use of
D(x)= f(ρ)rˆ+c, (2)
common structure in images such as vanishing points [3],
where c is the center or radial distortion, the radius vector higher-ordercorrelationsinthefrequencydomain[11]and
r=[r r ]T=x−c,theradiusρ =(cid:107)r(cid:107),thenormalizedra- straightlines[4].
1 2
diusvectorrˆ=r/ρ and f isascalarfunctionofρ. The methods relying on the existence of straight lines,
Severalmodelsfordistortioncorrectionhavebeenused the “plumb-line” methods, were pioneered by Brown [4].
previously.Intermsof f,themodelsincludethepolynomial Brown used a test-field with actual strung plumb-lines, but
model[13,2],where in general these methods do not require knowledge of the
locationsofthelines.Inparticular,whiteplumb-lineswere
f(ρ)=ρ(1+k1ρ2+k2ρ4+k3ρ6+...), (3) strungacrossablackbackground,withtheplumb-bobsim-
mersed in oil for stability. The photographic plate was ex-
andtheHarrismodel[27],where
posedtwicewiththecamerarotatedabouttheopticaxisby
f(ρ)= ρ . (4) 90◦, giving a nominally square grid of lines. A number of
(cid:112)
1+γρ2 points along the lines recorded on the photographic plate
were measured using a microscope (a Mann comparator).
More complex models such as [14] include additional
Calibration was then performed by minimizing the least-
tangentialdistortion
squares error in the image between distorted straight lines
D(x)=(cid:20)p1(ρ2+2r12)+2p2r1r2(cid:21)+f(ρ)rˆ+c, (5) andthemeasurespointsonthephotographedplumb-lines.
2p1r1r2+p2(ρ2+2r22) Plumb-linemethodsareapplicabletomanyimagetypes,
because nearly all scenes containing man-made structures
to account for decentering of the optical system (where p
1
have a large number of straight lines. Plumb-line methods
and p aretheparameters).Morerecentlytherationalfunc-
2
generally rely on the process of optimizing the distortion
tion model, where polynomial and perspective transforms
correction parameters to make lines that are curved by ra-
arecombinedintoarationalpolynomial[5,19]hasbeenpro-
dial distortion straight in the corrected imagery. The lines
posed.
can be manually selected, as in [26], or they can be found
Allofthesemodelscanhaveadequateperformancede-
automatically,asin[7,25,6]bydetectingedgelsandlinking
pending on the characteristics of the camera optics. In the
themintolinesegments.
following we limit our discussion and implementation to
The objective function for optimization can be formu-
a single model in order to bound the scope of the paper,
lating by undistorting the line segments and measuring the
thoughitshouldbenotedthatthemethodwepresentisex-
straightness by fitting a straight line [26,7]. Alternatively,
tensible to any model. The Harris model is chosen for its
thedistortionmodelcanbechosensothatstraightlinesbe-
reasonable performance, single parameter, and ease of in-
come specific family of curves, such as conics [6] or cir-
version(achievedbysimplynegatingγ).
cles [25]. The distortion can then be found by fitting these
curvestothedistortedlinesegments.
1.2.2 ParameterEstimationMethods
Avarietyofmethodsexistforestimatingcameradistortion
2 AlgorithmOverview
correction model parameters. Earlier efforts relied on im-
agerywithartificiallycreatedstructure,eitherintheformof
Weproposeamethodthatissimpleandrobusttohighlevels
atest-field,populatedwithobjectshavingknown3-Dworld
ofnoise,asshownintheresultssection.Inouralgorithmwe
coordinates, or using square calibration grids with lines at
calculate all image edgels, and then transform these into a
constantintervals[13,16,2].Alternativeapproachesdonot
one-dimensionalHoughspacerepresentationofangle.This
requireartificiallycreatedstructure,butusedmultipleviews
createsanorientationhistogramoftheedgelangles.Inthis
of the same scene. The calibration technique makes use of
form,curvedlineswillberepresentedatavarietyofangles,
constraintsduetoknowncameramotion(forinstancerota-
whilestraightlineswillbefoundonlyatone.Therefore,we
tion)[23],knownscenegeometrysuchasplanarscenes[21]
optimize the model distortion parameters which minimize
orgeneralmotionandgeometryconstrainedwiththeepipo-
the entropy (or spread) of the Hough space angular repre-
larconstraint[24,1,5].
sentation.Theindividualstepsare:
These approaches required access to the camera in or-
dertoperformaspecificoperation,suchasacquiringviews 1. Findsalientimageedgels,withnormalvectors.
3
wesplittheimageupintoaregulargrid,andperformsub-
sampling independently within each grid cell, so that each
cellcontainsthesamenumberofedgels.
(a) (b) (c)
2.2 Distortioncorrectionmodel
Fig.1 Salientedgedetectionusingtensorvoting.(a)Theoriginalim-
age,(b)thegradientmagnitude,(c)theedgesaliency. 2.2.1 Anisotropicextension
Asmentionedpreviously,forthebasicradialdistortioncor-
2. Performadistortioncorrectingtransformationtotheedgels.
rectionmodel,wehavebasedourdistortioncorrectionmodel
3. Computethe1-DangularHoughtransform.
ontheHarrismodeldefinedinEquation4.Inordertoextend
4. Computeanobjectivefunctiondefinedasthespread(en-
themodel’sflexibility,weaddananisotropiccomponent,so
tropy)ofthe1-DHoughtransform.
thatthemodelbecomes:
5. Optimizethetransformationparameterstominimizethe
entropy/spread based objective function, iterating steps
D(x)=rˆf(ρ)(1+g(θ))+c, (7)
2–4,
6. Usetheoptimizedtransformparameterstomaptheinput
where f(.)andg(.)arethedistortionfunctions.
imagetoacorrectedoutputimage.
Inparticular,wedefinetheanisotropyfunction,g(θ)to
Note that step 1, finding the edgels, is only required be:
once,duetothefactthattheedgels,ratherthantheunderly- g(θ)=a sin(θ+ψ )+a sin2(θ+ψ )+...
1 1 2 2
ingimage,canbetransformed.This,andtheotherstepsof
=b sinθ+b cosθ+(b sinθ+b cosθ)2+..., (8)
the process, are described in further detail in the following 1 2 3 4
respectivelyenumeratedsubsections. which can be rearranged as a Fourier series. This general-
purposeformulationhasnodiscontinuityastheanglewraps
aroundandconvenientlyavoidstheuseoftrigonometricfunc-
2.1 Salientedgeextraction
tions,sincecosθ =r /ρ,etc.
1
The structure that we are making use of in this paper con-
2.2.2 Edgeltransformation
sistsoflong,straightedges,whichappearaslong,smoothly
curvededgesintheinputimage.
In order to evaluate the cost function it is necessary to de-
Simplyperforminganedgedetectiondoesnotresultin
terminetheedgels’orientation,asdiscussedintheprevious
thesefeaturesbeingdominant,sotoenhancethelongedges,
section.Ifitwerenecessarytoregeneratethesebycomput-
weusetensorvoting[15]onthedensegradientimage.The
ing a completely new image for each new set of distortion
gradientisproducedbyfinitedifferences,andvotingismade
model parameters, however, the optimization process (in-
in proportion to the gradient magnitude. We use the stan-
volvingnumerousevaluationsofdifferentsetsofmodelpa-
dardkernelforsmooth,circularcurves,specificallytheker-
rameters) would be prohibitively slow. Furthermore, errors
nelgivenbytheequationsonpage60of [15].
duetoresamplingwouldbeintroducedwhentheimagewas
For edges, each pixel is represented by a 2×2 positive
undistortedateachiteration.
semi-definitematrixwitheigenvaluesλ andλ .Wewishto
1 2
Fortunately,theedgelsneedtobedeterminedonlyonce,
findpointswhicharesufficiently‘edgey’soweusetheedge
fromtheinputimage,duetothefactthatthedistortioncor-
saliencyfunctionφ,givenby:
rectingtransformationscanbeapplieddirectlytotheedgels,
φ =maxλ1,λ2−eminλ1,λ2. (6) providedthattheJacobiansofthetransformationsareknown.
Inordertoseethis,consideranedgelintheinputimageas
Thisdetectspointswheretheedgeinthemainorientationis partofaparameterisedcurvel(t)inR2.Thedistortioncor-
significantly brighter than other edges. We have found that
rection transformation, D, can be applied to this space to
e=2producesgoodresults.Anexampleoftheseareshown
createanewline:
inFigure1.Thenormaloftheseedgelsisgivenbytheeigen-
vectorcorrespondingtothelargesteigenvalue. m(t)=D(l(t)). (9)
The final stage is to discard the non-salient edges by
thresholding at φ =0. For computational efficiency of the Thelinetangentsarethenfoundbydifferentiatingtobe:
laterstages,thesetofedgespassingthethresholdissubsam-
(cid:34) (cid:35) (cid:34) (cid:35)
pled.Typically,wekeep100,000edgels.Inordertoprevent ∂m1 ∂l1
∂t =J ∂t , (10)
theresultbeingdominatedbyoneregionwithstrongedges, ∂m2 ∂l2
∂t ∂t
4
0.2 bemappedtoasinglepointinHoughspace.Inpracticethe
mx+bformulationisunwieldy,so
0.15
bability 0.1 ρ =xcosθ+ysinθ (13)
o
Pr
0.05
isused,whereρ istheshortestdistancefromthelinetothe
0
Distorted lines 0 5 10 15 20 25 30 35 origin,andθ isdefinedbythevectornormaltotheline.
Bin number
The Hough space can be quantized into discrete bins,
0.8
andeachedgelcanbeassignedtoaparticularbin.Givena
0.6
bability0.4 sisetthoefneidngdeilcsa,tesudpbpyortthefoarctchuemeuxliasttieonnceofoaf laarpgaertnicuumlabrelrinoef
o
Pr
0.2 edgels in the corresponding bin. The edgels from a curve
0 endupinadjacentbins,resultinginadiffuseclusterofnon-
Straight lines 0 5 10 15 20 25 30 35
Bin number zerobins,whileedgelsthatareallonthesamelineendup
Fig.2 Imagesofcurvedandstraightlineimagesandthecorrespond- in exactly the same bin. This important fact motivates the
ing1-DHoughtransforms. definitionofourobjectivefunction,asexplainedinthenext
section.
In general we anticipate the presence of a number of
wheretheJacobian,J,isgivenby:
parallel lines in the scene, and would like to utilize their
(cid:34)∂D1 ∂D1(cid:35) reinforcement of each other. We therefore marginalize the
J= ∂ll ∂l2 . (11) [ρ]×[θ]Houghspaceintoa1-Dθ spacebysummingover
∂∂Dll2 ∂∂Dl22 the ρ values for each θ. An example of some images with
curvedlinesandstraightlines,andtheircorresponding1-D
Theedgelsincludenormalsndefinedatdiscretepoints.The Hough transforms, are shown in Fig. 2. Note that the two
transformededgelnormals,h,canthenbefoundbyrotating principal line directions are mapped to two corresponding
the normals by 90◦ (making them tangents), transforming peaksinHoughspace.
thetangentsbyJandrotatingthetransformedtangentsback
tobenormals:
2.4 Entropy-basedobjectivefunction
(cid:20) (cid:21) (cid:20) (cid:21)
0 1 0−1
h= J n (12)
−10 1 0 The radial distortion correction method presented here is
motivatedbytheobservationthatcurvedlinesmaptospread
Note that we do not parameterise the line with a func- outpeaksinHoughspace,whilestraightlinesmaptoasin-
tion.Thelineanditsnormalisknown(andused)onlyata glebin.Therefore,itisdesirabletohaveanobjectivefunc-
discrete set of points, specifically where the edgels are de- tion that measures this spread. In information theory this
tected.Thismeansthatl(t)andn(t)canbeevaluatedatev- quality is represented by entropy [22]. We have therefore
eryvalueoft werequire.Sincetheedgeldetectionprocess normalized the 1-D Hough representation, and treat it as a
alsoprovidesthenormals,Jisonlyafunctionofthedistor- probabilitydistribution.Theobjectivefunctionisthen:
tionmodel,andisthereforecomputedanalyticallyfromthe
B
definitionofD.ThederivationofJfortheHarrismodelis C(H)≡−∑ p(H )log (p(H )), (14)
b 2 b
giveninAppendixA.
b=1
whereHisthediscretized1-DangularHoughtransformof
the edgels of an input image for a given set of model pa-
2.3 1-DAngularHoughtransform
rameters, p(H ) is the value of a given normalized Hough
b
bin, and B is the number of bins. Minimizing C therefore
TheHoughtransformisatechniqueforfindinglinesinim-
minimizesthespread.
ages [9]. It is sufficiently well-known so that only a trun-
catedexplanationwillbegivenhere.Thebasisofthetech-
niqueisthetransformofalinetoapointin“Hough”space. 2.5 Optimization
By way of example, if a line is defined by y=mx+b,
thenitcanberepresentedbyasinglepointina2-DHough The cost function has many local minima, so to optimize
spaceof[m]×[b].Anedgel,whichcontainsbothanormal it effectively a reasonable strategy is needed. As a broad
vector (and thus the slope of a line) and a discrete point overview,wehavefoundthattheMCDH(Monte-Carlodown-
(which a corresponding line would pass through), can also hill)strategyworksbest.Essentially,startingparametersare
5
selectedatrandom,andthedownhillsimplexalgorithm[17] x 10−5 Uncorrelated noise
4
(specificiallythevariationgivenin[18])isusedtooptimize Isotropic
Anisotropic
the cost from the selected starting points. The best result 3
isthenselected.Thedownhillsimplexalgorithmisiterated
until the residual drops below a threshold (10−15), or 1000 g2
iterations occur, whichever is sooner. The residual is com- 1
putedas:
0
70% Random noise 0 0.2 0.4 0.6 0.8 1
(cid:107)s −s (cid:107) Proportion of noise edgels
r= best worst 2,
(cid:107)sbest(cid:107)2 4x 10−5 Correlated noise
Isotropic
Anisotropic
wheresbest isthelocationofthebest(smallest)pointonthe 3
simplex,ands isthelocationoftheworst(largestpoint)
worst
g2
onthesimplex.
In order to implement this method effectively, several 1
techniques are needed. Parameters computed in the Hough
0
transformwillnotlieexactlyatthebincentres.Iftheparam- 70% Correlated noise 0 0.2 0.4 0.6 0.8 1
Proportion of noise edgels
eterissimplyplacedintheclosestbin,thenthequantization
Fig.3 (Left)Illustrationofthesimulateddata(shownwith100points
causedbythiscausesflatareastoexistinthecostfunction. perline).(Right)Performanceofthealgorithmonthesimulateddata,
Theflatareascaneasilyconfoundtheoptimizer,soinstead, withγ=10−5andγ=2×10−5.Themedianvalueofgammaisshown
theneighbouringbinsareincrementedusinglinearinterpo- witherrorbarsatthe10thand90thpercentile.
lation.
Theoptimizerismosteffectivewhentheparametersbe- 1. Foreachpoint,x(cid:48),inagriddefinedontheoutputimage
ingoptimizedhavesimilarordersofmagnitude,sinceother- I ,findthecorrespondingpointinthedistortedinputim-
1
wiseroundofferrorwillcauseerrorsinthecomputationof ageI givenbyD−1(x(cid:48)).
0
newsimplexes.Inthiscase,asimplescalebasedonapriori 2. Copythepixelback,performingtheassignment:
knowledgeofapproximateparametervaluessuffices.
For radial distortion, c is roughly in the middle of an I1(x(cid:48)):=I0(D−1(x(cid:48))).
image and is of the order 102 to 104 pixels. Given Equa-
Sinceimagesarediscrete,interpolationwillberequired
tion4,itisreasonabletoexpectγ tobeapproximatelyinthe
to find I (D−1(x(cid:48))). Bicubic interpolation has been used to
range±ρ−2,whereρ ≈|c˜|andc˜isthecentreoftheim- 1
max max
generatetheresultsshowninthefollowingsection.
age. Therefore, in the optimization, we solve for β, where
β =100γρ2 ,whichbringsβ intothesamerangeasc.
max
Similarly,wehavefoundthatthevaluesoftheanisotropic
3 Results
coefficients, b ,... are of order 10−3, so we instead solve
1
for d, where d =105b. The resulting optimization is per-
i i i 3.1 Synthetictests
formedoverthevector:
Inordertocomputethesensitivitytovariouskindsofnoise,
[c ,c ,β,d ,d ,d ,d ,d ,d ]. (15)
1 2 1 2 3 4 5 6 wecreateanumberofsyntheticimages,usingknownradial
Therandomselectionofthestartingparametersmustbe distortionlevels,andcomputetheresultingradialdistortion
basedaroundsomeknowledgeofthevalues.Thecentreof correction parameters using our technique. We investigate
radialdistortionisusuallyneartothecentreoftheimage,so the two correction models, namely the strictly radial cor-
we draw the initial centres from a normal distribution cen- rectionoftheunmodifiedHarrismodel,andtheanisotropic
tredatc˜ withastandarddeviationof |c˜|.Giventheapproxi- extensiondiscussedpreviously.Theimagesaredesignedto
20
matevaluesofγ,andthereforeβ,wedrawinitialvaluesof testtheperformanceofthealgorithminthepresenceofmea-
β fromN (0,100).Theinitialanisotropyparametersareset surementerrorintheplumb-lines,andbothuncorrelatedand
to zero. For our application and these parameters, we have correlatednon-plumb-linedata.
foundthat120MCDHiterationsissufficient. The 250×250 pixel test images consist of five good
qualitylines(nopointcanbewithin60pixelsofthecentre
of the image), consisting of 10 points on each line, drawn
2.6 Generatingtheoutputimage with barrel distortion and with random orientation errors.
Then (in the case of uncorrelated noise), a number of ran-
Inordertogeneratetheoutput(corrected)imagethefollow- domly placed, randomly oriented points are added. In the
ingprocedureisneeded: caseofcorrelatednoise,anumberofrandomlyplaced,sized
6
andorientedellipses(withthesameapproximatepointspac- thatthisassumesthatthecentreofradialdistortionisatthe
ingasthelines)areadded.Ouralgorithmisthenappliedto opticaxisofthecamera,whichisareasonableassumption.
thetestimagetoestimatetheparameters. In order to measure the quality of the camera calibra-
For every selected noise proportion for the simulation, tion,welocaliseaknown3Dmodelinanimage,andmea-
200testimagesarecreatedforeachoftwodifferentvalues suretherootmeansquare(RMS)errorbetweenthecontrol
ofγ.Theresultsofthese,andanillustrationofthesimulated points on the model and the measured points in the image.
dataisshowninFigure3.Ascanbeseen,thetechniqueis Themodelconsistsof13by9blackandwhitesquaresina
veryrobusttonoise,producinggoodresultswithupto70% chequerboard pattern on a flat plane. The model is warped
contamination and interestingly it does not perform much using a homography (the parameters of which must be de-
worseinthecaseofcorrelatednoise.Furthermore,although termined), then by the calibrated radial distortion function
the anisotropic extension (in this case) introduces six extra andrenderedintotheimage.Wethensearchnormaltothe
parameterstotheoptimization,thishasnosignificanteffect rendered edges, looking for edges in the image. We define
onthestabilityofthetechniqueinhighnoisesituations. theerrortobethedistanceoffsetinpixelsbetweentheren-
deredmodeledgeandthemeasurededgepositionintheim-
age The parameters of the homography are then adjusted
3.2 Exampleimages usingthedownhill-simplexalgorithmtominimizetheRMS
error. In particular, we search 15 pixels in either direction,
TheresultsontworealimagesareshowninFigure4.Inthe and take the point with the highest gradient, measured us-
correctedimages,real-worldstraightlineshavebeenanno- ingthekernel[−1 −2 −10121].Ifthehighestgradient
tated with lines to show that there is no significant amount does not exceed some threshold, or is not a local maxima,
ofcurvature.Theupperimageshowsthetechniqueoperat- thentheedgesearchisdiscardedanddoesnotcontributeto
ingonanimagewherethelinesbelongtovanishingpoints the RMS error. Quadratic interpolation is then used to find
(note that some distortion was artificially added to this im- theedgepositiontosubpixelaccuracy.Thesearchdistance
age in addition to the camera distortion present). Note the is sufficiently small that data association is trivial, and ro-
presenceofstrongcurvededgesalongtheboundariesofthe bustfittingisnotrequired.Amoredetailedtreatmentofvery
carsandpartsofthebuilding. closelyrelatedsystemsforaligningmodelsisgivenin[12,
The lower image shows the results on an aerial image 8].
ofacity.Althoughtherearetwoprincipaldirectionsinthis The results of this test are shown in Figure 5. As ex-
image,thecontrastontheprincipaledgesislowcompared pected,thetechniquewhichusesmany(inthiscaseapproxi-
to many of the other features. Additionally, the anisotropy mately100)imagesofastructuredsceneperformsbest.The
extensionwasrequiredtocorrecttheverystrongdistortion performance of our method varies somewhat with the con-
inthelowerleftcorner. tentsofthescene,butperformswelloncertainimages.One
interestingpropertyofourtechniqueiswellillustrated.The
propertyisthatedgelsfrommultiple,unrelatedimagescan
3.3 Comparisontoothertechniques
be aggregated to provide more straight edges. In the case
shown, the resulting calibration is superior to the calibra-
Togaugetheaccuracyofourmethod,wehavecomparedit
tiongeneratedfromasingleimageofacarefullyconstructed
to a technique based on structured scenes. In particular we
grid.
haveusedthetechniqueof[10],whichusesmultipleimages
of a planar grid of squares to determine the intrinsic and
extrinsiccameracalibrationparameters.Thissystemisable
4 Discussionandconclusions
tocalibratecubic,quintic,HarrisandHarriswithunitaspect
ratiocameramodels.Weusethelastlistedmodel,sincethis
In this paper, we have presented a new, simple and robust
matchesthecameramodelinthepaper.Inparticular,weuse
methodfordeterminingtheradialdistortionofanimageus-
thisprogramtooptimizethemodel:
ingtheplumb-lineconstraint.Thetechniqueworksbyfirst
(cid:20) (cid:21) (cid:20) (cid:21) (cid:20) (cid:21)(cid:20) (cid:21)
x u f 0 x 1 extractingsalientedgelsandthenminimizingthespreadof
i = + c , (16)
yi v 0 f yc (cid:112)1+α(x2+y2) a 1D angular Hough transform of these edgels. The tech-
c c
niqueissimpleandbecausenoedgefittingisperformed,the
where (u,v) is the optic axis, (xi,yi) is a coordinate in the technique is very robust to the presence of noise. Further-
image,and(xc,yc)isacoordinateinideal,normalizedcam- more,thetechniqueismoregenerallyapplicablethanother
era’simageplane.Sincethetechniqueisastructuredscene plumb-line techniques in that the lines used do not need to
technique, it also estimates the focal length of the camera, be continuous. The technique works on textures with prin-
f.Totranslatefromonemodeltotheother,setγ= α .Note cipal directions, as illustrated by the aerial image of a city,
f2
7
Fig.4 (Left)Original,distortedimages.(Right)Imagesundistortedusingourtechnique.Somelineswhicharestraightintheworldhavebeen
annotatedwithstraightlinesintheundistortedimage.Theimagesareofabuilding,showingstrongvanishingpointsandanaerialimageofacity.
wherethesalientedgedetectionresultsinalargenumberof References
relativelysmalledgefragments.
1. Barreto,J.P.,Daniilidis,K.:Fundamentalmatrixforcameraswith
radialdistortion.In:10thIEEEInternationalConferenceonCom-
Theproposedalgorithmhasanumberofparameters:the puterVision,vol.1,pp.625–632.Springer,Beijing,China(2005)
parameters of the tensor voting kernel, the number of bins 2. Beyer,H.:Accuratecalibrationofccd-cameras. IEEEComputer
and the parameters of the optimization. In practice, the se- SocietyConferenceonComputerVisionandPatternRecognition
lection of these parameters are not critical, and indeed the pp.96–101(1992)
samesetofparameterswasusedforthesimulateddata,the 3. Bra¨uer-Burchardt,C.,Voss,K.:Automaticcorrectionofweakra-
diallensdistortioninsingleviewsofurbanscenesusingvanish-
exampleimagesandthetestimagesshown.
ingpoints. In:9thInternationalConferenceonImageProcessing,
vol.3,pp.865–868.IEEEComputerSociety(2002)
Our method is flexible in that it does not impose con- 4. Brown,D.C.:Close-rangecameracalibration. Photogrammetric
Engineering37,855–866(1971)
straintsbeyondthepresenceofoneormorestraightedges:
5. Claus, D., Fitzgibbon, A.: A rational function lens distortion
itisnotarequirementthattheedgessharevanishingpoints,
modelforgeneralcameras. IEEEComputerSocietyConference
or structure of any particular kind. It is not even a require- onComputerVisionandPatternRecognition1,213–219(2005)
ment that the edgels belong to a related set of images. The 6. Claus,D.,Fitzgibbon,A.W.:Aplumblineconstraintfortheratio-
technique can be equally applied to edgels from multiple nalfunctionlensdistortionmodel. In:ProceedingsoftheBritish
MachineVisionConference,pp.99–108(2005)
imagesofunrelatedscenestakenwiththesamecamerapa-
7. Devernay,F.,Faugeras,O.:Straightlineshavetobestraight. Ma-
rameters. Finally, our method is widely applicable because
chineVisionandApplications13(1)(2001)
it is, in terms of RMS error, able to produce a calibration
8. Drummond,T.,Cipolla,R.:Real-timevisualtrackingofcomplex
to within three percentage points of a technique requiring
structures. IEEETransactionsonPatternAnalysisandMachine
accesstothecameraandstructuredscenes. Intelligence24(7),932–946(2002)
8
Testimages:
1 2 3
4 5 6
Calibrationresults:
Calibration c1 c2 γ×107 RMSerror
Uncalibrated 1024 768.0 0.000 >40‡
libCVDcameracalibration 1030 687.0 3.780 1.67
Testimage1 1075 708.6 3.700 2.38
Testimage2 1325 769.0 3.408 12.2†
Testimage3 1149 627.3 4.317 4.75
Testimage4 1012 739.1 2.814 6.08
Testimage5 1084 611.4 3.544 3.60
Testimage6 1226 968.0 3.405 12.8†
Testimages6and2 1047 725.6 4.075 2.21
Fig.5 Calibrationresultsforanalternativemethodandsometestimages,includingacalibrationgridandsomeimagesofurbanscenes.Allimages
are2048×1536pixels.‡Intheuncalibratedcase,theerrorsaresufficientlythatthesearchradiushadtobeincreasedto100pixels.Thisresults
intheRMSerrorbeinganunderestimate,sinceforlargeerrors,thesystemsometimesfindsacloseredge(sothemeasurederroristoosmall),or
failstofindtheedgeatall(solargeerrorsarenotincluded).†Insomecases,theerrorscausedalargenumberofthecorrespondencestobemissed
withasearchradiusof15pixels,soitwasincreasedto36pixels.Forthelastrowofthetable,theedgelsfromthetwoimageswereaggregated
andtreatedasasingleimage.Inourmodel,c=[c c ]andforthelibCVDmodel,u=c andv=c .
1 2 1 2
9. Duda,R.O.,Hart,P.E.:Useofthehoughtransformationtodetect 16. Lenz,R.,Tsai,R.:Techniquesforcalibrationofthescalefactor
linesandcurvesinpictures. CommunicationsoftheACM15(1), andimagecenterforhighaccuracy3-dmachinevisionmetrology.
11–15(1972) IEEETransactionsonPatternAnalysisandMachineIntelligence
10. Ethan Eade, et. al.: Camera calibration in [20]: progs/calib.cxx 10(5),713–720
(2008,CVStagSNAPSHOT 20080904) 17. Nelder,J.,Mead,R.:ComputerJournal7,308–313(1965)
18. Press,W.H.,Teukolsky,S.A.,Vetterling,W.H.,Flannery,B.P.:Nu-
11. Faird,H.,Popescu,A.C.:Blindremovaloflensdistortion.Journal
mericalRecipesinC. CambridgeUniversityPress(1999)
oftheOpticalSocietyofAmericaA18(9),2072–2078(2001)
19. Rosten,E.,Cox,S.:Accurateextractionofreciprocalspaceinfor-
12. Harris,C.,Stennett,C.:RAPID,avideorateobjecttracker.In:1st
mation from transmission electron microscopy images. In: Ad-
BritishMachineVisionConference,pp.73–77.BritishMachine
vances in Visual Computing. LNCS 4292, vol. 1, pp. 373–382
VisionAssosciation,Oxford(1990)
(2006)
13. Heikkila,J.:Geometriccameracalibrationusingcircularcontrol
20. Rosten, E., Drummond, T., Eade, E., Reitmayr,
points.IEEETransactionsonPatternAnalysisandMachineIntel-
G., Smith, P., Kemp, C., Klein Georg, et. al.:
ligence22(10),1066–1077(2000)
libCVD: Cambridge vision dynamics library (2008).
14. Heikkila¨,J.,Silve´n,O.:Afour-stepcameracalibrationprocedure http://savannah.nongnu.org/projects/libcvd
withimplicitimagecorrection.In:11thIEEEConferenceonCom- 21. Sawhney,H.S.,Kumar,R.:Truemulti-imagealignmentanditsap-
puter Vision and Pattern Recognition, pp. 1106–1112. Springer, plicationtomosaicingandlensdistortioncorrection.IEEETrans-
SanJuan,PuertoRico(1997) actionsonPatternAnalysisandMachineIntelligence21(3),235–
15. Lee,M.,Medioni,G.:Grouping·,–,→,(cid:13)−,intoregions,curves 243(1999)
andjunctions. InternationalJournalofComputerVisionandIm- 22. Shannon,C.M.:Amathematicaltheoryofcommunication. Bell
ageUnderstanding76(1),54–69(1999) SystemTechnicalJournal27,379–423and623–656(1948)
9
23. Stein,G.:Accurateinternalcameracalibrationusingrotation,with
analysisofsourcesoferror.In:5thIEEEInternationalConference
onComputerVision,p.230.Springer,BostonMA,USA(1995)
24. Stein,G.:Lensdistortioncalibrationusingpointcorrespondences.
In:10thIEEEConferenceonComputerVisionandPatternRecog-
nition,p.602.Springer,SanFransisco,California,USA(1996)
25. Strand,R.,Hayman,E.:Correctingradialdistortionbycirclefit-
ting.In:16thBritishMachineVisionConference.BritishMachine
VisionAssosciation,Oxford(2005)
26. Swaminathan,R.,Nayar,S.:Nonmetriccalibrationofwide-angle
lensesandpolycameras. IEEETransactionsonPatternAnalysis
andMachineIntelligence22(10),1172–1178(2000)
27. Tordoff,B.,Murray,D.W.:Theimpactofradialdistortiononthe
self-calibrationofrotatingcameras. ComputerVisionandImage
Understanding96(1),17–34(2004)
A DerivationofJfortheHarrismodel
(cid:20) (cid:21)
rˆ
Takingrˆtobeacolumnvector(i.e.rˆ= 1 ):
rˆ
2
∂D ∂rˆ ∂f(ρ)∂ρ
J= = f(ρ)(1+g(θ))+rˆ(1+g(θ))
∂x ∂x ∂ρ ∂x
∂g(θ(x))
+rˆf(ρ) (17)
∂x
Where:
∂rˆ (cid:34)∂rˆ1 ∂rˆ1(cid:35) 1(cid:20) rˆ2 −rˆ rˆ (cid:21)
= ∂x1 ∂x2 = 2 1 2 , (18)
∂x ∂∂xrˆ21 ∂∂xrˆ22 ρ −rˆ1rˆ2 rˆ12
∂f(ρ) =(1+γρ2)−32, (19)
∂ρ
∂∂ρx =(cid:104)∂∂xρ1 ∂∂xρ2(cid:105)=(cid:2)rˆ1 rˆ2(cid:3), (20)
and:
∂g (cid:104) (cid:105)
= ∂g ∂ρ (21)
∂x ∂x1 ∂x2
Thefunctiong(θ)isafunctionofsinθ andcosθ,where:
cosθ=r1/ρ=rˆ1,and (22)
sinθ=r2/ρ=rˆ2, (23)
wecanrewriteg(θ(x))as:
g(rˆ)=b rˆ +b rˆ +(b rˆ +b rˆ )2+(b rˆ +b rˆ )3, (24)
1 2 2 1 3 2 4 1 5 2 6 1
so,
∂g ∂g(rˆ)∂rˆ (cid:104) (cid:105)∂rˆ
= = ∂g(rˆ) ∂g(rˆ) (25)
∂x ∂rˆ ∂x ∂rˆ1 ∂rˆ2 ∂x
where:
∂g(rˆ)
=b +2b (b rˆ +b rˆ )+3b (b rˆ +b rˆ )2 (26)
∂rˆ1 1 3 3 1 4 2 5 5 1 6 2
∂g(rˆ)
=b +2b (b rˆ +b rˆ )+3b (b rˆ +b rˆ )2. (27)
∂rˆ2 2 4 3 1 4 2 6 5 1 6 2
SubstitutingEquations18,19,20,25,26and27intoEquation17gives
theanalyticsolutionfortheJ.