IEEE Transaction on Image processing (October)

252 Pages·2005·47.025 MB·English

OCTOBER2005 VOLUME14 NUMBER10 IIPRE4 (ISSN1057-7149)
PAPERS
Source/ChannelCoding
FastAlgorithmforDistortion-BasedErrorProtectionofEmbeddedImageCodes.... ... ... ... ... ... ... ... ....
.. ... ... .... ... ... ... ... ... ... ... ... ... ... ... .... ... ... . R.Hamzaoui,V.Stankovic´,andZ.Xiong 1417
MotionDetectionandEstimation
Image Registration Using Log-Polar Mappings for Recovery of Large-Scale Similarity and Projective Transformations
.. ... ... .... ... ... ... ... ... ... ... ...... ... ... ... .... ... ... ... ... ... ... . S.ZokaiandG.Wolberg 1422
Noise Modeling
ImageDenoisingBasedonWaveletsandMultifractalsforSingularityDetection . ... ..... .... ... .J.ZhongandR.Ning 1435
Restoration
FastImageRestorationwithBoundaryArtifacts. .. ... ... ... .... ... .... ..... ... ... ... ... ... ... ..S.J.Reeves 1448
Edge-BasedImageRestoration... ... ... ... ... ...... ... ... .... ... .. A.Rares¸,M.J.T.Reinders,andJ.Biemond 1454
ASpatiallyAdaptiveNonparametricRegressionImageDeblurring.. ..... .... V.Katkovnik,K.Egiazarian,andJ.Astola 1469
Salt-and-PepperNoiseRemovalbyMedian-TypeNoiseDetectorsandDetail-PreservingRegularization .. ... ... ....
.. ... ... .... ... ... ... ... ... ... ... ... ... ... ... .... ... ... .. R.H.Chan,C.W.Ho,andM.Nikolova 1479
(ContentsContinuedonBackCover)
(ContentsContinuedfromFrontCover)
Segmentation
ANonparametricStatisticalMethodforImageSegmentationUsingInformationTheoryandCurveEvolution. ... ....
.. ... ... .... ... ... ... ... ... ... ... ... ... ... J.Kim,J.W.Fisher,III,A.Yezzi,M.Çetin,andA.S.Willsky 1486
SegmentingaLow-Depth-of-FieldImageUsingMorphologicalFiltersandRegionMerging.. ... .... ..... ... ..C.Kim 1503
BayesianImageSegmentationUsingLocalIso-IntensityStructuralOrientation... .....W.C.K.WongandA.C.S.Chung 1512
AdaptivePerceptualColor-TextureImageSegmentation.... .....J.Chen,T.N.Pappas,A.Mojsilovic´,andB.E.Rogowitz 1524
ImageSegmentationandSelectiveSmoothingbyUsingMumford–ShahModel.. ... ..... .... ... .S.GaoandT.D.Bui 1537
Image SequenceProcessing
Rate-DistortionOptimalVideoSummaryGeneration.. ..... .... Z.Li,G.M.Schuster,A.K.Katsaggelos,andB.Gandhi 1550
3-DModel-BasedVehicleTracking. .. ... ... ... ...... ... ... .... .J.Lou,T.Tan,W.Hu,H.Yang,andS.J.Maybank 1561
WaveletsandMultiresolutionProcessing
ImageDecompositionviatheCombinationofSparseRepresentationsandaVariationalApproach ... ... ... ... ....
.. ... ... .... ... ... ... ... ... ... ... ... ... ... ... .... ... ... ..J.-L.Starck,M.Elad,andD.L.Donoho 1570
QuantizationandHalftoning
InverseHalftoningAlgorithmUsingEdge-BasedLookupTableApproach... ... ...... ... ...K.–L.ChungandS.–T.Wu 1583
AuthenticationandWatermarking
ErgodicChaoticParameterModulationwithApplicationtoDigitalImageWatermarking. ..... ....S.ChenandH.Leung 1590
Modeling
“Shape Activity”: A Continuous-State HMM for Moving/Deforming Shapes With Application to Abnormal Activity
Detection. .... ... ... ... ... .... ..... ... ... ... ... ... .. N.Vaswani,A.K.Roy-Chowdhury,andR.Chellappa 1603
Image SearchandSorting
RelevanceFeedbackUsingGeneralizedBayesianFrameworkWithRegion-BasedOptimizationLearning. ... ... ....
.. ... ... .... ... ... ... ... ... ... ... ... ... ... ... .... ... ... ... ... ... ... ...C.-T.HsuandC.-Y.Li 1617
Video Coding
Transform and Embedded Coding Techniques for Maximum Efficiency and Random Accessibility in 3-D Scalable
Compression .. ... ... ... ... ... ... ... ... ..... .... ... .... ... ... ... ... ... ...R.LeungandD.Taubman 1632
InterpolationandSpatialTransformations
ImageUp-SamplingUsingTotal-VariationRegularizationWithaNewObservationModel. .... .. H.A.AlyandE.Dubois 1647
EDICS—Editor’sInformationClassificationScheme.. ... ... .... ... ... ... .... ..... ... ... ... ... ... ... .... 1660
InformationforAuthors.. ... ... ... ... ... ... ... ... ... .... ...... ... ... ... ... ... ... ... ... ... ... .... 1661
ANNOUNCEMENTS
CallforPapers—IEEESignalProcessingSociety2006InternationalWorkshoponMultimediaSignalProcessing . ....... 1663
CallforPapers—IEEEOdyssey2006 . ... ... ... ... ... ... .... ... .... ..... ... ... ... ... ... ... ... ... .... 1664
IEEE SIGNAL PROCESSING SOCIETY
TheSignalProcessingSocietyisanorganization,withintheframeworkoftheIEEE,ofmemberswithprincipalprofessionalinterestinthetechnologyoftransmission,recording,repro-
duction,processing,andmeasurementofspeechandothersignalsbydigitalelectronic,electrical,acoustic,mechanical,andopticalmeans,thecomponentsandsystemstoaccomplish
theseandrelatedaims,andtheenvironmental,psychological,andphysiologicalfactorsconcernedtherewith.AllmembersoftheIEEEareeligibleformembershipintheSocietyand
willreceivethisTRANSACTIONSuponpaymentoftheannualSocietymembershipfeeof$27.00plusanannualsubscriptionfeeof$50.00.Forinformationonjoining,writetotheIEEE
attheaddressbelow.MembercopiesofTransactions/Journalsareforpersonaluseonly.
SOCIETY PUBLICATIONS
Publications Board Chair Trans. on Signal Processing Trans.onImageProcessing Trans. on Speech
ARYE NEHORAI, VP-Publications A. H. SAYED, Editor in-Chief J. KOVAC˘EVIC´ andAudioProcessing
Univ. of Illinois at Chicago Univ. California Editor-in-Chief I.TRANCOSO,Editor-in-Chief
Chicago,IL60607-7053 LosAngeles,CA90095-0001 CarnegieMellonUniv. INESC ID
Pittsburgh,PA15213 Lisboa,Portugal
Signal Processing Letters SP Magazine Trans. on SP, IP, and SAP
A.-J.VAN DERVEEN K. J. R. LIU, Editor-in-Chief SPS Publications Office
Editor-in-Chief Univ. Maryland IEEE Signal Processing Society
DelftUniv.Technol. CollegePark,MD20742 Piscataway,NJ08855
2628CDDelft,TheNetherlands TRANSACTIONS AREA EDITORS
P. MOULIN J. ZERUBIA G. SHARMA
Univ.Illinois INRIA Univ. Rochester
Urbana,IL SophiaAntipolis,France Rochester, N.Y.
TRANSACTIONS ASSOCIATE EDITORS
T. AACH G. DANUSER R. KIMMEL G. MARCU N. K. RATHA I. SELESNICK
Univ. Luebeck Dept.CellBiology Stanford Univ. Apple Computer IBMT.J.WatsonRes.Ctr. PolytechnicUniv.
Luebeck,Germany TheScrippsRes.Ctr. Stanford,CA Cupertino,CA Hawthorne,NY Brooklyn,NY
J. BLANC-TALON LaJolla,CA A. KOKARAM NSi.emMeEnRsLCEoTmputer-Aided STAN REEVES GUn.iSv.HRARoMchAester
DGA/D4S/MRIS R. ESCHBACH Trinity College Diagnosis Auburn Univ. Rochester,NY
France XWeerbosxterC,oNrYp. Dublin,Ireland Jerusalem,Israel AuburnAL L. STANKOVIC
T. BLU Z. FAN A. KUBA CSa.nMofiO-SLIyNnAthelaboRecherche NIn.st.RNOUatG.OTNelecommun. UPondivg.oMricoan,tSeneerbgirao,
EPFL Xerox Corp. Univ. Szeged LaBege,France Evry,France andMontenegro
LJ.auMsa.nBnUe,HSMwAiNtzNerland WMe.bAst.eTr,.NFYIGUEIREDO SRz.egLeEdN,ZHungary AUn.ivN.OTSeRxAaTsINaItADallas ER.ocShAesBtEeRrInstit.ofTechnol. SUP.ondiSvgT.oAMrNicKoanO,tVSeIneCerbgirao,
ETH InstitutoSuperiorTecnico Linköping Univ. Richardson,TX Rochester,NY andMontenegro
Zurich,Switzerland Lisboa,Portugal Norrköping,Sweden F. M. B. PEREIRA T. SZIRANYI
BUBa.nriCov.nAiSRstsPuiE,dNIitTadIlEiyRSIalerno HUOnr.liavFn.OdCRoOe,nOFtSLrHalFlorida RXW.eerboLsxOteCrC,EoNrYp. ILGniIssOtbiVtoAuatN,oNPSIourPtpuOegGraiGolIrTecnico AHPae.lwoSlAeAtIltDt-oP,aCckAard MBDu.TdATaApSUeszBtTM,AHAKuNnIgary
Universita’FedericoIIdiNapoli Univ.NewSouthWales
VINCENTCASELLES ODo.CGo.MGoUCLEoRmYmUZunications L. LUCCHESE Napoli,Italy PU.PSCALEMBIER Sydney,Australia
UniversitatPimpeuFabra LaboratoriesUSA,Inc. Oregon State Univ. R. RAJKUMAR Barcelona,Spain T. D. TRAN
Barcelona,Spain SanJose,CA Corvalis,OR CarnegieMellonUniv. JohnsHopkinsUniv.
Pittsburgh,PA M. SAMUELIDES Baltimore,MD
H. CHOI Z. KATO B. MACQ G. RAMPONI EcoleNationalSuperieurede Z. XIONG
Rice Univ. Univ. Szeged Univ.CatholiquedeLouvain Univ. Trieste l’Aeronautiqueetdel’Espace TexasA&MUniv.
Houston,TX Szeged,Hungary Louvain-la-Neuve,Belgium Trieste,Italy Toulouse,France CollegeStation,TX
Officers
W. CLEON ANDERSON, President and CEO LEAHH.JAMIESON,VicePresident,PublicationServicesandProducts
MICHAEL R. LIGHTNER, President-Elect MARCT.APTER,VicePresident,RegionalActivities
MOHAMED EL-HAWARY, Secretary DONALDN.HEIRMAN,President,IEEEStandardsAssociation
JOSEPH V. LILLIE, Treasurer JOHN R.VIG, Vice President,Technical Activities
ARTHUR W. WINSTON, Past-President GERARDA.ALPHONSE,President,IEEE-USA
MOSHEKAM,VicePresident,EducationalActivities
JOHNA.REAGAN,Director,DivisionIX—SignalsandApplications
Executive Staff
DONALD CURTIS, Human Resources MATTHEWLOEB,CorporateStrategy&Communications
ANTHONY DURNIAK, Publications Activities RICHARD D. SCHWARTZ, Business Administration
JUDITH GORMAN, Standards Activities CHRIS BRANTLEY, IEEE-USA
CECELIA JANKOWSKI, Regional Activities MARY WARD-CALLAN, Technical Activities
BARBARACOBURNSTOLER,EducationalActivities SALLYA.WASELIK,InformationTechnology
IEEE Periodicals
Transactions/Journals Department
Staff Director: FRAN ZAPPULLA
Editorial Director: DAWN MELLEY Production Director: ROBERT SMREK
ManagingEditor:MARTINJ.MORAHAN AssociateEditor:MEGANVOREL
IEEETRANSACTIONSONIMAGEPROCESSING(ISSN1057-7149)ispublishedmonthlybytheInstituteofElectricalandElectronicsEngineers,Inc.Responsibilityforthecontentsrests
upontheauthorsandnotupontheIEEE,theSociety/Council,oritsmembers.IEEECorporateOffice:3ParkAvenue,17thFloor,NewYork,NY10016-5997.IEEEOperations
Center:445HoesLane,P.O.Box1331,Piscataway,NJ08855-1331.NJTelephone:[phone].Price/PublicationInformation:Individualcopies:IEEEMembers$20.00(first
copyonly),nonmembers$89.00percopy.(Note:Postageandhandlingchargenotincluded.)Memberandnonmembersubscriptionpricesavailableuponrequest.Availableinmicrofiche
andmicrofilm.CopyrightandReprintPermissions:Abstractingispermittedwithcredittothesource.Librariesarepermittedtophotocopyforprivateuseofpatrons,providedthe
per-copyfeeindicatedinthecodeatthebottomofthefirstpageispaidthroughtheCopyrightClearanceCenter,222RosewoodDrive,Danvers,MA01923.Forallothercopying,reprint,
orrepublicationpermission,writetoCopyrightsandPermissionsDepartment,IEEEPublicationsAdministration,445HoesLane,P.O.Box1331,Piscataway,NJ08855-1331.Copyright
©2005bytheInstituteofElectricalandElectronicsEngineers,Inc.Allrightsreserved.PeriodicalsPostagePaidatNewYork,NYandatadditionalmailingoffices.Postmaster:Send
addresschangestoIEEETRANSACTIONSONIMAGEPROCESSING,IEEE,445HoesLane,P.O.Box1331,Piscataway,NJ08855-1331.GSTRegistrationNo.125634188.PrintedinU.S.A.
DigitalObjectIdentifier10.1109/TIP.2005.857134
IEEETRANSACTIONSONIMAGEPROCESSING,VOL.14,NO.10,OCTOBER2005 1417
Fast Algorithm for Distortion-Based Error
Protection of Embedded Image Codes
RaoufHamzaoui,Member,IEEE,VladimirStankovic´,Member,IEEE,andZixiangXiong,SeniorMember,IEEE
Abstract—We consider a joint source-channel coding system
that protects an embedded bitstream using a finite family of
channel codes with error detection and error correction capa-
bility. The performance of this system may be measured by the Fig. 1. (Top) Fixed-length channel codewords with variable-length
expected distortion or by the expected number of correctly de- information blocks. (Bottom) Fixed-length information blocks with
coded sourcebits. Whereas arate-based optimalsolution canbe variable-lengthchannelcodewords.Thewhiteareascorrespondtoinformation
bitsandtheshadedareastoprotectionbits.
foundinlineartime,thecomputationofadistortion-basedoptimal
solution is prohibitive.Under the assumption of the convexity of
the operational distortion-rate function of the source coder, we
error probability is an affine function of the channel packet
givealowerboundontheexpecteddistortionofadistortion-based
length, the channel code rates of an optimal solution should
optimalsolutionthatdependsonlyonarate-basedoptimalsolu-
tion. Then, we propose a local search (LS) algorithm that starts be nondecreasing in the information block number, which
fromarate-basedoptimalsolutionandconvergesinlineartimeto significantly reduces the complexity of an exhaustive search.
alocalminimumoftheexpecteddistortion.Experimentalresults Also in the fixed-length information block case, Chande and
for a binary symmetric channel show that our LS algorithm is
Farvardin [5] provide a dynamic programming solution to the
near optimal, whereas its complexity is much lower than that of
optimization problem and report a quadratic time complexity
thepreviousbestsolution.
in the target transmission rate. In the fixed-length channel
IndexTerms—Jointsource-channelcoding,imagetransmission,
codeword case, however, no fast exact solution is known.
localsearch,unequalerrorprotection.
To the best of our knowledge, the best approximation to an
optimal solution was given in [2]. It is based on a Viterbi
I. INTRODUCTION algorithm and has a quadratic time complexity in the number
ONE OF THE most efficient systems for the progressive of transmitted channel codewords . However, this result is
transmissionofimagesovermemorylessnoisychannels guaranteed only for channel code rates that are a subset of
, where and are positive
without feedback was proposed by Sherwood and Zeger [1].
integers with . For channel codes that do not fulfill
Thebasicideaistouseanembeddedwaveletcoderforsource
this condition, including rate-compatible punctured codes, the
codingandaconcatenationofanoutercyclicredundancycheck
(CRC) coderand an innerrate-compatiblepuncturedconvolu- worstcasetimecomplexityisexponentialin .
An alternative to minimizing the expected distortion is to
tional (RCPC) coder for channel coding. Error propagation is
maximizetheexpectednumberofcorrectlydecodedsourcebits
avoided by stopping the decoding when the first error is de-
(this approach was introduced by Chande et al. in [6] for a
tected.Intheoriginalsetting[1],theinformationbitswereor-
ganized in blocks of fixed length, each of which was mapped similar system that uses a feedback channel). Though subop-
timal in the distortion sense, the rate-based optimization has
toachannelcodewordofavariablelength.However,itismore
twomainadvantages:anoptimalsolutioncanbecomputedwith
convenienttofixthesizeofthechannelcodewordsandtoallow
a linear-time algorithm (see [5] for fixed-length information
theblocksofinformationbitstohaveavariablelength[2],[3]
(Fig. 1). blocksand[3]forfixed-lengthchannelcodewords),anditisin-
dependentofboththesourcecoderperformanceandtheimage;
Achallengingproblemforthissystemistofindanallocation
thus,itcanbedeterminedbythereceiver,whichavoidstheneed
of the total transmission rate between the source coder and
forsideinformation.In[5],experimentalresultsshowthatthe
the channel coder that minimizes the expected distortion. For
the fixed-length information block case, Lu et al. [4] show solutionstothetwooptimizationproblemshaveasimilarper-
that, by assuming that the logarithm of the block decoding formancefortheSPIHTcoder[7]andanRCPCchannelcoder
[thelossinaveragepeaksignal-to-noiseratio(PSNR)wasless
than0.2dBforthe512 512grayscaleLennaimage].Hedayat
ManuscriptreceivedMarch3,2004;revisedSeptember10,2004.Thispaper
andNosratinia[8]analyticallyconfirmtheseresultsundermany
waspresentedinpartatDCC’02,theDataCompressionConference,Snowbird,
UT,April2002.Theassociateeditorcoordinatingthereviewofthismanuscript assumptions, including an independent identically distributed
andapprovingitforpublicationwasDr.AriaNosratinia. Gaussian source and a perfect progressive source coder that
R.HamzaouiiswiththeDepartmentofComputerandInformationScience,
achievestherate-distortionfunction.
University of Konstanz, 78457 Konstanz, Germany (e-mail: hamzaoui@
inf.uni-konstanz.de). Inthispaper,westudytherate-basedandthedistortion-based
V. Stankovic´ and Z. Xiong are with the Department of Electrical Engi- optimization problems in the context of fixed-length channel
neering, Texas A&M University, College Station, TX 77843 USA (e-mail:
codewords.InSectionIII,weprovideatheoreticalupperbound
[email protected];[email protected]).
DigitalObjectIdentifier10.1109/TIP.2005.854497 onthedifferenceinexpecteddistortionbetweenthesolutionsof
1057-7149/$20.00©2005IEEE
1418 IEEETRANSACTIONSONIMAGEPROCESSING,VOL.14,NO.10,OCTOBER2005
thetwooptimizationproblemsundertheassumptionofthecon- the minimization of (1) by the maximization of the expected
vexity of the operational distortion-rate function of the source numberofcorrectlyreceivedsourcebits
coder.Wealsoconjecturethatthetotalnumberofinformation
bits of a distortion-based optimal solution is smaller than or
equal to that of a rate-based optimal solution. In Section IV, (2)
we propose a fast local search (LS) algorithm that starts from
arate-basedoptimalsolutionandtriestominimizetheexpected
thenanoptimalsolutioncanbecomputedin time[3].
distortionbysuccessivelyreducingthenumberofinformation
Maximizing (2) is reasonable for an efficient embedded coder
bits (or equivalently by increasing the number of protection
becauseweexpecttheaveragedistortiontodecreasewhenthe
bits). Section V presents numerical results for SPIHT [7] and
average number of correctly received source bits increases.
JPEG2000 [9]. We show, in particular, that the LS algorithm
Note, however, that the two optimizations do not necessarily
yields a comparable solution to that obtained with the Viterbi
yield the same EPS (see Section V). In the following, we say
algorithmof[2],butatamuchlowercomplexity.
thatanEPSthatminimizes(1)isdistortionoptimalandthatan
EPSthatmaximizes(2)israteoptimal(RO).
II. OPTIMIZATIONCRITERIA InanROsolution,thechannelcoderatesarenondecreasing
with the packet number [3]. This is not necessarily true in a
Weconsiderasource-channelcodingsystemthatusesanem- distortion-optimal solution as shown by the following counter
beddedsourcecoderandafinitefamily ofchannel example.Supposethatwehavetwopackets( )andtwo
codes with error detection and error correction capability. We channel code rates and with ( ). Then
recall that a source coder is called embedded if for any inte- , , ,and arethefourpossibletwo-
gers and with ,theoutputbitstreamoflength packetEPSs.Suppose,now,that and .
is a prefix of the output bitstream of length . Given a Let , ,
transmission bit budget and a channel packet size , the , ,
channelencodertransforms successiveblocksof ,and .Then
thesourcecoderoutputbitstreamintoasequenceof channel
codewordsoffixedlength .Supposethatwewanttosendsuc-
cessivelythe packets overa memorylessnoisy channeland
let ( )bethesetofcode
ratescorrespondingto .Then,weusean -packet
Thus, the EPS is distortion optimal but . An
error protection scheme (EPS) ,
-packet EPS that minimizes (1) under the constraint
which encodes the th information block with a channel code
willbecalledconstraineddistortionoptimal.Inthe
rate .Ifthedecoderdetectsanerror,thenthedecoding
aboveexample, isaconstraineddistortion-optimalEPS.
isstopped,andtheimageisreconstructedfromthecorrectlyde-
Note thatthe constrainedminimization reduces the number of
codedpackets.Weassumethatallerrorscanbedetected.
candidatesfrom to .
For , let denote the probability of a de-
codingerrorinapacketoflength protectedbycode .We
may assume without loss of generality that III. RATE-BASEDVERSUSDISTORTION-BASEDPROTECTION
. For the -packet EPS , the
When islarge,thesearchspaceistoolargetoallowthede-
probability of a decoding error in the first packet is
terminationofadistortionoptimalEPSbyexhaustivesearch.In-
,theprobabilitythatnodecodingerrorsoccurinthefirst
steadofadistortionoptimalsolution,onecoulduseanROone,
packets, , with an error in the next one is
butwhatisinthiscasethelossinquality?Thefollowingpropo-
, and the probabilitythat
sitionshowshowtocomputeatightupperboundonthequality
all packets are correctly decoded is
lossifweassumethattheoperationaldistortion-ratefunctionof
.Thus,theexpecteddistortionforthe -packetEPS
thesourcecoderisnonincreasingandconvex.
is
Proposition1: Let betheoperationaldistortion-ratefunc-
tion of the source coder. Suppose that is nonincreasing and
convex.Let beadistortion-optimal -packetEPSandlet
(1)
beanRO -packetEPS.Then, .
Proof: Let be an -packet EPS. Then,
where isaconstant,andfor , isthere- . Thus, since is convex, Jensen’s
constructionerrorusingthefirst packets.Notethatif denotes inequalitygives .Ontheotherhand,
theoperationaldistortion-ratefunctionofthesourcecoder,then, . Since is nonincreasing, this gives
for , , where and ,whichcompletestheproof.
for , with being The proposition says that the approximation error
thenumberofsourcebitsinthe thpacket.Sincethenumberof is bounded by
possible -packetEPSsisequalto ,bruteforcecannotbe ,whichcanbeeasilycomputedbecause can
usedtominimize(1)when islarge.However,ifwereplace bedeterminedin timewiththealgorithmof[3].
HAMZAOUIetal.:FASTALGORITHMFORDISTORTION-BASEDERRORPROTECTIONOFEMBEDDEDIMAGECODES 1419
Thefollowingconjecture1comparesunderthesameassump- largest code rate smaller than and de-
tionthetotalnumberofinformationbitsforanROandadistor- fine to be the EPS obtained from by
tion-optimalEPSs. protecting packet with .
Conjecture1: Let betheoperationaldistortion-ratefunc- 3. If , set ,
tion of the source coder. Suppose that is nonincreasing and , , , and go to step 2.
convex. Let be a distortion-optimal -packet EPS and let 4. If and is greater than the rate
beanRO -packetEPS.Then, andthe of packet in , set . If
inequalityisstrictif isnotRO. and is equal to the rate of packet ,
Theconjectureiscorroboratedbyalloursimulations,andit set and . If and ,
canbeprovedfor (see[11]). set . If and , stop.
5. Go to step 2.
IV. LOCALSEARCHALGORITHM
In this section, we propose a heuristic LS algorithm that Proposition 2: The LS algorithm converges after a finite
rapidlyfindsalocalminimumoftheexpecteddistortion(1)(see number of steps. Moreover, its time complexity is in
[12,p.3]forthedefinitionofalocalminimum).Experimental theworst case.
results in Section V show that this local minimum is near a Proof: When the current solution is updated, one
globalone.Wefirstdefinetheneighborsofasolution. channel code rate is decreased while all the other ones are
Definition 1: Let be a set of code rates and let kept fixed. This ensures convergence in a finite number of
be an -packet EPS with steps. More precisely, the initialization (step 1) requires at
nondecreasingcoderates.Wesaythat most steps [3]. In the worst case, the refinement part
isaneighborof if of the algorithm starts from the -packet EPS
a) ; andconvergesto in steps.Thus,thetime
b) thereexistsaunique suchthat ; complexityis intheworstcase.
c) thecoderatesof arenondecreasing. Simulationsshowthatthealgorithmcanbeslightlyimproved
Wedenotethesetofneighborsof by andthecode byremoving fromstep3(thismodificationchangesthe
rate inDefinition1b)by . setofneighborsofasolution).Weusedthisvariantinourex-
We also sort the neighbors of by order of decreasing perimentalresults.
.Moreprecisely,forpositiveinteger ,the thneighbor
of an EPS is the EPS such that is the V. RESULTS
th largest code rate in the set . For
In this section, we compare the time complexity and the
example, let and
expected mean-squared error (MSE) of an RO solution com-
be a four-packet EPS. Then has three neighbors,
putedwiththealgorithmof[3],thesolutioncomputedwiththe
being the firstone,
Viterbi algorithm of [2] (this algorithm uses the monotonicity
the second, and the third. Note that
constraint on the channel code rates), and the solution of the
.
LSalgorithmofSectionIV.TheCPUtimewasmeasuredona
OurLSalgorithmworksbyiterativeimprovement.Westart
1466-MHz AMD Athlon XP 1700 processor. The test images
fromanROsolution.Then,weconsiderthefirstneighborofthis
were the standard 8 bits per pixel (bpp) grayscale 512 512
solution. If the expected distortion of this neighbor is smaller
Lenna, Goldhill, and Peppers. The embedded source coders
thanthatofthecurrentsolution,thenweupdatethecurrentso-
were Jim Fowler’s implementation of the SPIHT algorithm
lutionandrepeattheprocedure;otherwise,weconsiderthenext
[13] and the Kakadu implementation of JPEG2000 in the
neighborandrepeattheprocedure.Ifthereisnoneighborthat
distortion scalable mode [9]. Note that the operational dis-
isbetterthanthecurrentsolution,thealgorithmreturnsthecur-
tortion-rate curves of these coders can be well modeled with
rentsolution.Notethat,motivatedbyConjecture1,allsolutions
nonincreasing convex functions [14]. The packets were sent
testedbythealgorithmhavefewerinformationbitsthananRO
overabinarysymmetricchannel(BSC).Werecallthat and
EPS.ApseudocodeforourLSalgorithmisgivenbelow.
denotethenumberofpacketssentandthelengthofachannel
codeword, respectively. Thus, the transmission rate in bpp is
LS algorithm for images.
Initialization. 1. Set , , and Inafirstexperiment,theembeddedbitstreamwasprotected
. Use the algorithm of [3] to compute withaconcatenationofaCRC-32coderandarate-compatible
an RO -packet EPS . puncturedturbo(RCPT)coder[15].Thegeneratorpolynomial
Refinement. of the CRC code was (32,26,23,22,16,12,11,10,8,7,5,4,2,1,0).
2. Let be the th largest code rate used The turbo coder consisted of two identical recursive system-
by . Let be the index of the first atic convolutional coders with memory length four and gen-
packet that protects with . If , erator polynomials (31, 27) (octal). The mother code rate was
stop. Otherwise, let be the th ,andthepuncturingratewas20,yielding41pos-
sible channel code rates. The length of a packet was equal to
1Thisconjecturewaserroneouslygivenasapropositionin[10]. 2048bits,consistingofavariablenumberofsourcebits,
1420 IEEETRANSACTIONSONIMAGEPROCESSING,VOL.14,NO.10,OCTOBER2005
TABLE I TABLE III
EXPECTEDMSEATVARIOUSTRANSMISSIONRATESFORAROSOLUTION, EXPECTEDMSEATVARIOUSTRANSMISSIONRATESFORAROSOLUTION,
ASOLUTIONFOUNDWITHTHEVITERBIALGORITHM[2],ANDONE ASOLUTIONFOUNDWITHTHEVITERBIALGORITHM[2],ANDONE
OBTAINEDWITHOURLSALGORITHM.RESULTSAREFORTHE OBTAINEDWITHOURLSALGORITHM.RESULTSAREFORTHE
512(cid:2)512LENNAIMAGE,THESPIHTSOURCECODER,AN 512(cid:2)512PEPPERSIMAGE,THESPIHTSOURCECODER,AN
RCPTCHANNELCODER,ANDABSCWITHBER=0:01 RCPTCHANNELCODER,ANDABSCWITHBER=0:01
TABLE II
EXPECTEDMSEATVARIOUSTRANSMISSIONRATESFORAROSOLUTION,
ASOLUTIONFOUNDWITHTHEVITERBIALGORITHM[2],ANDONE TABLE IV
OBTAINEDWITHOURLSALGORITHM.RESULTSAREFORTHE EXPECTEDMSEATVARIOUSTRANSMISSIONRATESFORAROSOLUTION,A
512(cid:2)512GOLDHILLIMAGE,THESPIHTSOURCECODER,AN SOLUTIONFOUNDWITHTHEVITERBIALGORITHM[2]ANDONEOBTAINED
RCPTCHANNELCODER,ANDABSCWITHBER=0:01 WITHOURLSALGORITHM.RESULTSAREFORTHE512(cid:2)512LENNAIMAGE,
JPEG2000,THECHANNELCODERUSEDIN[2],ANDABSCWITHBER=0:1
32 CRC bits, 4 bits to set the turbo encoder into a state of all
zeroes,andprotectionbits.Weusediterativemaximumapos- Thefastestalgorithmwastheoneof[3],and,exceptforthe
terioridecoding,whichwasstoppedifnocorrectsequencewas lowest transmission rates, it provided high-quality solutions.
foundafter20iterations. TheLSalgorithmwasalwaysabletoimprovethesolutionsof
Theprobabilityofapacketdecodingerrorforeachchannel [3]. Moreover, its performance was close to the lower bound,
coderatewascomputedwithsimulations.Weusedonlyasubset whileitsCPUtimewasupto timeslowerthanthatofthe
ofthesetof41admissibleRCPTcoderates.Indeed,whenmany Viterbi algorithm. For example, for transmission rate 1 bpp,
code rates have the same decoding error probability, only the 128 packets were sent. Thus, the total number of candi-
largest one has to be kept. Also, one can ignore any code rate datesis ,andthenumberofcandidateswithnondecreasing
whose residual bit-error rate is greater than the bit-error rate coderates is 7177979000.Forthe Lenna image,the
(BER).ForBER0.01,theretainedcoderateswere , LS algorithm checked only 112 candidates in the refinement
, , , , stage. In contrast, the total number of considered paths in the
, and . The corresponding numbers of in- Viterbitrelliswasequalto3649922,ofwhich52661hadthe
formation bits per packet were , , maximumlengthof128.
, , , , Wealsotestedthealgorithmsforaconcatenationofthe16-bit
and . The probabilities of packet decoding error CRC coder used in [1] and an RCPC coder with mother code
were , , , memorylength6,mothercoderate1/4,generatorpolynomials
, , , and (147,163,135,135),andpuncturingperiod8.Here,thepacket
. lengthwas 512bits.Thesimulationsweredoneforvarious
Table I compares the performance of the algorithms for the BERs(0.1,0.01,0.001).Inallcases,ourLSalgorithmwassig-
SPIHT bitstream of the Lenna image. The lower bound of nificantlyfasterthantheViterbialgorithm.
Proposition1isalsoprovided.Wepointoutthatthetransmis- TheViterbialgorithmof[2]ismuchfasterforchannelcode
sion rate does not include the overhead needed to specify the rates in , where .
solutionwhenadistortion-basederrorprotectionisused.Since We now compare the LS algorithm with the Viterbi algorithm
we consider only EPSs with nondecreasing code rates and for such channel code rates. Let us consider, for example, the
since generally , one can use run-length encoding to puncturedturbocodesusedin[2].Here, 512bytes,andthe
compresstheoverheadto bits coderatesare11/12,10/12,9/12,8/12,6/12,5/12,and4/12.For
in the worst case. Alternatively, the encoder can approximate BER ,onlycoderates4/12,5/12,and6/12withrespective
the distortion-rate curve with a parametric model [14] and packet decoding error probabilities 0.00001, 0.0003, and 0.88
sendonlythemodelparameterstothedecoder,whichcanthen arerelevant.Evenwiththesettingsof[2],theLSalgorithmwas
recomputethesolutiononitsside. uptotentimesfasterthantheViterbialgorithm.Moreover,the
Tables II and III show the results for Goldhill and Peppers, MSE performance of the two algorithms was almost identical
respectively.WeobtainedsimilarresultsforBER0.1(see[11]). (Table IV).
HAMZAOUIetal.:FASTALGORITHMFORDISTORTION-BASEDERRORPROTECTIONOFEMBEDDEDIMAGECODES 1421
VI. CONCLUSION [9] D.TaubmanandM.Marcellin,JPEG2000:ImageCompressionFunda-
mentals,StandardsandPractice. Norwell,MA:Kluwer,2001.
We provided an easily computable tight lower bound [10] R.Hamzaoui,V.Stankovic´,andZ.Xiong,“Rate-basedversusdistortion-
on the smallest possible expected distortion for the joint basedoptimaljointsource-channelcoding,”inProc.DataCompression
Conf.,J.A.StorerandM.Cohn,Eds.,Snowbird,UT,Apr.2002,pp.
source-channel coding system of [2], [3]. This lower bound is
63–72.
usefultoevaluatethequalityofapproximatesolutions.Wealso [11] R. Hamzaoui, V. Stankovic´, and Z. Xiong. (2003) “Rate-based
proposed an LS algorithm that finds a high-quality local min- versusdistortion-basedoptimal errorprotectionofembedded codes,”
Konstanzer Schriften in Mathematik und Informatik, Preprint no.
imumoftheexpecteddistortion.Ourgoalwastominimizethe
194. Univ. Konstanz, Konstanz, Germany. [Online]. Available:
expected distortion; the extension to maximizing the expected http://www.inf.uni-konstanz.de/Preprints/
PSNRisstraightforward. [12] E.H.L.AartsandJ.K.Lenstra,Eds.,LocalSearchinCombinatorial
Optimization. NewYork:Wiley,1997.
[13] J.E.Fowler,“Anopensourcesoftwarelibraryforquantization,com-
ACKNOWLEDGMENT pressionandcoding,”inProc.ApplicationsofDigitalImageProcessing,
vol.SPIE4115,Aug.2000,pp.294–301.
TheauthorswouldliketothankM.Hillerforimplementing
[14] Y.Charfi,R.Hamzaoui,andD.Saupe,“Model-basedreal-timeprogres-
theViterbialgorithmof[2],G.Constantinescuforhelpingwith sivetransmissionofimagesovernoisychannels,”inProc.IEEEWCNC,
theprogramming,andB.Banisterforfruitfuldiscussions. vol.2,NewOrleans,LA,Mar.2003,pp.784–789.
[15] D. N. Rowitch and L. B. Milstein, “On the performance of hybrid
FEC/ARQ systems using rate compatible punctured turbo (RCPT)
REFERENCES codes,”IEEETrans.Commun.,vol.48,no.6,pp.948–959,Jun.2000.
[1] P.G.SherwoodandK.Zeger,“Progressiveimagecodingfornoisychan-
nels,”IEEESignalProcess.Lett.,vol.4,no.5,pp.189–191,May1997.
[2] B.A.Banister,B.Belzer,andT.R.Fischer,“Robustimagetransmission
usingJPEG2000andturbo-codes,”IEEESignalProcess.Lett.,vol.9,
no.4,pp.117–119,Apr.2002.
[3] V.Stankovic´,R.Hamzaoui,andD.Saupe,“Fastalgorithmforrate-based RaoufHamzaoui,photographandbiographynotavailableatthetimeofpub-
optimalerrorprotectionofembeddedcodes,”IEEETrans.Commun., lication.
vol.51,no.11,pp.1788–1795,Nov.2003.
[4] J. Lu, A. Nosratinia, and B. Aazhang, “Progressive source-channel
codingofimagesoverburstyerrorchannels,”inProc.IEEEInt.Conf.
ImageProcessing,Chicago,IL,Oct.1998,pp.127–131.
[5] V.ChandeandN.Farvardin,“Progressivetransmissionofimagesover
memorylesschannels,”IEEEJ.Sel.AreasCommun.,vol.18,no.6,pp.
850–860,Jun.2000. VladimirStankovic´,photographand biographynotavailableatthetimeof
[6] V. Chande, H. Jafarkhani, and N. Farvardin, “Joint source-channel publication.
codingofimagesforchannelswithfeedback,”presentedattheIEEE
InformationTheoryWorkshop,SanDiego,CA,Feb.1998.
[7] A.SaidandW.A.Pearlman,“Anewfastandefficientimagecodecbased
onsetpartitioninginhierarchicaltrees,”IEEETrans.CircuitsSyst.Video
Technol.,vol.6,no.6,pp.243–250,Jun.1996.
[8] A.HedayatandA.Nosratinia,“Rateallocationinsource-channelcoding
ofimages,”inProc.IEEEInt.Conf.ImageProcessing,vol.1,Thessa- ZixiangXiong,photographandbiographynotavailableatthetimeofpublica-
loniki,Greece,Oct.2001,pp.189–192. tion.
1422 IEEETRANSACTIONSONIMAGEPROCESSING,VOL.14,NO.10,OCTOBER2005
Image Registration Using Log-Polar Mappings
for Recovery of Large-Scale Similarity and
Projective Transformations
Siavash Zokaiand George Wolberg, Senior Member, IEEE
Abstract—This paper describes a novel technique to recover fromdifferentsensors(i.e.,multisensordatafusion),2)finding
large similarity transformations (rotation/scale/translation) and changes in images taken at different times or under different
moderate perspective deformations among image pairs. We in-
conditions, 3) inferring three-dimensional (3-D) information
troduce a hybrid algorithm that features log-polar mappings
from images in which either the camera or the objects in the
and nonlinear least squares optimization. The use of log-polar
techniquesinthespatialdomainisintroducedasapreprocessing scenehavemoved,and4)formodel-basedobjectrecognition.
moduletorecoverlargescalechanges(e.g.,atleastfour-fold)and The most common task associated with image registration
arbitraryrotations.Althoughlog-polartechniquesareusedinthe is the generation of large panoramic images for viewing and
Fourier–Mellin transform to accommodate rotation and scale in
analysis. Image mosaics, created by warping and blending
the frequency domain, its use in registering images subjected to
togetherseveraloverlappingimages,arecentraltothisprocess.
verylargescalechangeshasnotyetbeenexploitedinthespatial
domain.Inthispaper,wedemonstratethesuperiorperformance Othercommonregistrationtasksincludeproducingsuper-reso-
ofthelog-polartransforminfeaturelessimageregistrationinthe lutionimagesfrommultipleimagesofthesamescene,change
spatial domain. We achieve subpixel accuracy through the use detection, motion stabilization, topographic mapping, and
of nonlinear least squares optimization. The registration process
multisensorimagefusion.
yieldstheeightparametersoftheperspectivetransformationthat
This work attempts to register two images using one global
bestalignsthetwoinputimages.Extensivetestingwasperformed
on uncalibrated real images and an array of 10,000 image pairs perspectivetransformationeveninthepresenceofarbitraryro-
withknowntransformationsderivedfromtheCorelStockPhoto tation angles and large scale changes (up to 5 zoom). Our
Libraryofroyalty-freephotographicimages. work is motivated by the problem of registering airborne im-
Index Terms—Image registration, Levenberg–Marquardt non- ages.Theseimagesaretakenatvastlydifferenttimes,altitudes,
linearleast-squaresoptimization,log-polartransform,perspective anddirections.Therefore,theimagesdifferbylargerotationand
transformation,similaritytransformation. scale.Also,thepitchandrollintroducesmoderateperspective.
Ingeneral,imagesofa3-Dscenedonotdifferbyjustoneper-
spectivetransformationbecausethedepthbetweenthecamera
I. INTRODUCTION
and the objects introduces parallax. A global transformation
DIGITALimageregistrationisabranchofcomputervision
cannotalignallfeaturesinsuchcases.Wemust,therefore,place
that deals with the geometric alignment of a set of im-
constraintsoncameramotionand/orour3-Dscenetoproduce
ages.Thesetmayconsistoftwoormoredigitalimagestaken
images that are free of parallax. One constraint requires the
of a single scene at different times, from different sensors, or
cameramotiontobelimitedtorotation,pan,tilt,andzoomabout
from different viewpoints. A large body of research has been
afixedpoint,e.g,onatripod.Ifthisconstraintisnotsatisfied,
drawntothisareaduetoitsimportanceinremotesensing,med-
thenwemaystillhaveimagesfreeofparallaxiftheobject’s3-D
icalimaging,computergraphics,andcomputervision.Despite points inthescenearefarawayfromthecamera,i.e.,
comprehensiveresearchspanningoverthirtyyears,robusttech- .Thismeansthatthesceneisflatandwearelooking
niquestoregisterimagesinthepresenceoflargedeformations at a planar object. In either case, we assume that the scene is
remains elusive. Most techniques fail unless the input images staticandthelightingisfixedbetweenimages.Nevertheless,we
aremisalignedbymoderatedeformations. have relaxed these conditions to accommodate local disparity
Thegoalofregistrationistoestablishgeometriccorrespon- andlinearchangesinillumination.
dence between the images so that they may be transformed, AsurveybyBrown[1]introducesaframeworkinwhichall
compared,andanalyzedinacommonreferenceframe.Regis- registrationtechniquescanbeunderstood.Theframeworkcon-
tration is often necessary for 1) integrating information taken sistsofthefeaturespace,similaritymeasure,searchspace,and
search strategy. The feature space extracts the information in
ManuscriptreceivedApril27,2004;revisedOctober11,2004.Thisworkwas theimagesthatwillbeusedformatching.Thesearchspaceis
supportedinpartbyanONRHBCU/MIResearchandEducationProgramGrant theclassoftransformations,ordeformationmodels,thatisca-
(N000140310511)andaPSC-CUNYGrant.Theassociateeditorcoordinating
pableofaligningtheimages.Thesearchstrategydecideshow
thereviewofthismanuscriptandapprovingitforpublicationwasDr.Luca
Lucchese. tochoosethenexttransformationfromthisspace,tobetestedin
S.ZokaiiswithBrainstormTechnologyLLC,NewYork,NY10011USA. the search for the optimal transformation. The similarity mea-
G.WolbergiswiththeDepartmentofComputerScience,CityCollegeof
suredeterminestherelativemeritforeachtest.Searchcontinues
NewYork,NewYork,NY10031USA(e-mail:[email protected]).
DigitalObjectIdentifier10.1109/TIP.2005.854501 accordingtothesearchstrategyuntilatransformationisfound
1057-7149/$20.00©2005IEEE
ZOKAIANDWOLBERG:IMAGEREGISTRATIONUSINGLOG-POLARMAPPINGS 1423
Fig.1. Airborneimagery.(a)Observedimages.(b)Referenceimage.(c)Registrationoverlays.
whosesimilaritymeasureissatisfactory.Numerousregistration The primary drawback of the optimization-based approach
techniqueshavebeenproposedbasedonchoosingaspecificfea- is that it may fail unless the two images are misaligned by a
ture,deformationmodel,optimizationmethod,and/orsimilarity moderatedifferenceinscale,rotation,andtranslation.Inorder
measure.See[2]forarecentsurveyofimageregistrationtech- to address this problem, we introduce a log-polar registration
niques. moduletobringtheimagesintoapproximatealignment,evenin
For image registration, we need to recover the geo- thepresenceofarbitraryrotationanglesandlargescalechanges.
metric transformation and/or intensity function. Let Itspurposeistofurnishagoodinitialestimatetotheperspec-
and be the reference and observed images, re- tiveregistrationmodulethatisbasedonnonlinearleastsquares
spectively. The relationship between these images is optimization.
,where isatwo-dimensional The scope of this work shall prove useful for various ap-
(2-D) geometrictransformation operatorthatrelatesthe plications, including the registration of aerial images, and the
coordinates in to the coordinates in and is the formation of image mosaics. Note that aerial imagery may be
intensityfunction. acquiredfromuncalibratedairbornecamerassubjectedtoyaw,
Theestimationoftheintensityfunction isusefulwhenwe pitch,androllatvariousaltitudes.Sincetheterrainappearsflat
wanttoregisterimagestakenfromdifferentsensorsorwhenil- from moderatelyhigh altitude,it is an ideal candidatefor reg-
luminationischangedbyautomaticgainexposureofacamera. istrationusingasingleperspectivetransformation.Anexample
Comparametricequationshavebeenintroducedtomodelthein- demonstratingtheregistrationoftwoaerialimagesinthepres-
tensityfunction [3].Althoughtheseequationsarenonlinear, enceoflargescale/rotationandmoderateperspectiveisshown
a piecewise linear method has been developed to estimate inFig.1.Theimage inFig.1(a) isautomatically registeredto
and simultaneously[4]. thatinFig.1(b),asdepictedbythehighlightedrectangle.
In Section II, we discuss related work on the standard Lev-
Mutualinformationisasimilaritymeasurethathasrecently
enberg–Marquardtalgorithm(LMA)andlog-polartechniques.
beenintroducedformultimodalmedicalimageregistration[5],
Section III describes a modified LMA for improving the per-
[6]. Correlation ratio is another similarity measure for multi-
formance of the standard LMA and Section IV presents our
modalimageregistrationandhasproventoperformbetterthan
proposed log-polar method. In Section V, we demonstrate the
mutualinformation[7].Multimodalimageregistrationhasbeen
successofthelog-polartransforminrecoveringlargedeforma-
studiedextensivelyinthemedicalimagingdomain.Inthiswork,
tions bycomparingregistrationaccuracywithand withoutthe
weassumethattheintensityfunctionislinear.Similaritymea-
log-polarregistrationmodule.Asignificantincreaseincorrect
sureslikethezero-meannormalizedsumofsquareddifferences
matches is attributed to our algorithm. A secondary compar-
(SSD)andcorrelationcoefficientareinvarianttothelinearin-
isonwasmadebyreplacingthelog-polarmodulewiththewell-
tensitychanges.
knownFourier–Mellintransform.Again,ourlog-polarmodule
Thispaperdescribesahierarchicalimageregistrationsystem.
provedsuperior to the Fourier–Mellin transform for achieving
We model the mapping function as a perspective transforma-
highperspectiveregistrationaccuracy.
tion.Thealgorithmestimatestheperspectiveparametersneces-
sarytoregisteranytwomisaligneddigitalimages.Theparame-
II. PREVIOUSWORK
tersareselectedtominimizetheSSDbetweenthetwoimages.
They are computed iteratively in a coarse-to-fine hierarchical Inthissection,wediscussrelatedworkontheLMAandthe
framework using a variation of the Levenberg–Marquadt non- log-polartechniques.InSectionII-A,wepresentabackground
linearleastsquaresoptimizationmethod.Thisapproachyields oftheLevenberg–Marquardtnonlinearleast-squaresoptimiza-
a robust solution that precisely registers images with subpixel tionalgorithmthatisusefulforachievingsubpixelregistration
accuracy. accuracy.Thelog-polartransformisdescribedinSectionII-B.