Computer vision using local binary patterns

224 Pages·2011·5.005 MB·English

by  Matti Pietikäinen, Abdenour Hadid, Guoying Zhao, Timo Ahonen (auth.)

Computer Vision Using Local Binary Patterns
Computational Imaging and Vision
ManagingEditor
MAXVIERGEVER
UtrechtUniversity,Utrecht,TheNetherlands
SeriesEditors
GUNILLABORGEFORS,CentreforImageAnalysis,SLU,Uppsala,Sweden
RACHIDDERICHE,INRIA,SophiaAntipolis,France
THOMASS.HUANG,UniversityofIllinois,Urbana,USA
KATSUSHIIKEUCHI,TokyoUniversity,Tokyo,Japan
TIANZIJIANG,InstituteofAutomation,CAS,Beijing,China
REINHARDKLETTE,UniversityofAuckland,Auckland,NewZealand
ALESLEONARDIS,ViCoS,UniversityofLjubljana,Ljubljana,Slovenia
HEINZ-OTTOPEITGEN,CeVis,Bremen,Germany
JOHNK.TSOTSOS,YorkUniversity,Toronto,Canada
This comprehensive book series embraces state-of-the-art expository works and advanced
researchmonographsonanyaspectofthisinterdisciplinaryfield.
Topicscoveredbytheseriesfallinthefollowingfourmaincategories:
• ImagingSystemsandImageProcessing
• ComputerVisionandImageUnderstanding
• Visualization
• ApplicationsofImagingTechnologies
Onlymonographsormulti-authoredbooksthathaveadistinctsubjectarea,thatiswhereeach
chapterhasbeeninvitedinordertofulfillthispurpose,willbeconsideredfortheseries.
Volume40
Forfurthervolumes:
www.springer.com/series/5754
Matti Pietikäinen (cid:2) Abdenour Hadid (cid:2)
Guoying Zhao (cid:2) Timo Ahonen
Computer Vision
Using Local Binary
Patterns
MattiPietikäinen GuoyingZhao
MachineVisionGroup MachineVisionGroup
DepartmentofComputerScienceand DepartmentofComputerScienceand
Engineering Engineering
UniversityofOulu UniversityofOulu
POBox4500 POBox4500
90014Oulu 90014Oulu
Finland Finland
[email protected].fi [email protected].fi
AbdenourHadid
TimoAhonen
MachineVisionGroup
NokiaResearchCenter
DepartmentofComputerScienceand
PaloAlto,CA
Engineering
USA
UniversityofOulu
[email protected]
POBox4500
90014Oulu
Finland
[email protected].fi
ISSN1381-6446
ISBN978-0-85729-747-1 e-ISBN978-0-85729-748-8
DOI10.1007/978-0-85729-748-8
SpringerLondonDordrechtHeidelbergNewYork
BritishLibraryCataloguinginPublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
LibraryofCongressControlNumber:2011932161
MathematicsSubjectClassification: 68T45,68H35,68U10,68T10,97R40
©Springer-VerlagLondonLimited2011
Apartfromanyfairdealingforthepurposesofresearchorprivatestudy,orcriticismorreview,asper-
mittedundertheCopyright,DesignsandPatentsAct1988,thispublicationmayonlybereproduced,
storedortransmitted,inanyformorbyanymeans,withthepriorpermissioninwritingofthepublish-
ers,orinthecaseofreprographicreproductioninaccordancewiththetermsoflicensesissuedbythe
CopyrightLicensingAgency.Enquiriesconcerningreproductionoutsidethosetermsshouldbesentto
thepublishers.
Theuseofregisterednames,trademarks,etc.,inthispublicationdoesnotimply,evenintheabsenceofa
specificstatement,thatsuchnamesareexemptfromtherelevantlawsandregulationsandthereforefree
forgeneraluse.
Thepublishermakesnorepresentation,expressorimplied,withregardtotheaccuracyoftheinformation
containedinthisbookandcannotacceptanylegalresponsibilityorliabilityforanyerrorsoromissions
thatmaybemade.
Coverdesign:deblik
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Humansreceivethegreatmajorityofinformationabouttheirenvironmentthrough
sight,andatleast50%ofthehumanbrainisdedicatedtovision.Visionisalsoakey
componentforbuildingartificialsystemsthatcanperceiveandunderstandtheiren-
vironment.Computervisionislikelytochangesocietyinmanyways;forexample,
itwillimprovethesafetyandsecurityofpeople,itwillhelpblindpeoplesee,andit
willmakehuman-computerinteractionmorenatural.Withcomputervisionitispos-
sibletoprovidemachineswithanabilitytounderstandtheirsurroundings,control
thequalityofproductsinindustrialprocesses,helpdiagnosediseasesinmedicine,
recognizehumansandtheiractions,andsearchforinformationfromdatabasesus-
ingimageorvideocontent.
Texture is an important characteristic of many types of images. It can be seen
inimagesrangingfrommultispectralremotelysenseddatatomicroscopicimages.
Atexturedareainanimagecanbecharacterizedbyanonuniformorvaryingspatial
distribution of intensity or color. The variation reflects some changes in the scene
being imaged. For example, an image of mountainous terrain appears textured. In
outdoorimages,trees,bushes,grass,sky,lakes,roads,buildingsetc.appearasdif-
ferenttypesoftexture.Thespecificstructureofthetexturedependsonthesurface
topography and albedo, the illumination of the surface, and the position and fre-
quency response of the viewer. An X-ray of diseased tissue may appear textured
duetothedifferentabsorptioncoefficientsofhealthyanddiseasedcellswithinthe
tissue.
Texturecanplayakeyroleinawidevarietyofapplicationsofcomputervision.
Thetraditionalareasofapplicationconsideredfortextureanalysisincludebiomedi-
calimageanalysis,industrialinspection,analysisofsatelliteoraerialimagery,doc-
umentimageanalysis,andtexturesynthesisforcomputergraphicsoranimation.
Texture analysis has been a topic of intensive research since the 1960s, and a
widevarietyoftechniquesfordiscriminatingtextureshavebeenproposed.Mostof
theproposedmethodshavenotbeen,however,capabletoperformwellenoughfor
real-world textures and are computationally too complex to meet the real-time re-
quirementsofmanyapplications.Inrecentyears,verydiscriminativeandcomputa-
tionallyefficientlocaltexturedescriptorshavebeendeveloped,suchaslocalbinary
v
vi Preface
patterns(LBP),whichhasledtoasignificantprogressinapplyingtexturemethods
tovariouscomputervisionproblems.Thefocusoftheresearchhasbroadenedfrom
2Dtexturesto3Dtexturesandspatiotemporal(dynamic)textures.
With this progress the emerging application areas of texture analysis will also
cover such modern fields as face analysis and biometrics, object recognition, mo-
tion analysis, recognition of actions, content-based retrieval from image or video
databases,andvisualspeechrecognition.Thisbookprovidesanexcellentoverview
how texture methods can be used for solving these kinds of problems, as well as
moretraditionalapplications.EspeciallytheuseofLBPinbiomedicalapplications
andbiometricrecognitionsystemshasgrownrapidlyinrecentyears.
Thelocalbinarypattern(LBP)isasimpleyetveryefficientoperatorwhichlabels
thepixelsofanimagebythresholdingtheneighborhoodofeachpixelandconsiders
theresultasabinarynumber.TheLBPmethodcanbeseenasaunifyingapproach
to the traditionally divergent statistical and structural models of texture analysis.
Perhaps the most important property of the LBP operator in real-world applica-
tionsisitsinvarianceagainstmonotonicgraylevelchangescaused,forexample,by
illumination variations. Another equally important is its computational simplicity,
whichmakesitpossibletoanalyzeimagesinchallengingreal-timesettings.LBPis
alsoveryflexible:itcanbeeasilyadaptedtodifferenttypesofproblemsandused
togetherwithotherimagedescriptors.
The book is divided into five parts. Part I provides an introduction to the book
contentsandanin-depthdescriptionofthelocalbinarypatternoperator.Acompre-
hensivesurveyofdifferentvariantsofLBPisalsopresented.PartIIdealswiththe
analysisofstillimagesusingLBPoperators.Applicationsintextureclassification,
segmentation,descriptionofinterestregions,content-basedimageretrievaland3D
recognitionoftexturedsurfacesareconsidered.ThetopicofPartIIIismotionanaly-
sis,withapplicationsindynamictexturerecognitionandsegmentation,background
modelinganddetectionofmovingobjects,andrecognitionofactions.PartIVdeals
withfaceanalysis.TheLBPoperatorsareusedforanalyzingstillimagesandimage
sequences. The specific application problem of visual speech recognition is pre-
sentedinmoredetail.Finally,PartVprovidesanintroductiontosomerelatedwork
bydescribingrepresentativeexamplesofusingLBPindifferentapplications,such
asbiometrics,visualinspectionandbiomedicalapplications,forexample.
Wewouldliketothankallco-authorsofourLBPpapersfortheirinvaluablecon-
tributionstothecontentsofthisbook.Firstofall,specialthankstoTimoOjalaand
David Harwood who started LBP investigations in our group in fall 1992 during
DavidHarwood’svisitfromtheUniversityofMarylandtoOulu.SincethenTimo
OjalamademanycentralcontributionstoLBPuntil2002whenourveryfrequently
cited paper was published in IEEE Transactions on Pattern Analysis and Machine
Intelligence. Topi Mäenpää played also a very significant role in many develop-
mentsofLBP.Otherkeycontributors,inalphabeticorder,includeJieChen,Xiaoyi
Feng,YimoGuo,ChuHe,MarkoHeikkilä,ViliKellokumpu,StanZ.Li,JiriMatas,
TomiNurmela,CordeliaSchmid,MattiTaini,ValtteriTakala,andMarkusTurtinen.
We also thank the anonymous reviewers, whose constructive comments helped us
improvethebook.
Preface vii
MatlabandCcodesofthebasicLBPoperatorsandsomevideodemonstrations
canbefoundfromanaccompanyingwebsiteatwww.cse.oulu.fi/MVG/LBP_Book.
ForabibliographyofLBP-relatedresearchandlinkstomanypapers,seewww.cse.
oulu.fi/MVG/LBP_Bibliography.
Oulu,Finland MattiPietikäinen
Oulu,Finland AbdenourHadid
Oulu,Finland GuoyingZhao
PaloAlto,CA TimoAhonen

Contents
PartI LocalBinaryPatternOperators
1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 TheRoleofTextureinComputerVision . . . . . . . . . . . . . . 3
1.2 MotivationandBackgroundforLBP . . . . . . . . . . . . . . . . 4
1.3 ABriefHistoryofLBP . . . . . . . . . . . . . . . . . . . . . . . 6
1.4 OverviewoftheBook . . . . . . . . . . . . . . . . . . . . . . . . 7
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
2 LocalBinaryPatternsforStillImages . . . . . . . . . . . . . . . . . 13
2.1 BasicLBP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2.2 DerivationoftheGenericLBPOperator . . . . . . . . . . . . . . 13
2.3 MappingsoftheLBPLabels:UniformPatterns. . . . . . . . . . . 16
2.4 RotationalInvariance . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4.1 RotationInvariantLBP . . . . . . . . . . . . . . . . . . . 19
2.4.2 RotationInvarianceUsingHistogramTransformations . . . 20
2.5 ComplementaryContrastMeasure . . . . . . . . . . . . . . . . . 21
2.6 Non-parametricClassificationPrinciple . . . . . . . . . . . . . . . 23
2.7 MultiscaleLBP . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.8 Center-SymmetricLBP . . . . . . . . . . . . . . . . . . . . . . . 25
2.9 OtherLBPVariants . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.9.1 Preprocessing . . . . . . . . . . . . . . . . . . . . . . . . 26
2.9.2 NeighborhoodTopology . . . . . . . . . . . . . . . . . . . 31
2.9.3 ThresholdingandEncoding . . . . . . . . . . . . . . . . . 32
2.9.4 MultiscaleAnalysis . . . . . . . . . . . . . . . . . . . . . 35
2.9.5 HandlingRotation . . . . . . . . . . . . . . . . . . . . . . 37
2.9.6 HandlingColor . . . . . . . . . . . . . . . . . . . . . . . 38
2.9.7 FeatureSelectionandLearning . . . . . . . . . . . . . . . 39
2.9.8 ComplementaryDescriptors . . . . . . . . . . . . . . . . . 42
2.9.9 OtherMethodsInspiredbyLBP . . . . . . . . . . . . . . . 42
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
ix