Camera-Trap Images Segmentation using Multi-Layer Robust Principal Component Analysis

0.52 MB·

by  Jhony-Heriberto Giraldo-Zuluaga

CAMERA-TRAPIMAGESSEGMENTATIONUSINGMULTI-LAYERROBUSTPRINCIPAL
COMPONENTANALYSIS
Jhony-HeribertoGiraldo-Zuluaga(cid:63),AlexanderGomez(cid:63),AugustoSalazar(cid:63),andAnge´licaDiaz-Pulido†
(cid:63) GrupodeInvestigacio´nSISTEMIC,FacultaddeIngenier´ıa,UniversidaddeAntioquiaUdeA,
Calle70No. 52-21,Medell´ın,Colombia
† InstitutodeInvestigacio´ndeRecursosBiolo´gicosAlexandervonHumboldt,
Calle28ANo. 15-09,Bogota´ D.C,Colombia
7
1
0
2
ABSTRACT measuring other biological rates [1]. Camera traps generate

n Cameratrappingisatechniquetostudywildlifeusingau- largevolumesofinformation,forexampleacameratrapping
a
J tomatic triggered cameras. However, camera trapping col- studycangenerateuntil200000images,where1%ofthein-
7 lectsalotoffalsepositives(imageswithoutanimals),which formationisvaluable[2]. Asaconsequence,biologistshave
2 mustbesegmentedbeforetheclassificationstep. Thispaper toanalyzethousandsofphotographsinamanualway. Nowa-
presents a Multi-Layer Robust Principal Component Analy- days, software solutions cannot handle the increment of the

] sis(RPCA)forcamera-trapimagessegmentation. OurMulti- number of images in camera trapping [3]. Accordingly, it is
V
Layer RPCA uses histogram equalization and Gaussian fil- importanttodevelopalgorithmstoassistthepost-processing
C
ter as pre-processing, texture and color descriptors as fea- ofcamera-trapimages.
.
s tures, and morphological filters with active contour as post- Backgroundsubtractiontechniquescouldhelptosegment
c
processing. The experiments focus on computing the sparse animalsfromcamera-trapimages. Thereisasignificantbody
[
andlow-rankmatriceswithdifferentamountsofcamera-trap researchofbackgroundsubtractionfocusedinvideosurveil-
1 images. WetestedtheMulti-LayerRPCAinourcamera-trap lance [4]. Nevertheless, there are not enough methods that
v
database. Toourbestknowledge,thispaperisthefirstwork can handle the complexity of natural dynamic scenes [5].
0
8 proposing Multi-Layer RPCA and using it for camera-trap Camera-trapimagessegmentationisimportantforanimalde-
1 imagessegmentation. tectionandclassification. Camera-trapsimagesusuallyhave
8 IndexTerms— Camera-trapimages,Multi-LayerRobust rippingwater,movingshadows,swayingtreesandleaves,sun
0
Principal Component Analysis, background subtraction, im- spots,scenechanges,amongothers. Consequently,themod-
.
1 agesegmentation. elsusedtosegmentthosetypesofimagesshouldhaverobust
0 feature extractors. There are some segmentation methods in
7
the literature applied to camera-trap images segmentation.
1 1. INTRODUCTION
Reddy and Aravind proposed a method to segment tigers on
:
v camera-trap images, using texture and color features with
i Studying and monitoring of mammals and birds species can
X activecontours[6]. Theydonotmakeanobjectiveevaluation
beperformedusingnon-invasivesamplingtechniques. These
of their method. Ren et al. developed a method to segment
r techniques allow us to observe animal species for conserva-
a imagesfromdynamicscenes,includingcamera-trapimages;
tionpurposes,e.g. toestimatepopulationsizesofendangered
the method uses Bag of Words (BoW), Histogram of Ori-
species. Camera trapping is a method to digitally capture
ented Gradients (HOG), and graph cut energy minimization
wildlife images. This method facilitates the register of ter-
[7]. They do not show the results on camera-trap images.
restrialvertebralspecies,e.g. crypticspecies. Consequently,
Zhang et al. developed a method to segment animals from
camera traps can generate large volumes of information in
videosequences,usingcamera-trapimages;themethoduses
shortperiodsoftime. Thus,thecontributionsincameratrap-
BoW,HOG,andgraphcutenergyminimization[8].Theyob-
pingareimportantforbetterspeciesconservationdecisions.
tained0.8695ofaveragef-measureontheirowncamera-trap
Cameratrapsaredevicestocaptureanimalimagesinthe
dataset.
wild. Thesedevicesconsistofadigitalcameraandamotion
detector. They are triggered when the motion sensor detects Robust Principal Component Analysis (RPCA) is a
movementanddependentonthetemperatureofthesourcein methodderivedfromPrincipalComponentAnalysis. RPCA
relationtotheenvironmenttemperature. Biologistcanmoni- assumesthatadatamatrixcanbedecomposedinalow-rank
torwildlifewithcameratrapsfordetectingrarespecies,delin- and a sparse matrix. RPCA has newly seen significant ac-
eatingspeciesdistributions,monitoringanimalbehavior,and tivity in many areas of computer sciences, particularly in
background subtraction. As a result, there are some algo-
rithmstosolvetheRPCAproblem[9,10,11,12,13]. Inthis
work, we proposed a Multi-Layer RPCA approach in order
to segment animals from camera-trap images. Our method
combines color and texture descriptors as feature extractor,
and solve the RPCA problem with some state-of-the-art al- (a)OriginalImage (b)Groundtruth
gorithms. To our knowledge, this paper is the first work in
proposing a Multi-Layer RPCA approach and using it for Fig. 1: Ground truth example for the evaluation process, (a)
camera-trapimagessegmentation. originalimage,(b)groundtruthormanualsegmentedimage.
Thepaperisorganizedasfollows. Section2showsmate-
rialandmethods.Section3describestheexperimentalframe-
work. Section4presentstheexperimentalresultsandthedis- way on uniform regions such as water, the sky, and others.
cussion. Finally,Section5showstheconclusions. Colordescriptorscouldovercomethetexturedescriptorlimi-
tation[16]. TheMulti-Layerapproachproposedinthispaper
wastestedincameratrapsforwildlifeimagesegmentation.
2. MATERIALSANDMETHODS
Thissectionshowsabriefexplanationofthealgorithmsand M(x,y)=βf (x,y)+(1−β)f (x,y) (3)
t c
metricsusedinthispaper.
2.3. EvaluationMetrics
2.1. RobustPrincipalComponentAnalysis
An image can be decomposed in a low-rank and sparse ma- Thef-measuremetricwaschosentoevaluatetheperformance
trix. Equation 1 shows the RPCA problem, where M is the of the Multi-Layer RPCA. Equation 4 shows the f-measure,
data matrix, L is the low-rank matrix, and S is the sparse where precision and recall are extracted from the confusion
0 0
matrix.Thelow-rankmatrixisthebackgroundandthesparse matrix. Precisionistheproportionofpredictedpositivesthat
matrixistheforegroundinbackgroundsubtraction. are correctly real positives. In the same way, recall denotes
the proportion of the real positives that are correctly pre-
M =L +S (1) dicted[17]. Theconfusionmatrixiscomputedcomparingthe
0 0
groundtruth(GT)withtheautomaticsegmentedimages.
The RPCA problem can be solved with the convex pro-
gramPrincipalComponentPursuit(PCP).Thisprogramcom-
precision∗recall
putesLandS, takingtheobjectivefunctionintheEquation f-measure=2 (4)
precision+recall
2,where||L|| denotesthenuclearnormofthelow-rankma-
∗
trix,||S|| denotesthel -normofthesparsematrix,andλis
1 1
a regularizing parameter. There are some algorithms to per- 3. EXPERIMENTALFRAMEWORK
formPCPsuchasAcceleratedProximalGradient(APG),and
AugmentedLagrangemultiplier(ALM)[14].
This section introduces the database used in this paper, the
experimentsexecuted, andtheimplementationdetailsofour
minimize ||L|| +λ||S||
∗ 1 (2) Multi-LayerRPCA.
subjectto L+S =M
2.2. Multi-LayerRobustPrincipalComponentAnalysis 3.1. Database
Equation 3 shows the data matrix M in our Multi-Layer The Alexander von Humboldt Institute realizes samplings
RPCA method, where β ∈ [0,1] is a weight value indicat- with camera traps in different regions of the Colombian for-
ing the contribution of the texture function to the overall est. Weselect25camerasfrom8regions,whereeachcamera
datamatrix. Functionf (x,y)denotesthetexturedescriptor has a relative unalterable environment. Each camera was
t
extracted from each image, using the classic Local Binary placedinitssitebetween1to3months. Weextract30days
Pattern (LBP) [15]. LBP describes the texture of an image and30nightsofimagesfromthosecameras,indaytimecolor
using theneighborhood ofeach pixel. Function f (x,y) de- and nighttime infrared formats respectively. The database
c
notes the color transformation for each image, converting to consistsof1065GTimagesfromthe30daysand30nights.
grayscaleinthiscase. OurMulti-LayerRPCAcomputesthe The images have a spatial resolution of 3264x2448 pixels.
LandS matricesfromtheM matrixintheEquation3. Tex- The length of each day or night data set varies from 9 to 72
ture descriptors can work robustly into rich texture regions images, depending on the animal activity that day or night.
withlightvariation. However,theydonotworkinaefficient Figure1showsanexampleoftheGTimages.
Sparse
LBP matrix Smpaatrrsixe ThHreasrhdo ld Median filter
features
imRaagwe RPCA MuRltPi-LCaAyer coAncttoivuer s Opening
Histogram Color
equalization features Low-rank Lomwa-trraixnk Closing Opening
matrix
Background Foreground
Fig.2: Blockdiagramofthepre-processingmethodsusedin
theExperiment1,3,and4.
Fig.4: Blockdiagramofthepostprocessing.
Sparse
LBP matrix nally,Thef-measurewascomputedcomparingeachGTwith
features
eachforeground.Theaveragef-measurewascomputedasthe
Raw Gaussian
RPCA
image filter meanofallf-measures. Theresultsaredisplayedasaplotof
Color
features Low-rank theaveragef-measurevsβ.
matrix
Fig.3: Blockdiagramofthepre-processingmethodsusedin 3.3. ImplementationDetails
theExperiment2and3.
TheRPCAalgorithmswerecomputedusingtheSobraletal.
library[19]. Therestofthesourcecodewasdevelopedusing
theimageprocessingtoolboxofMatlab.
3.2. Experiments
The experiments computed the background models with 4. RESULTS
different conditions and amount of images, observing the
robustness of the Multi-Layer RPCA and the influence of Thissectionshowstheresultsanddiscussionsoftheexperi-
pre-processingintheresults. Allexperimentsperformedour mentsintroducedintheSection3. Theseresultsusethemet-
method with β = [0,0.05,0.1,0.15,...,1]. Accordingly, ricsexplainedintheSection2.3.
Experiment 1 uses histogram equalization as pre-processing Figures5aand5bshowtheaveragef-measurevsβofthe
in the color transformed image, Figure 2 shows the pre- Experiments1and2forallRPCAalgorithmschosen. Table
processing for each raw image. The background model is 1showsthesummaryofthebestresultsforeachexperiment.
computed with entire days e.g. we take all images of day 1 APG-PARTIALwasthebestalgorithmintheExperiments1
and solve the RPCA problem in the Experiment 1. Experi- and2. Daytimeimageshaverichtextureregions. Incontrast,
ment 2 computes the background model with entire nights; nighttimeimageshaveuniformcolor.Texturerepresentations
Figure3showsthepre-processingforeachrawimage.Exper- are more important on daytime images in the Experiment 1
iment3takesentiredaysandnightse.g.wetakeallimagesof due to β = 0.6. On the contrary, color descriptors are more
day1andnight1tosolvetheRPCAproblem. Experiment3 important on nighttime images in the Experiment 2 due to
usestwopre-processes,daytimeimagesusesthepre-process β = 0.3. Those results show the importance of combining
inFigure2andnighttimeimagesusesthepre-processinFig- thecolorandtexturedescriptors. Figure5showstheperfor-
ure3. Experiment4takesentiredaysandnightssuchasthe mancesoftheRPCAnormalalgorithmswhenβ = 0. Thus,
Experiment3,butitonlyusesthepre-processinginFigure2 ourMulti-LayerRPCAoutperformstheRPCAnormalmeth-
forallimages. ods.
We tested 9 algorithms to solve the RPCA problem in Figures5cand5dshowtheaveragef-measurevsβofthe
this paper. Active Subspace RPCA (AS-RPCA) [9]; Ex- Experiments 3 and 4. Table 1 shows that dividing the pre-
act ALM (EALM), Inexact ALM (IALM), Partial APG processingperdaytimeornighttimeintheExperiment3does
(APG-PARTIAL) and APG [10]; Lagrangian Alternating not make a big difference in the results, but it increases the
Direction Method (LSADM) [11]; Non-Smooth Augmented fine-tuning parameters. Table 1 shows that NSA2 was the
Lagrangian v1 (NSA1) and Non-Smooth Augmented La- bestalgorithmintheExperiments3and4,contrarytotheEx-
grangian v2 (NSA2) [12]; Probabilistic Robust Matrix Fac- periments1and2whereAPG-PARTIALwasthebest.NSA2
torization(PRMF)[13]. algorithm is a better choice than other RPCA algorithms, if
The foreground was obtained applying a post-process to we cannot differentiate between daytime and nighttime im-
the sparse matrix. The post-processing was the same for all ages, or if it is difficult to do so. On the other hand, APG-
experiments. This stage includes a hard threshold, morpho- PARTIALisbetter,ifwehaveinformationabouttheinfrared
logicalfilters,andanactivecontourswithanegativecontrac- activation.
tion bias [18]. Figure 4 shows the post-processing used. Fi- Figure 6 shows two visual results of the Multi-Layer
0.75
0.7
ure 0.7 AESA-LRMPCA ure0.65 AESA-LRMPCA
as IALM as IALM
f-me0.65 AALSPPAGGD-PMARTIAL f-me 0.6 AALSPPAGGD-PMARTIAL
NSA1 0.55 NSA1
NSA2 NSA2
0.6 PRMF PRMF
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
beta beta
(a)Averagef-measurevsβfortheexperiment1 (b)Averagef-measurevsβfortheexperiment2
0.7 0.7
e0.65 AS-RPCA e0.65 AS-RPCA
ur EALM ur EALM
f-meas 0.6 IAALASPPLAGGMD-PMARTIAL f-meas 0.6 IAALASPPLAGGMD-PMARTIAL
0.55 NSA1 0.55 NSA1
NSA2 NSA2
PRMF PRMF
0.5
0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1
beta beta
(c)Averagef-measurevsβfortheexperiment3 (d)Averagef-measurevsβfortheexperiment4
Fig. 5: Results of the proposed experiments, (a) average f-measure vs β per days, (b) average f-measure vs β per nights, (c)
averagef-measurevsβ perdaysandnightswithtwodifferentpre-processes, (d)averagef-measurevsβ perdaysandnights
withonepre-process.
Table 1: β values and algorithms for the best performances
ofeachexperiment.
Experiment Algorithm β Avgf-measure
(a)Ground (b)Sparse (c)Foreground Experiment1 APG-PARTIAL 0.6 0.7539
Experiment2 APG-PARTIAL 0.3 0.7393
Experiment3 NSA2 0.45 0.7259
Experiment4 NSA2 0.55 0.7261
(d)Ground (e)Sparse (f)Foreground
Fig. 6: Visual results of the Multi-Layer RPCA, (a) origi- posed algorithm is composed of pre-processing, RPCA al-
naldaytimeimage,(b)sparsematrixafterthehardthreshold gorithm, and post-processing. The pre-processing uses his-
withAPG-PARTIALandβ = 0.6,(c)foreground,(d)origi- togram equalization, Gaussian filtering, or a combination of
nalnighttimeimage,(e)sparsematrixafterthehardthreshold both.TheRPCAalgorithmcomputesthesparseandlow-rank
withAPG-PARTIALandβ =0.3,(f)foreground. matrices for background subtraction. The post-processing
computes morphological filters and an active contours with
a negative contraction bias. We proved the Multi-Layer
RPCA. Figure 6a shows a daytime image without any pre- RPCA algorithm in a camera-trap images database from the
processing. Figure 6d shows an original nighttime image. Colombianforest. Thedatabasewasmanuallysegmentedto
Figures 6b and 6e show the sparse matrix after the hard extract the f-measure of each automatic segmented image.
threshold. Figures 6c and 6f show the foreground image. Wereach0.7539and0.7393ofaveragef-measureindaytime
These color results are made with the GT images. Yellow- and nighttime images respectively. The average f-measure
colored regions mean pixels that are on the GT and the au- was computed with all GT images. To our best knowledge,
tomaticsegmentedimages. Redandgreenregionsarevisual this paper is the first work in proposing Multi-Layer RPCA
representationsoftheunderandoversegmentation. andusingitforcamera-trapimagessegmentation.
5. CONCLUSIONS Acknowledgment. ThisworkwassupportedbytheColom-
bianNationalFundforScience,TechnologyandInnovation,
We proposed a Multi-Layer RPCA for camera-trap image Francisco Jose´ de Caldas - COLCIENCIAS (Colombia).
segmentation, using texture and color descriptors. The pro- ProjectNo. 111571451061.
6. REFERENCES [13] N.Wang,T.Yao,J.Wang,andD.-Y.Yeung,“Aproba-
bilistic approach to robust matrix factorization,” in Eu-
[1] A.F.O’Connell,J.D.Nichols,andK.U.Karanth,Cam- ropean Conference on Computer Vision, pp. 126–139,
era traps in animal ecology: methods and analyses. Springer,2012.
SpringerScience&BusinessMedia,2010.
[14] E. J. Cande`s, X. Li, Y. Ma, and J. Wright, “Robust
[2] A. Diaz-Pulido and E. Payan, “Densidad de ocelotes principal component analysis?,” Journal of the ACM
(leopardus pardalis) en los llanos colombianos,” Mas- (JACM),vol.58,no.3,p.11,2011.
tozoolog´ıaneotropical,vol.18,no.1,pp.63–71,2011.
[15] T. Ojala, M. Pietikainen, and T. Maenpaa, “Multireso-
[3] E.H.Fegraus,K.Lin,J.A.Ahumada,C.Baru,S.Chan-
lution gray-scale and rotation invariant texture classifi-
dra, and C. Youn, “Data acquisition and management
cationwithlocalbinarypatterns,”IEEETransactionson
software for camera trap data: A case study from the
patternanalysisandmachineintelligence,vol.24,no.7,
team network,” Ecological Informatics, vol. 6, no. 6,
pp.971–987,2002.
pp.345–353,2011.
[16] J.YaoandJ.-M.Odobez,“Multi-layerbackgroundsub-
[4] S. Brutzer, B. Ho¨ferlin, and G. Heidemann, “Evalu-
tractionbasedoncolorandtexture,”in2007IEEECon-
ation of background subtraction techniques for video
ference on Computer Vision and Pattern Recognition,
surveillance,”inComputerVisionandPatternRecogni-
pp.1–8,IEEE,2007.
tion(CVPR),2011IEEEConferenceon,pp.1937–1944,
IEEE,2011. [17] D. M. Powers, “Evaluation: from precision, recall and
f-measuretoroc, informedness, markednessandcorre-
[5] V. Mahadevan and N. Vasconcelos, “Spatiotemporal
lation,”2011.
saliencyindynamicscenes,”IEEETransactionsonPat-
tern Analysis and Machine Intelligence, vol. 32, no. 1, [18] V. Caselles, R. Kimmel, and G. Sapiro, “Geodesic ac-
pp.171–177,2010. tivecontours,”Internationaljournalofcomputervision,
vol.22,no.1,pp.61–79,1997.
[6] K. P. K. Reddy and R. Aravind, “Segmentation of
camera-traptigerimagesbasedontextureandcolorfea- [19] A.Sobral,T.Bouwmans,andE.-h.Zahzah,“Lrslibrary:
tures,” inCommunications(NCC),2012NationalCon- Low-rank and sparse tools for background modeling
ferenceon,pp.1–5,IEEE,2012. and subtraction in videos,” in Robust Low-Rank and
Sparse Matrix Decomposition: Applications in Image
[7] X. Ren, T. X. Han, and Z. He, “Ensemble video ob-
and Video Processing, CRC Press, Taylor and Francis
ject cut in highly dynamic scenes,” in Proceedings of
Group,2015.
the IEEE Conference on Computer Vision and Pattern
Recognition,pp.1947–1954,2013.
[8] Z. Zhang, T. X. Han, and Z. He, “Coupled ensemble
graphcutsandobjectverificationforanimalsegmenta-
tion from highly cluttered videos,” in Image Process-
ing (ICIP), 2015 IEEE International Conference on,
pp.2830–2834,IEEE,2015.
[9] G. Liu and S. Yan, “Active subspace: Toward scalable
low-ranklearning,”Neuralcomputation,vol.24,no.12,
pp.3371–3394,2012.
[10] Z.Lin,M.Chen,andY.Ma,“Theaugmentedlagrange
multiplier method for exact recovery of corrupted low-
rankmatrices,”arXivpreprintarXiv:1009.5055,2010.
[11] D. Goldfarb, S. Ma, and K. Scheinberg, “Fast alter-
nating linearization methods for minimizing the sum
oftwoconvexfunctions,” MathematicalProgramming,
vol.141,no.1-2,pp.349–382,2013.
[12] N. S. Aybat, D. Goldfarb, and G. Iyengar, “Fast first-
order methods for stable principal component pursuit,”
arXivpreprintarXiv:1105.2126,2011.