Map-guided Hyperspectral Image Superpixel Segmentation Using Proportion Maps

6.2 MB·

by  Hao Sun

MAP-GUIDEDHYPERSPECTRALIMAGESUPERPIXELSEGMENTATIONUSING
PROPORTIONMAPS
HaoSun* andAlinaZare†
DepartmentofElectricalandComputerEngineering,UniversityofMissouri*
DepartmentofElectricalandComputerEngineering,UniversityofFlorida†
7 ABSTRACT 2. METHOD
1
0 A map-guided superpixel segmentation method for hyper- Theproposedmap-guidedhyperspectralunmixing-basedsu-
2
spectralimageryisdevelopedandintroduced. Theproposed perpixel segmentation algorithm has three stages. Stage one
n approachdevelopsahyperspectral-appropriateversionofthe consistsofobtaininganinitialsuperpixelsegmentationresult
a
J SLIC superpixel segmentation algorithm, leverages map in- using a modified version of SLIC [5]. Stage two consists
formation to guide segmentation, and incorporates the semi- of applying the semi-supervised Partial Membership Latent
6
supervised Partial Membership Latent Dirichlet Allocation Dirichlet Allocation (sPM-LDA) to obtain unmixing results
] (sPM-LDA) to obtain a final superpixel segmentation. The [6]. Finally, stage three obtains a final superpixel segmenta-
V proposed method is applied to two real hyperspectral data tionbyclusteringtheproportionvectorsfromstagetwo.
C sets and quantitative cluster validity metrics indicate that
. the proposed approach outperforms existing hyperspectral
s 2.1. Stage1: Map-GuidedHyperspectralSLIC
c superpixelsegmentationmethods.
[
To perform an initial superpixel segmentation, we extend
Index Terms— Hyperspectral, superpixel, SLIC, Open-

1 StreetMap, partial membership, latent dirichlet allocation, Simple Linear Iterative Clustering (SLIC) [5] to hyperspec-
v tral imagery. Previously in the literature, dimensionality
clustervalidity,segmentation
5 reduction was applied to HSI prior to allow for application
4
of SLIC [7]. In this paper, we develop a new approach for
7
1. INTRODUCTION extending SLIC to HSI. In our method, no dimensionality
1
0 reduction is applied. Instead, the full spectral information
. Many effective superpixel image segmentation algorithms is used (as opposed to the Lab features used in SLIC origi-
1
0 have been developed in the literature for gray-scale or RGB nally)incombinationwithspatialinformation. Theproposed
7 imagery. However,veryfewoftheseapproachesaredirectly hyperspectral SLIC (HSLIC) is iterative: (1) initial cluster
1 applicable to hyperspectral imagery. In [1], the ultrametric centers are obtained with a regular spatial sampling of the
v: contourmap(UCM)algorithmwasextendedtohyperspectral image where the cluster centers are vectors in which spec-
i imagery (HSI) through the use of principal component anal- tralinformationisconcatenatedwithspatialcoordinates; (2)
X
ysis to reduce the image dimensionality to three dimension then,eachpixelisassignedtoanearestclustercenterusinga
r suchthatUCMcanbedirectlyapplied. Thenormalizedcuts distance measure that includes a spectral and a spatial term;
a
algorithm has also been extended for hyperspectral imagery and (3) cluster centers are then updated to be equal to the
[2]. In [3], a hyperspectral superpixel algorithm was devel- average of all pixels assigned to the cluster. This process is
opedusingagraph-basedapproachthatreliedonthesumof iterateduntiltheconvergence.
squareddifferencesbetweenneighboringpixels.
The superpixel segmentation approach proposed here
differs from existing hyperspectral superpixel segmentation
methodsprimarilyintwoways:(1)theproposedmethoduses
proportionmapsobtainedthroughunmixingasadimensionality-
reducedversionoftheimagescene;and(2)themethodrelies
on map data (specifically, data from crowdsourced Open-
StreetMap[4])toguidetheunmixingandsegmentation.
TheauthorswishtothanktheNationalGeospatial-IntelligenceAgency
Fig. 1. OpenStreetMap polygons overlaid on the Pavia Uni-
forsupportofthisresearchundertheprojectentitled“NIP:FunctionsofMul-
versityHSIimage.
tipleInstancesforHyperspectralAnalysis.”
Toleveragemapinformation,ourmap-guidedSLICalgo- work,weusethepolygonlabelsobtainedfromOSMtopro-
rithm merges superpixels that intersect with a common map vide partial labels to the sPM-LDA approach. For example,
element. Forexample,fromOpenStreetMap(OSM),mapel- superpixelsobtainedthroughmerginginstage1thatoverlap
ementssuchasroadsandbuildingprofilescanbeobtained,as withbuildingpolygonsaregivenapartiallabelof“building.”
showninFig. 1. Thesemapelementscanberoughlyaligned
to the hyperspectral imagery using a affine transformation. 2.3. Stage3: FinalSuperpixelSegmentation
In our implementation, this transformation is obtained us-
K-means clustering is applied to the proportion vectors that
ing manual selection of corresponding points. However, ap-
are obtained in Stage 1. Then, connected components anal-
proachessuchasmapconflationcanbeusedaswell[8].After
ysisisappliedtorelabeleachspatialclusterasanindividual
alignment,thesuperpixelsobtainedusinghyperspectralSLIC
superpixel. Finally, a “clean-up” post-processing is done to
methoddescribedabovewhichoverlapwithsharedmappoly-
mergesmalldisjointsegmentsintothelargestneighboringsu-
gons are merged into one superpixel. The full map-guided
perpixel. Specifically,ifthenumberofpixelsinasuperpixel
hyperspectralSLICissummarizedinAlgorithm1.
is less than a prescribed threshold, those pixels are merged
intothelargestneighboringsuperpixel.
Algorithm 1 Map-Guided Hyperspectral SLIC Superpixel
Segmentation
Input: HSIData,K(numberofsuperpixels),m(scalingfac- 3. EXPERIMENTALRESULTS
tor)
1: Initialize cluster centers {ck}Kk=1 by sampling pixels at Otwuorrperaolphoysepderssuppecetrrpailxedlatsaegsmetes.ntaTthioenfimrsetthdoatdaissetapwpalisedcotlo-
regulargridsteps.
2: Perturb{ck}Kk=1 inann×nneighborhoodtothelowest l(eRcOteSdISby)atthUeRnievfleerscittiyveofOPpatvicisa,SIytasltyeminIJmualygi2n0g0S2p.eTchtreoimmeatgeer
gradientposition.
contains340×610pixelsandconsistsof103bands[9].
3: Repeat
DuringthehyperspectralSLICapplication,K =500and
4: foreachck do
m = 20. During unmixing using sPM-LDA, the number of
5: Calculatethespectraldistanceofpixelxi andck over
allbandsλ: d =(cid:80)B (cid:107)x (λ)−c (λ)(cid:107)2 endmembers was set to 6, λ = 1, α = 0.3, (cid:15) = 5% and
spectral λ=1 i k 2 T =200andtheblueroofbuildingandredroofbuildingwere
6: Calculatethespatialdistanceofpixelxiandck:
(cid:112) partiallylabeledusingthesemi-supervisedapproachoutlined
d = (a −a )2+(b −b )2 where a,b
spatial xi ck xi ck in [10]. During the final stage, K = 6 was used during K-
arepixelcoordinates.
meansclustering. Fig. 2showstheresultsreturnedfromhy-
7: Assigneachpixeltoackina2S×2Ssquareneighbor-
perspectralSLIC,theextracteddatafromOSMandthefinal
hood based on the minimum spectral and spatial dis-
tance: d =d + md segmentationresult. UnmixingresultsareshowninFig. 3.
xi,ck spectral S spatial
8: endfor
9: Updateclustercentersasthemeanofallpixelsassigned
tothecluster.
10: Untilstoppingcriterionisreached.
11: Alignmapdatatoimagery
12: forEachMapPolygondo
13: Mergeallsuperpixelsthatoverlapwiththispolygon
14: endfor
2.2. Stage2: UnmixingwithSemi-supervisedPM-LDA (a) (b) (c)
Afterobtainingthemap-guidedSLICsuperpixels,thehyper-
Fig. 2. Superpixels on Pavia University: (a)Superpixels
spectral is unmixed to obtain proportion maps. These pro-
from SLIC; (b)Buildings extracted from OpenStreetMap;
portionmapsareusedasadimensionalityreducedversionof
(c)Superpixelsbymap-guidedhyperspectralSLIC
theimagerywhicharethenusedtoobtainthefinalsuperpixel
segmentation. Inthiswork, weusethesemi-supervisedpar-
tialmembershiplatentDirichletallocation(sPM-LDA)toun- The proposed superpixel segmentation results as shown
mixtheinputimageryandobtainproportionmaps.sPM-LDA in Fig. 4 are compared with four other methods: hyper-
is described in [6]. sPM-LDA uses an input superpixel seg- spectral SLIC (HSLIC) (Sec.2.1), map-guided hyperspec-
mentation to guide unmixing. Also, sPM-LDA can leverage tral SLIC (HSLIC+OSM) (Sec.2.1), hyperspectral normal-
partiallabelinformationtoimproveunmixingresults. Inthis ized cuts (HNC) [2], and hyperspectal ultra contour map
(a) Painted metal (b) Redroof (c) Baresoil (a) HSLIC (b) HSLIC+OSM (c) HNC
sheets
(d) HUCM (e) PM-LDA (f) sPM-LDA
(d) Asphalt (e) Shadow (f) Vegetation
Fig.4. SuperpixelSegmentationresultsonPaviaUniversity
Fig. 3. Estimated proportion maps using semi-supervised
PM-LDAonPaviaUniversity
for these three indices. As shown in Table 1, the proposed
methodoutperformsthecomparisonapproaches.
(HUCM)[1] and a PM-LDA based superpixel segmentation
Dunn Davies-Bouldin Silhouette
(which is the same as the proposed method except without
HSLIC 0.033±0.01 16.569±0.61 −0.836±0.03
finalpost-processingasdescribedinSec.2.3).
HSLIC+OSM 0.040±0.00 16.119±0.79 −0.795±0.03
ExaminingFig. 4,itcanbeseenthathyperspectralSLIC HNC 0.049±0.02 11.445±0.66 −0.724±0.05
andhyperspectralSLIC+OSMbothreturnoversegmentedre- HUCM 0.038±0.01 19.296±0.94 −0.750±0.03
sults. ResultsfromHNCandHUCMdonotoversegmentbut PM-LDA 0.061±0.01 10.995±0.83 −0.7056±0.05
Proposedmethod 0.095±0.01 8.712±0.73 -0.561±0.05
oftencrossdistinctboundariesintheimageryresultinginin-
correctsuperpixelsegmentation. ThePM-LDAbasedsuper- Table 1. Dunn, Davies-Bouldin, Silhouette index results ±
pixel segmentation (without postprocessing) includes many standarddeviationonPaviaUniversity.
verysmallsuperpixels(sosmallthattheyappearlikenoise).
In comparison, our proposed approach can segment the im- The proposed approach was also applied to the MUUFL
agery into semantically meaningful superpixels with regions Gulfporthyperspectraldata[14].Thisimagewascollectedby
associatedwithmappolygonsremainingintact. theCASI-1500hyperspectralimageroverthecampusofthe
Toquantitativelyevaluatethesuperpixelsegmentationre- University of Southern Mississippi-Gulfport in Long Beach,
sults, the Dunn [11], Davies-Bouldin (DB) [12], Silhouette MississippiinNovember2010. Theimageisconsistingof72
indices[13]areusedtomeasuretheperformance. Clusterva- bandswiththewavelengthrangeof375to1050nm.
liditymeasuresarequantitativemethodstoevaluateclustering DuringthehyperspectralSLICapplication,K =500and
results based on intra-cluster compactness and inter-cluster m = 20. During unmixing using sPM-LDA, the number of
separability. Inourevaluation,eachsuperpixelwastreatedas endmembers was set to 7, λ = 1, α = 0.3, (cid:15) = 10% and
adistinctcluster. Thus,goodclustervaliditymeasurevalues T = 200 and the grey roof buildings were partially labeled
indicate that the spectral signatures within a superpixel are usingthesemi-supervisedapproachoutlinedin[10]. During
similartoeachbutdistinctfromothersuperpixels.Thebigger thefinalstage,K = 7wasusedduringK-meansclustering.
Dunn,SilhouetteindicesandthesmallerDBindexwillindi- Thesuperpixelsegmentationresultsfortheproposedmethod
cate a better partition of the image. We run each algorithm andcomparisonalgorithmsareshowninFig. 5.
for10times,andcalculatethemeansandstandarddeviations The results obtained over the MUUFL Gulfport data are
4. CONCLUSION
A new map-guided unmixing-based hyperspectral image su-
perpixel segmentation method is proposed. The proposed
methodobtainssuperpixelsegmentationresultsthatleverage
any available map-information and produces a superpixel
segmentationwithsemanticallymeaningfulsegments.
5. REFERENCES
(a) HSLIC (b) HSLIC+OSM
[1] M. Sethi, Y. Yan, A. Rangarajan, R. Vatsavai, and S. Ranka,
“Scalablemachinelearningapproachesforneighborhoodclas-
sificationusingveryhighresolutionremotesensingimagery,”
in Proc. of the 21th ACM SIGKDD Int. Conf. on Knowledge
DiscoveryandDataMining.ACM,2015,pp.2069–2078.
[2] D.B. Gillis and J.H. Bowles, “Hyperspectral image segmen-
tationusingspatial-spectralgraphs,” inSPIEDefense, Secu-
rity, and Sensing. Int. Soci. for Optics and Photonics, 2012,
pp.83901Q–83901Q.
(c) HNC (d) HUCM [3] D.R.Thompson,L.Mandrake,M.S.Gilmore,andR.Castan˜o,
“Superpixelendmemberdetection,” IEEETrans.Geosci.Re-
moteSens.,vol.48,no.11,pp.4023–4033,2010.
[4] M. Haklay and P. Weber, “Openstreetmap: User-generated
streetmaps,” IEEEPers.Commun., vol.7, no.4, pp.12–18,
2008.
[5] R. Achanta, A. Shaj, K. Smith, A. Lucchi, P. Fua, and
S. Su¨sstrunk, “Slic superpixels compared to state-of-the-art
superpixelmethods,” IEEETrans.PatternAnal.Mach.Intell.,
vol.34,no.11,pp.2274–2282,2012.
(e) PM-LDA (f) sPM-LDA [6] S.ZouandA.Zare, “Hyperspectralunmixingwithendmem-
bervariabilityusingpartialmembershiplatentdirichletalloca-
Fig. 5. Superpixel Segmentation on Gulfport with 6 algo- tion,” Int.Conf.onAcoustics,Speech,andSignalProcessing,
rithms 2016.
[7] X.Zhang,S.E.Chew,Z.Xu,andN.D.Cahill, “Slicsuper-
pixelsforefficientgraph-baseddimensionalityreductionofhy-
perspectralimagery,”inSPIEDefense+Security.Int.Soci.for
OpticsandPhotonics,2015,pp.947209–947209.
qualitatively similar to those obtained for Pavia University.
[8] W.Song,J.M.Keller,T.L.Haithcoat,andC.H.Davis, “Au-
QuantitativeevaluationshowninTable2usingclustervalidity
tomatedgeospatialconflationofvectorroadmapstohighres-
metricsonMUUFLGulfportindicatesthatproposedmethod olutionimagery,” IEEETrans.ImageProcess.,vol.18,no.2,
outperformscomparisonmethodsonthisdatasetaswell. pp.388–400,2009.
[9] S.Holzwarth,A.Muller,M.Habermeyer,R.Richter,A.Hau-
sold, S. Thiemann, and P. Strobl, “Hysens-dais 7915/rosis
imaging spectrometers at dlr,” in Proc. of the 3rd EARSeL
Dunn Davies-Bouldin Silhouette
HSLIC 8.220×10−4±0.00 12.772±0.08 −0.906±0.03 WorkshoponImagingSpectroscopy,2003,pp.3–14.
HSLIC+OSM 9.042×10−4±0.00 12.638±0.08 −0.882±0.02
[10] S.Zou, “Semi-supervisedinteractiveunmixingforhyperspec-
HNC 0.014±0.01 12.570±0.09 −0.795±0.03
HUCM 0.040±0.01 9.390±0.04 −0.659±0.03 tral image analysis,” M.S. thesis, Univ. Missouri-Columbia,
PM-LDA 0.067±0.01 8.049±0.05 −0.546±0.03 Columbia,MO,2016.
Proposedmethod 0.078±0.02 8.035±0.05 -0.515±0.03
[11] J.C.Dunn, “Well-separatedclustersandoptimalfuzzyparti-
Table 2. Dunn, Davies-Bouldin, Silhouette indices ± stan- tions,” J.Cybern.,vol.4,no.1,pp.95–104,1974.
darddeviationonGulfport
[12] D.L.DaviesandD.Bouldin, “Aclusterseparationmeasure,”
IEEETrans.PatternAnal.Mach.Intell.,,no.2,pp.224–227,
1979.
[13] P.J.Rousseeuw, “Silhouettes: agraphicalaidtotheinterpre-
tationandvalidationofclusteranalysis,” Journalofcomputa-
tionalandappliedmathematics,vol.20,pp.53–65,1987.
[14] P.Gader, A.Zare, R.Close, J.Aitken, andG.Tuell, “Muufl
gulfport hyperspectral and lidar airborne data set,” Univ.
Florida, Gainesville, FL, USA, Tech. Rep. REP-2013-570,
2013.