Surface Reconstruction with Data-driven Exemplar Priors

6.3 MB·

by  Oussama Remil

Surface Reconstruction with Data-driven Exemplar Priors
OussamaRemila,b,QianXiea,XingyuXiea,KaiXuc,JunWanga,d,∗
aNanjingUniversityofAeronauticsandAstronautics,China
bEcoleMilitairePolytechnique,Algeria
cNationalUniversityofDefenseTechnology,China
dHuaiyinInstituteofTechnology,China
Abstract
In this paper, we propose a framework to reconstruct 3D models from raw scanned points by learning the prior knowledge of a
specificclassofobjects. Unlikepreviousworkthatheuristicallyspecifiesparticularregularitiesanddefinesparametricmodels,our
7shapepriorsarelearneddirectlyfromexisting3Dmodelsunderaframeworkbasedonaffinitypropagation. Givenadatabaseof
1
3D models within the same class of objects, we build a comprehensive library of 3D local shape priors. We then formulate the
0
2problemtoselectas-few-as-possiblepriorsfromthelibrary,referredtoasexemplarpriors. Thesepriorsaresufficienttorepresent
the3Dshapesofthewholeclassofobjectsfromwheretheyaregenerated. Bymanipulatingthesepriors,weareabletoreconstruct
n
ageometrically faithful models with the same class of objects from raw point clouds. Our framework can be easily generalized to
Jreconstruct various categories of 3D objects that have more geometrically or topologically complex structures. Comprehensive

2experimentsexhibitthepowerofourexemplarpriorsforgracefullysolvingseveralproblemsin3Dshapereconstructionsuchas
1preservingsharpfeatures,recoveringfinedetailsandsoon.

]Keywords: 3Dlocalshapepriors,data-drivenexemplarpriors,affinitypropagation,surfacereconstruction.
G
C
.1. Introduction heuristics and hence only suitable for particular types of ob-
s
c jects, e.g., planar or man-made ones. This limitation prevents
[ Surface reconstruction from raw point clouds of real-world
these methods from reconstructing models with more general

objects is of great practical importance in computer aided de-
1 geometrywhichisnotcapturedbythepre-definedpriors.
signandcomputergraphics. Givenasetofunorganizedpoints
v
0sampled from the surface of an object, the goal is to recover A geometric surface can be reconstructed using a combi-
3theoriginalsurface. Thisinverseproblemisknowntobeinher- nation of local patches, which can be referred to as local pri-
2entlyill-posedifsolvedwithoutanypriorassumption. Despite ors. The number of these priors could be huge, while most of
3therecentadvancesin3Dscanningtechnology,thepointclouds themsharethesamepropertiesandtheyaremorelikelytohave
0
acquired with scanning devices inevitably contain imperfec- thesameshape,speciallywhenthesepriorsareextractedfrom
.
1tions, such as noise and density anisotropy. Consequently, it modelswithinthesameclassofobjects. Actually, themodels
0makestheproblemevenmorechallengingtoensurethatthere- belonging to the same class can possess a significant number
7
1constructed surface is geometrically and topologically faithful ofredundantlocalpriorssharingthesamegeometry,therefore
:againsttheoriginalonewiththeco-existenceofsharpfeatures theseobjectscanbeonlyrepresentedbyasmallnumberofdis-
v
(suchasedgesandcorners)anddataartifacts. tinct local patches. Based on this observation, we propose to
i
X Extensive study has been conducted on surface reconstruc- use a clustering method to learn a set of local priors from a
rtion over the past few decades [1]. Existing methods rely on database of 3D models, rather than pre-defining heuristic re-
acertain prior assumptions either on the sampling process, or constructionpriors.Thesepriorsareabletorepresentthewhole
about the nature of the surface being processed. Those gen- classofobjectsfromwhichtheyareextractedwithmuchfewer
eralpriorassumptions,suchasthesmoothsurfaceandGaussian parameters, and they can be used to faithfully reconstruct the
noisedistribution,oftenleadtoinferiorresultsfortherawpoint raw point cloud of an object belonging to that same class, see
clouds acquired from real-world objects [2]. Several methods Figure1.
taketheadvantageofexplicitlydefinedpriorsinspiredbyaset
In our framework, we assume no prior knowledge from the
of example shapes [3, 4]. Recently, a few works leverage the
shape,andthepotentialpriorsthatrepresenttheshapeoftheob-
repetitionandsymmetryassumptionstoextractstructuralpriors
jectsaretreatedunknowninadvance. Ourtaskismoregeneral
forsurfacecompletionandreconstruction[5,6].Amajordraw-
in that it extracts the priors automatically with a learning pro-
backcommontothesemethodsisthatthepriorsaredefinedby
cess. Givenadatabaseof3Dmodelswithinthesameclassof
objects,wesamplepoint-setneighborhoodstobuildacompre-
∗Correspondingauthor hensivelibraryoflocalshapepriorsfromallthose3Dshapes.
Emailaddress:[e-mail](JunWang) Ourpurposeistoselectarepresentativesetofpriorsfromthe
PreprintsubmittedtoCAD January13,2017
Object
Input Cloud Database Sparse Priors Matching and Consolidation Poisson Ours
Figure1:Givenanoisyandsparsepointcloudofstructuralcomplexmechanicalpartasinput,oursystemproducestheconsolidated
points by aligning exemplar priors learned from a mechanical shape database. With the additional information such as normals
carriedbyourexemplarpriors,ourmethodachievesbetterfeaturepreservationthandirectreconstructionontheinputpointcloud
(e.g.,Poisson).
library which can reconstruct any model from the same class • Wedeviseaframeworkforsurfacereconstructionfromex-
ofobjects. Weformulatetheproblemasminimizingtherecon- isting 3D models, which requires no interaction and can
structionerrorofalldatabasemodelsusingonlytherepresenta- easily be generalized to reconstruct various categories of
tivepriors. Tosolvethisproblem,werefertoAffinityPropaga- 3D objects that have more complex structures from raw
tion(AP)algorithm[7],giventhesimilaritiesbetweenallprior scanneddata.
library,itfindsthesetofpriorsthatminimizetheoverallsumof
similaritiesbetweenallpriors,namelyexemplarpriors. These • Weutilizeawell-knownclusteringmethod(AffinityProp-
priorsareabletoreconstructthe3Dshapewiththesameclass agation)toautomaticallylearntheexemplarpriorsfroma
fromrawpointclouds. databaseof3Dshapes,whichcanrepresentthe3Dshape
ofthewholeclassofobjects.
Since our priors are directly generated from the database
models,theycarrytheadditionalgeometricalinformationsuch
as normals and point feature labels (i.e., regular point, edge
2. RelatedWork
point and corner point), and also contain abundant small fea-
tures. This can be advantageous in two ways: 1) assist in the
Surfacereconstruction. SurfaceReconstructionfromrange
detection of sharp features from the input scan and 2) recover
scanned data is a long-standing research topic which has re-
thefinedetailswithintheinputpointdataeventhoughtheraw
ceived extensive attention from both the computer vision and
pointcloudisfairlynoisyandsparse. Asaresult,thequalityof
computer graphics communities. We refer the reader to the
therawpointdatacanbesignificantlyenhanced,whichensures
comprehensive survey on recent surface reconstruction tech-
thefollowingqualitysurfacereconstruction.
niques[1]. Shapereconstructionalgorithmsaimtoconvertan
It is worth mentioning that Gal et al. [2] also proposed an
input point cloud produced by aligned range scans that may
example-basedsurfacereconstructionmethodforscannedpoint
have noisy and incomplete data into a high-quality surface
sets, which utilizes a library of local shape priors built from a
model. An increasing number of surface reconstruction meth-
specifically chosen models. Their prior library is created by ods merge to handle this difficult and ill-posed problem. One
addingallthegeneratedlocalpatchesfromthedatabasemod-
class of these methods is Delaunay based surface reconstruc-
els. As a consequence, the number of the priors in the library
tion methods. They interpolate the surface by taking a subset
could be huge, which makes the matching process quite time-
of points from an unorganized input cloud as mesh vertices,
consuming. As mentioned before, the priors generated from a
suchastheCocone[8,9]andthePowerCrust [10]algorithms.
databasewithinthesameclasscouldhaveduplicatepriorsthat
These methods frequently produce jaggy surfaces in the pres-
share the same geometric properties, thus the redundancy of
ence of noise, and they require a pre-processing phase to im-
thepriorswouldbeimmoderate. Incontrast,ourmethodlearns
provetheirsmoothness.
only a set of representative priors from the library, which can
Another class of algorithms use surface approximation to
beusedforthereconstructionprocesswithmuchlessdifficulty
generateameshfromaniso-surfaceofabest-fitimplicitfunc-
andtime.
tion of the input point cloud [11, 12, 13]. Recently, a sig-
We demonstrate through experiments on the synthetic and nificant effort has been placed on reconstructing surfaces with
real scan point clouds that our exemplar priors are an efficient sharpfeatures[14,15,16,17].Thistypeofmethodscanhandle
tool to solve a variety of problems for the surface reconstruc- dataimperfectionstosomeextentinthepresenceofhighqual-
tion. Tosumup,themaincontributionsofthispaperare: ity normals. However, normal estimation and orientation are
2
challenging problems, especially when the point data are cor- idea as [2] by using the self-similarity patches to enhance the
rupted. Incontrast, ouralgorithmdirectlyusesthenormalsof input cloud data and reconstruct a faithful surface. Instead of
theexistingmodelsfortheinputrawscans,whichusuallycan usingthewholelibraryofpatches,ourmethodlearnsonlyaset
beaccuratelyestimatedfromthetriangularmeshmodels. An- ofexemplarpriorspermittingmuchlessdifficultyandtime.
other difficulty is to preserve sharp features of the input point
dataforthoseapproaches,theycanpreservethesharpfeatures
3. Overview
in the face of a low level of noise. However, they have dif-
ficulties in obtaining satisfactory results in the presence of a Ourproposedmethodaimstolearnasetofexemplarpriors
reasonablyhighlevelofnoise. fromagiven3Dshaperepository,andusethemtoreconstruct
Point cloud consolidation. Point cloud consolidation aims to surfaces from raw scanned point data. The shape repository
clean up raw input data, removing a variety of data artifacts, usually consists of a collection of 3D models with the same
and to provide essential geometric attributes. A variety of al- shape category. In our work, we explore the 3D shapes from
gorithms have been proposed for this purpose to benefit pro- availablerepositoriessuchasthePSBbenchmark[32],theCo-
cessingtaskssuchassamplingandsurfacereconstruction. For Seg dataset [33] and the scape database [34]. Our algorithm
example, Schall et al. [18] presented a noise removal method comprisesthreemajorphases,asillustratedinFigure2.
on the static and time-varying range data. Rosman et al. [19] Inthefirstphase,webuildaninitiallibraryof3Dshapepri-
introduced a framework for point cloud denoising by patch- orsfromasetofdatabasemodels. Specifically,the3Dshapes
collaborative spectral analysis. The Moving Least Squares aresampled, theseedcentersofthepriorsaredeterminedand
(MLS)basedmethodscanhandlenon-uniformdataandnoise, the 3D shape priors are created within a bounding sphere of a
and are ideally designed to produce smooth surfaces [20, 21]. predefinedradius.Thegeometricinformationsuchasshapede-
ThevariantsofMLShavebeendevelopedtohandlesharpfea- scriptors,normals,labelsforpointsbelongingtosharpfeatures,
turesbyfittingalocalparametricsurfaceateachpointfromthe areextractedandstoredforallthepriorsinSection4.1.Thepri-
inputcloud[14,17]. orsaretheninsertedintoasearchstructurebasedontheirshape
Recently, a locally optimal projection (LOP) operator was descriptors.
introduced by Lipman et al. [16]. Given an input noisy point The initial library consists of a large amount of shape pri-
cloud,itoutputsanewpointsetwhichmorefaithfullyadheres ors, whichmaycontainnumerousredundantelements. Toad-
totheunderlyingshape. Onthisbasis,Huangetal.[22]mod- dressthisissue,weaimtoextractthemostrepresentativepriors
ified and extended the LOP (weighted locally optimal projec- fromthelibrary. Weformulatetheproblemasminimizingthe
tion) to deal with non-uniform distributions. These methods reconstructionerrorofalldatabasemodelsusingonlytherep-
often fail to recover small features and details when the input resentative priors. For that purpose, we perform affinity prop-
data are fairly sparse and severely noisy. Comparatively, on agation [7], which given the similarities between all priors, it
thebasisoftheexemplarpriors,ourmethodiscapableofaug- selectssomeexemplarpriorsthatminimizetheoverallsumof
mentingandconsolidatingtherawpointdatasuchthatthedata similarities between all the priors and the exemplar ones (see
qualitygetssignificantlyenhanced. Asaresult,thefinedetails Section4.2). Asaresult,asmallnumberofexemplarpriorsare
andsmallfeaturescanberecoveredsuccessfully. obtained.
Non-local filtering. Non-local filtering methods has gained Oncetheexemplarpriorsareselectedfromthelibrary, they
interest over the past decade, Previous works in the image can be utilized to augment an input raw point scan from the
processing field such as [23] attempt to tackle this problem, same class of objects where the priors are created in the first
by introducing the notion of non-local means (NLM). The place. Givenaninputcloud,wefirstselecttheseedcentersfor
idea behind is to denoise a pixel by exploiting pixels of the thelocalneighborhoodsinsuchawayalltheinputcloudiscov-
whole image that may entail the same information. This idea ered,andthenweconstructthelocalneighborhoodswithinthe
was also introduced for surfaces, defined as meshes or point sameboundingsphereusedtocreatethepriors. Priorsmatch-
clouds [24, 25, 17, 26, 27] where they extend the non-local ingandalignmentproceduresarethenperformedtomatcheach
meansconcepttomeshandpointclouddenoising. Schnabelet localneighborhoodtoitsnearestexemplarprior,transformthe
al.[28]presentedaholefillingmethodwherethereconstruction exemplarpriortothelocalframe,andfinallyregisterthecorre-
isguidedbyasetofprimitiveshapeswhichhasbeendetected sponding prior onto the input scan using ICP algorithm (see
ontheinputpointcloudsuchasplanesandcylinders. Zhenget Figure 2). Consequently, the input raw point cloud is aug-
al.[5]proposedascan-consolidationapproachtoenhanceand mentedusingtheexemplarpriorsandiswellconsolidated. By
consolidate imperfect scans of urban models using local self- leveraging the moving least squares technique, we are able to
similarity. Guillemotetal.[29]introducedanonlocalpointset generate the faithful surface to the input data, where sharp
surfacemodelthatisabletoexploitthesurfaceself-similarity. features and fine details are successfully recovered (see Sec-
More related to our work, [30, 31] proposed a compression tion4.3).
technique that exploits self-similarity within a point-sampled
surface. In [30] the patches are clustered by similarity and
4. Algorithm
replacedbytherepresentativepatchesofeachcluster,whilein
[31] each patch is represented as a sparse linear combination Inthissection,wediscusseveryphaseoftheproposedalgo-
over a dictionary. Our work investigates the same non-local rithm in more detail. We first construct a library of 3D shape
3
Prior library construction Exemplar priors learning
… … … … …
Shape repository Priors library Search structure Affinity propagation Exemplar priors
Surface reconstruction
… …
…
Input cloud Locals Priors matching Augmentation procedure Final result
Figure2: Anoverviewofouralgorithm. Weextractpriorsfroma3Dshapedatabasewithinthesamecategory(e.g., mechanical
parts)toconstructapriorlibrary. Theaffinitypropagationclusteringmethodisthenperformedonthepriorlibrarytoobtaintheset
ofrepresentativepriors,calledtheexemplarpriors. Givenaninputpointcloud,weconstructitslocalneighborhoodsandperform
priorsmatchingtofindthesimilarexemplarpriortoeachlocalneighborhood.Subsequently,weutilizethematchedexemplarpriors
toconsolidatetheinputpointscanthroughanaugmentationprocedure,withwhichwecangeneratethefaithfulsurfacewheresharp
featuresandfinedetailsarewellrecovered.
priors from a given set of database models, we compute prior where(cid:107)c − p(cid:107)isthegeometricdistancebetweenthecenterc
j j
descriptors, and perform Affinity Propagation to identify the andthepoint p.
subsetofexemplarpriors. Basedonthesepriorsonly,wecon- Oncethepriorsareconstructed,westoretheinformationof
solidatetherawpointcloudswithinthesameclassoftheinput each prior such as normals and points belonging to sharp fea-
3Dshapesandreconstructfaithfulsurfacemodels. turesforlateruse, weperformweightedPrincipalComponent
Analysis (wPCA) for each prior, in such a way the principal
4.1. PriorLibraryCreation axescoincidewiththecoordinatesystemaxesanditscenterof
Prior library. A prior Pr of a 3D model M is defined as massistranslatedtotheorigin,inaddition,wescalethelargest
thesetofpoints p∈MlyingwithinasphereofradiusRanda principal component to 1 for all priors. Given a database of
centerc.Givena3Dshaperepositorywiththeassumptionsthat models,weareabletoconstructourinitialpriorlibrarywithin
themodelsarerepresentedastrianglemeshesandtheybelong thesamestandardcoordinatesystem.
to the same class, we construct a library of such local priors, Shape descriptor. Before we cluster similar priors into
and use them as input for the learning process. The first step groups,werequiretomeasurehowthepriorsaresimilartoeach
is to sample a set of points P uniformly for each model, then other,inanotherword,weneedtodefineashapedescriptorto
weselectasubsetS ofseedcentersc fromPwhichsatisfythe present each prior. Similar to [2], we introduce the geometric
j
followingproperty: theseedcentersareusedtocreatethelocal moments as our priors descriptor thanks to their efficient and
priors within a bounding sphere of radius R in such a way the easy computation. Since our library priors are formed by dis-
pointcloudPistotallycovered. cretepoints,weyieldanapproximationtotheintegralformof
Theseedcentersareselectedinadart-throwingfashionsame themoments.
as [31]. Given the sampled set of point P for each model, we GivenasetofpointsX={xi,yi,zi}iN=1,their(p,q,r)moment
pick a point as a first center seed, label all its neighborhoods isapproximatedwiththesummationoverallthepoints. Inor-
withinaradiusRascovered,andtraversePuntilanon-covered der to make the moment-based representation invariant to the
pointisfoundandanewseedcenterisaddedtoS,theprocess numberofverticesofeachprior,weintroduceaweightωi for
repeatsuntilallpoints Parecovered. NotethattheradiusRis eachpoint{xi,yi,zi}thatisproportionaltotheareaofthepolyg-
chosenasarelativevaluetothediagonaloftheboundingbox onalface[35]wherethatpointissampled. Moreformally:
ofeachmodel. Thepriorsarethendenotedby:
1 (cid:88)N
M (X)= ωxpyqzr (2)
Pr(c )={p|p∈ P,c ∈S,(cid:107)c −p(cid:107)≤R} (1) p,q,r N i i i i
j j j i=1
4
Wethenchooseasubsetofmoments{M }uptoanorder thegeometricdistancebetweentheoriginalmodelM andthe
p,q,r i
dasourpriordescriptor,andobtainthevectormomentsV(Pr): reconstructedmodelMˆ usingtheexemplarpriorsfromL(cid:63).
i
(cid:13) (cid:13)
V(Pr)={M0,0,1(Pr),M0,1,0(Pr),...,Md,0,0(Pr)}. (3) Ei =(cid:13)(cid:13)Mˆi−Mi(cid:13)(cid:13) (4)
Since the geometric moments can inherently represent the
In our implementation, we choose a sequence of moments up
shape,wereplacethegeometricdistancesbetweenmodelsMˆ
to d = 6, resulting in a 83-dimension descriptor vector. As i
and M with the sum of the Euclidean distances between the
aresult, weareabletocomputethecorrespondingdescriptors i
geometricmomentvectorsoftheirpriors.
forallpriorsinthelibrary,seeAlgorithm1.
Affinity propagation. In the last decade, many clustering
methods have been proposed. One particularly elegant and
Algorithm1Priorlibrarycreation
powerful method is the affinity propagation (AP) proposed by
Input: DatabasemodelsM,i={1,...,m}.
i [7],whichcanbeadaptedtosolveourselectionproblem. Itis
Output: PriorslibraryL.
anunsupervisedlearningalgorithmforexemplar-basedcluster-
functionPriorlibrarycreation(Mi,R) ing that finds a set of data points that best describe the input
fori=1tomdo databasedonthesimilaritiesbetweenthem. Itassociateseach
P←M(i); (cid:46)samplepointsandstorenormalsand data point with an exemplar within the input data in a way to
featurepoints. maximize the overall sum of similarities between data points
S ←subset(P); (cid:46)findseedcentersforpriors. andtheirexemplars.
for j=1tosize(S)do Thealgorithmtakesasinputacollectionofreal-valuedsim-
Pr(cj)←Equation1; (cid:46)generatetheprior. ilaritiesbetweenthepriors,wherethesimilaritys(i, j)between
Pr(cj)←WeightedPCA(Pr(cj)); priorsiand jindicateshowwellprior jissuitedtobetheexem-
Pr(cj)←Canonicalscaling(Pr(cj)); plarforpriori,hencethesimilaritymeasure s(i, j)issettothe
V(Pr(cj))←Equation3; (cid:46)computethe negativeEuclideandistancebetweenthegeometricmomentsof
descriptor. priorsiand j:
descriptortLot←hel{iPbrra(cryj).,V(Pr(cj))}; (cid:46)addthepriorandits s(i, j)=−(cid:13)(cid:13)(cid:13)V(Pri)−V(Prj)(cid:13)(cid:13)(cid:13),i(cid:44) j (5)
endfor Supposethattherearenpriorsinthelibrary, eachofwhich
endfor hasthegeometricmomentsexpressedbyan83-dimensionvec-
return(L); tor.Weconstructthen×nsimilaritymatrixsbyapplyingequa-
endfunction tion(5)foreachpairofpriorsinthelibrary.
Affinity propagation does not require the number of cluster
to be specified. It takes as input real values for all s(j, j) = p
4.2. ExemplarPriorsLearning calledaspreferences,andthesevaluesinfluencethenumberof
exemplarpriors(aprior jwithahighpreferencevalues(j, j)is
Since the prior library is created from 3D models belong-
morelikelytobechosenasanexemplar). Incaseallthepriors
ingtothesamecategoryofshapes,itmaycontainasignificant
are equally suitable as exemplars, their preferences should be
number of redundant priors. In this section, we aim to extract
settoasharedvaluesuchasthemedianoftheinputsimilarities,
arelativelysmallnumberofrepresentativepriorsorexemplars
whichresultsinamoderatenumberofexemplars(see [36]for
fromthelibraryusingtheaffinitypropagation(AP)method.
moredetail).
Problemformulation. GivenalibraryofpriorsL,ourpur-
Todecidewhichpriorsareexemplars,twokindsofmessage
poseistoreconstructalldatabasemodelsbyusingonlyasubset
arepassedbetweenthepriorsinthelibrary:(1)theresponsibil-
ofexemplarspriorsfromL, denotedbyL(cid:63), whichminimizes
ityr(i, j)sentfromprioritocandidateexemplar j,representing
thecostfunctiondefinedbythereconstructionerrorofallthese
howwellprior jissuitedtobetheexemplarofprioriand(2)
models. Specifically,wetakeadvantageofthesimilaritymea-
the availability a(i, j) sent from candidate exemplar j to prior
surebetweeneachpairofpriorstoidentifytheexemplarpriors
i, indicating how well for prior i to choose prior j as its ex-
thatminimizethestandardsquarederrormeasureofsimilarity.
emplar. Bothmessagesareupdatedduringeachiterationuntil
Fortunately,thisproblemcanbesolvedbytheaffinitypropaga-
convergencebyusingthefollowingequations:
tionmethodthatrecursivelytransmitsreal-valuedpairwisedata
pointsimilaritiesuntilagoodsetofexemplardataisobtained. ∀i, j:r(i, j)= s(i, j)− max{s(i, j(cid:48))+a(i, j(cid:48))} (6)
j(cid:48),j(cid:48)(cid:44)j
Itsearchesforclusterssoastominimizeanobjectivefunction,
i.e.,thereconstructionerror. ∀i, j: for j=i:
LetMmi=1 bethesetofmdatabasemodelsandL(cid:63) thesetof a(i, j)= (cid:88) max{0,r(i(cid:48), j)} (7)
k exemplar priors, i.e., L(cid:63) ∈ L. To reconstruct all the input
i(cid:48),i(cid:48)(cid:44)i
models Mm , we search for each prior in L the most similar
i=1 ∀i, j: for j(cid:44)i:
priorfromL(cid:63),wethenalignthecorrespondingpriorsfromL(cid:63)
(cid:110) (cid:88) (cid:111)
ontotheirsimilarpriorsfromeachmodelMi toformanentire a(i, j)=min 0,r(j, j), max{0,r(i(cid:48), j)} (8)
model, Mˆ . Hence, we define the reconstruction accuracy as
i i(cid:48),i(cid:48)(cid:60){i,j}
5
we assign to each prior i a variable cˆ, a value of cˆ = j for firstpointasaseedandsearchitsneighboringpointsetwithin
i i
i (cid:44) jmeansthattheprioriisassignedtotheclusterwhere jis a bounding sphere of radius R, and we label the neighboring
anexemplarprior;andcˆ = jindicatesthattheprior jservesas pointsascoveredandcontinue withthesamemanneruntilall
j
anexemplar. Atanypointofthealgorithm,responsibilitiesand pointsarecovered. Asaresult, thesetoflocalneighborhoods
availabilities can be combined to make the exemplar decision Nb is obtained. We transform them to a canonical placement
andidentifytheclusterassignmentscˆ (i=1,...,n)definedas: usingtheweightedPCApositioningexplainedbeforeandstore
i
them for later use. Subsequently, the shape descriptor V(Nb)
cˆi =argmax{a(i, j)+r(i, j)} (9) is calculated for each transformed local neighborhood using
j
Equation3.
Convergence. Thealgorithmcanbemonitoredbyimposing Oncetheshapedescriptorsofalllocalneighborhoodswithin
certain terminal conditions: a maximal number of iterations, Pareobtained, wesearchforeachlocalneighborhoodNbthe
thechangesinthemessagesarebelowathresholdorthelocal most similar exemplar prior from L(cid:63) by computing the Eu-
decisionsstayconstantforsomenumberofiterations. Weini- clidean distances between their corresponding geometric mo-
tialize the availabilities to zero and for each iteration, we (1) ments.Tofindmorereliablematches,wedonotjustrelyonthe
update the responsibilities given the availabilities; (2) update distancebetweenshapedescriptors,butalsouseamoreflexible
theavailabilitiesgiventheresponsibilities;(3)combinebothof matchingrefinementprocedure,whichproceedsonthebasisof
themtomonitortheexemplardecision;(4)terminatethealgo- theconfidencescoremetricdefinedwiththemeansquareder-
rithm when the decisions do not change for 10 iterations, see ror of the point distances [2]. Thus, the similar priors can be
Algorithm2. reliablyfoundforallthelocalneighborhoodswithintheinput
scan.
Algorithm2ExemplarPriorsLearning After finding the corresponding exemplar prior for each lo-
Input: PriorlibraryLwithnpriors. cal neighborhood, we apply the inverse of the transformations
Output: IndicesoftheexemplarpriorsInd. stored before to each matched exemplar prior, and then align
functionLearningPriors(L) them into the input data P. The Iterative Closest Point (ICP)
fori=1tondo registrationtechnique[37]isthenperformedtorefinethealign-
for j=1ton,j(cid:44)ido mentofmatchedpriorstoP. Consequently,therawpointscan
s(i, j)←equation(5); (cid:46)constructsimilarity is consolidated and thus the data quality is significantly im-
proved.
matrix
endfor Having the consolidated point cloud, we exploit the similar
endfor reconstructionmethodin[2]basedonthemovingleastsquares
p←median(s); (cid:46)themedianofsasavectorof technique [20]. As a result, we are able to produce geometri-
preferences callyfaithfulsurfaces,wheresharpfeaturesandfinedetailsare
fori=1tondo wellpreserved.
s(i,i)← p(i);
endfor Algorithm3SurfaceReconstruction
∀i, j:a(i, j)=0; (cid:46)initializeavailabilities. Input: RawPointScanP,ExemplarPriorsL(cid:63).
unconverged=true; Output: ReconstructedSurfaceS.
whileunconvergeddo functionSurfaceReconstruction(P,L(cid:63))
r(i, j)←equation(6); (cid:46)updateresponsibilities. C ←subset(P); (cid:46)centersofthelocalneighborhoods.
a(i, j)←equations(7)and(8); (cid:46)update fori=1tosize(C)do
availabilities. c←C(i);
cˆi ←equation(9); (cid:46)computeclusterassignments. Nb(c)←equation1; (cid:46)generatethelocal
unconverged←checkconvergence; neighborhood.
endwhile WeightedPCA(Nb(c)); (cid:46)storethetransformation.
Ind ←cˆi; (cid:46)getindicesofexemplarpriors. Canonicalscaling(Nb(c)); (cid:46)storethescale.
return(Ind); V(Nb(c))←equation(3); (cid:46)geometricmomentfor
endfunction thelocalneighborhood.
Index←ANN(V(Nb(c)),V(L(cid:63))); (cid:46)nearest
exemplarprior.
4.3. ConsolidationandReconstruction Pr←InversetransformationofL(cid:63)(Index);
Oncetheexemplarpriorsareobtainedfromtheaffinityprop- Pr←ICP(Nb(c),Pr);
P←P+Pr; (cid:46)augmentedpointcloud.
agation algorithm, they can be used to consolidate and recon-
endfor
structrawpointcloudssameasin[2],seeAlgorithm3.
S ←MLS(P); (cid:46)movingleastsquare.
Given an input raw point scan P, we construct a set of lo-
return(S);
cal neighborhoods Nb from P, as we process the initial priors
endfunction
in Section 4.1. We select a set of local centers by picking a
6
Chair repository Exemplar priors Mechanical part repository Exemplar priors
Figure3: Asetofexemplarpriorsforchairandmechanicalpartdatasets. Inthechairdatabase,thereareatotalof24chairswith
differentdesignstyles. Usingouralgorithm, 208exemplarpriorsareobtained. Theoretically, those208representativepriorscan
be used to reconstruct any type of chairs with the same style. For the mechanical parts, we choose 18 model and generate 196
exemplarpriors. Foreaseofexposition,weillustrateonly50exemplarpriorsforbothdatasets.
…
Ours
Gal et al
…
(a) (b) (c) (d) (e)
Figure 4: Comparison to Gal et al. [2]. (a) Training datasets with a set of learned exemplar priors. (b) Input point data and the
reconstructionresultfromPoisson. (c)Augmentationoftheinputpointdatausingtheexemplarpriorsandthereconstructedmodel
fromourmethod. (d)AugmentationoftheinputcloudusingallpriorsinthelibraryandthereconstructedresultfromGaletal. (e)
Thereconstructionerrorsandtherunningtimeintermsofdifferentnumbersofexemplarpriors.
5. Results Note that our method learns only a small set of exemplar pri-
ors(around5%ofthepriorlibrarysize),whilethereconstruc-
tion in Gal et al. [2] utilizes all priors from the library. Based
In this section, we run our reconstruction method on a va-
riety of raw point clouds, and we validate its effectiveness in on all original priors, apparently, the reconstruction quality is
the best and thus the error is the lowest. However, The pri-
terms of feature preservation and detail recovery. In order to
orsgeneratedfromacollectionof3Dmodelsofthesameclass
demonstrate the power of our exemplar priors (Figure 3 illus-
could have many duplicate priors that share similar geometric
trates the exemplar priors obtained for chair and mechanical
properties, thus the number of priors presented in the library
partdatasets),wecomprehensivelycompareourmethodtothe
could be huge, which makes the matching process quite time-
other related methods on both benchmark data and raw point
consuming. AscanbeseeninFigure4(e),whenthenumberof
scans. Additionally, we perform quantitative evaluations to
chosenpriorsdecreases,thereconstructiontimedecreasesand
measurethereconstructionqualityofouralgorithm.
theerrorbecomeshigher. Incontrastto[2],ourmethodmakes
agoodtrade-offbetweenthenumberofexemplarpriorsk and
5.1. ParameterEffects thereconstructionquality,andconsolidatesinputcloudsbased
onalimitednumberofpriors,whichoffersmoreefficientpriors
Numberofchosenpriors. Figure4showstheeffectofthe
matching, resultinginfasterreconstructionsuptothreeorders
number of chosen priors on the reconstruction quality and the
ofmagnitudecomparedto[2]. Whenusingthesetofexemplar
running time. Given two collections of 3D models, we gener-
priorslearnedbyouralgorithm,thereisagoodbalancebetween
atetheirrespectivepriorlibraries. TheinputrawscansinFig-
thereconstructionqualityandtherunningtime. Thissuggests
ure4(a)aresparseandcontainacertainlevelofnoise. Using
thatweareabletoreconstructageometricallyfaithfulmodels
ouralgorithm,wecanplotthereconstructionerrorsandtherun-
from raw point data with much less difficulty and time, even
ningtimeintermsofthedifferentnumbersofexemplarpriors.
7
…
…
(a) (b) (c)
(a) (b) (c)
Figure6:PreservingsharpfeaturesonthemodelofaMechani-
Figure5: PreservingsharpfeaturesontheFandisk model. (a)
calpart.(a)Thetrainingmechanicalpartdatasetandabenchof
Training dataset with a set of learned exemplar priors. (b) In-
exemplarpriorslearnedbyouralgorithm. (b)Therawscanned
putpointdatawith10%Gaussiannoiseandthereconstruction
pointdatawith10%Gaussiannoiseandthereconstructionre-
result using RIMLS. (c) Augmentation of the input point data
sultfromRIMLS.(c)Theaugmentedpointdataandtherecon-
using the exemplar priors and The reconstructed model from
struction result from our method. The close-up views clearly
our method. Comparatively, the sharp features are better pre-
show that our method is capable of recovering sharp features
servedbyourmethod.
eveninthepresenceofnoise.
though the scanned model is geometrically and topologically
eredwith ourapproach, sincetheyare originatedfrom thetri-
complex.
angular meshes. Figure 6 shows the reconstruction result of a
Priorsize. Ourreconstructionmethodhasonemajorparam-
mechanicalpartusingourmethod. WerunRIMLSontheinput
eter: priors radius R, which affects the reconstruction quality
scan data to get the reconstruction result in Figure 6(b), from
and the computation time. Note that the value of the radius R
which sharp features are severely blurred. By learning from a
shouldbechosendependingonthesizeofthemodelspresentin
setofmechanicalmodels,ouralgorithmchoosesasetofexem-
thedataset. IfthevalueofRistoobig,thenthepriorspossess
plarpriorsinFigure6(a),basedonwhichthelow-qualityscan
moremeaningfulsemanticswithasmallnumberofredundant
ispleasinglyconsolidated. Thereconstructionresultoftheen-
elements, resulting in a big number of exemplar priors so that
hancedinputcloudisproducedinFigure6(c),whereallsharp
thelearningphaseisabittime-consuming. Inaddition,thelo-
edgesarewellpreserved.
calneighborhoodsmaynotfindtheirsimilarexemplarpriorsin
Detail recovery. To demonstrate the effectiveness of our
thematchingprocess,whichcandefinitelyharmthereconstruc-
method in terms of detail recovery, we generate the surface
tion quality. On the contrary, if the value of the radius is too
modelsfromtheinputscansusingouralgorithmandthePois-
small,thenumberofpriorsinthelibrarycouldbehuge,while
sonreconstructionmethodonthearmadilloandthechairmod-
most of them look like a flat disk, leading to a small number
els. Figure7demonstrateshowourmethodcaneffectivelyre-
of exemplar priors. As a result, both the learning and the re-
coversmalldetailsfromnoisyrawpointdata. Givenadataset
construction phases are quite time-consuming. We choose R
ofarmadillomodelfrom[32,38],werunouralgorithmtolearn
between 0.05 and 0.1 times the diagonal of the bounding box
asetofexemplarpriors(Figure7(a)illustratesonly15priors
of each 3D model. In practice, We have conducted a number
oftestingsondifferentinputscans,andfoundthatthevaluesof from the 324 learned priors), priors matching and point cloud
augmentation procedures are shown in Figures 7(b) and 7(c)
Rintheinterval[0.05,0.1]timesthediagonalofthebounding
respectively. The reconstructed surface using Poisson method
boxofeach3Dmodelalwaysproducepleasingresultsinallex-
in Figure 7(d) from the input scan lacks significant details as
periments. Fordatasetsthatareverysimilartotheshapebeing
reconstructed (e.g., humans and armadillos), we set R = 0.1. shown in the zoom-in views. Comparatively, our method suc-
ceedsinrecoveringfinefeaturesbasedontheexemplarpriors.
Forthedatasetsthatundergosomegeometricvariations,wefix
Figure 8 illustrates the reconstruction results on a chair
Rat0.05.
model. We run the poisson reconstruction method on the in-
putscan,wherethesurfaceisbumpyandimportantdetailsare
5.2. Reconstruction
missing, as shown in Figure 8(b). By learning from a dataset
Sharpfeaturepreservation. Figure5comparestherecon- of chair models [39, 40], our algorithm chooses a total of
structionresultsonthefandiskscanfromRIMLS[17]andour 116exemplarpriors(8ofthesepriorsaredemonstratedinFig-
method. As can be seen in Figure 5(a), our algorithm is able ure8(a)).Usingthoseexemplarpriors,ourmethodiscapableof
to find the exemplar priors that can represent the mechanical recoveringfinedetails,evenfromasparseandnoisyrawscan,
part dataset. Using these priors to match and augment the lo- asdemonstratedinFigure8(c).
calneighborhoodsoftheinputscancaninherentlyimprovethe Pose invariant. Another important characteristic of our
quality of the input cloud and preserve the sharp features in learnedexemplarpriorsisthattheycandealwithmodelsfrom
Figure 5(c), while the sharp features are highly damaged by thesameclassofobjectswithdifferentposes,inanotherword,
RIMLS method in Figure 5(b). From the close-up views, the they can be used to reconstruct a new model with a new pose
normals of the points around the sharp edges are well recov- different from those used in the learning process. Figure 9
8
…
(a) (b) (c) (d) (e)
Figure7: DetailrecoveryontheArmadillomodel. Thefirstandsecondrowsrepresentthefrontandthebackviewsrespectively.
(a) The armadillo database used to learn the exemplar priors. (b) The input noisy raw data with a set of learned priors matched
with their local neighborhoods. (c) The alignment of the exemplar priors into the input point cloud, where initial alignments are
represented on the left and the final augmented point cloud is shown on the right. (d) Poisson reconstruction result of the input
pointcloud. (e)Ourreconstructionresult. Poissonreconstructionlackssignificantdetailswhileourmethodsuccessfullyrecovers
finedetails.
…
…
(a) (b) (c)
(a) (b) (c) (d)
Figure 8: Detail recovery on the Chair model. (a) The chair
Figure9: Dealingwithdistinctposes. (a)Thetraininghuman
databaseusedtolearntheexemplarpriors. (b)ThePoissonre-
databasewithvariousposesandasetoflearnedexemplarpri-
constructionresultontheinputrawpoints. (c)Thereconstruc-
ors. (b) The input point scans. (c) The augmentation proce-
tion result using our method. Compared to the poisson recon-
dure on the input point data. (d) Our reconstruction results.
struction,ourmethodcanrecovermoredetailsbenefitingfrom
Althoughthecorrespondingposesoftheinputscansarenotin-
thefinedetailscarriedbytheexemplarpriors,asdemonstrated cludedinthedatabase,ourexemplarpriorsstillcaneffectively
inzoom-inviews.
consolidatethemandreconstructgeometricallyfaithfulmodels.
constructgeometricallyfaithfulsurfacemodels,wheretheim-
showsthereconstructionresultsonhumanmodelswithdistinct
portantdetailscanbewellrecovered,seeFigure9(d).
poses from SCAPE database [34]. We train our algorithm on
30 human models of the same person with a variety of poses,
5.3. Comparisonstostate-of-the-artreconstructiontechniques
asaresult,296exemplarpriorsareobtainedtobeabletorep-
resent the human model, as can be seen in Figure 9(a). Given To demonstrate the effectiveness of our method, we com-
sparse and noisy input scans of human models in Figure 9(b), pareittothestate-of-the-artreconstructiontechniques,includ-
weusethelearnedpriorstoconsolidateandaugmenttheinput ingRIMLS[17],APSS[14]andPoisson[13]. Wetryourbest
clouds. Figure9(c)illustrates4stagesoftheaugmentationpro- to fine-tune the parameters for all those methods in order to
cedure where the exemplar priors are matched and aligned to achieve the best experimental results. The experimental data
theircorrespondinglocalneighborhoods. Thegenerateddense includesurfacereconstructionbenchmarkdata[1](Anchorand
pointcloudscontainfaithfulnormalssincetheyareoriginated Daratechpointclouds),aswellastherawscanneddataoftwo
from the input mesh models, and therefore we are able to re- mechanicalpartstakenfrom[41].
9
Raw point cloud APSS RIMLS Poisson Ours
Figure 10: Reconstruction comparison on the Anchor model from APSS(resolution=3203), RIMLS (resolution=3203), Poisson
(depth=12)andourmethod. APSSandRIMLScanpreservesharpfeaturestosomeextent,whileanumberofartifactsareyielded
intheresults. Poissonproducesthefavorablysmoothmodel,wheresharpfeaturesareblurred. Incontrast,ourresultisrelatively
better.
Raw point cloud APSS RIMLS Poisson Ours
Figure11: ReconstructioncomparisonontheDaratechmodelfromAPSS(resolution=3203),RIMLS(resolution=3203),Poisson
(depth=12)andours. Ourmethodoutperformstheothersintermsofsharpfeaturepreservationinthefaceofnoise.
Benchmarkdata.Figure10comparesthereconstructionre- shown in the close-up views, the edges of the cylindrical hole
sultsoftheAnchormodelwithsharpfeatures. Theinputpoint arecollapsedbyAPSSandRIMLS.Poissonsmoothsoutthose
datacontainacertainlevelofnoise. RIMLSandAPSSareable sharpedges,which,however,arewellpreservedbyourmethod.
to retain sharp features to some extent, while there are some Figure 13 compares the reconstruction results of the Wheel
bumpy artifacts. Moreover, a number of small holes exist in modelwithrawpointdata.Incontrast,ourmethodoutperforms
theirresultsduetonoise. Poissonyieldsasmooth,closedsur- the other three methods in terms of recovering sharp features,
facethankstoitsreconstructioncharacteristics.However,sharp evenforafairlynoisyscan.
featuresareseverelyblurred. Incontrast, ourmethodconsoli-
datestheinputpointdatabasedontheexemplarpriorslearned 5.4. QuantitativeEvaluations
from a set of mechanical part models, and hence produces a
Theresultsshownabovehavevisuallydemonstratedthesu-
quality surface, where sharp features are well preserved. Fig-
periority of our method regarding sharp feature preservation
ure 11 presents the comparison results of the Daratech model
and detail recovery on raw scanned points. In this section,
with small details. APSS, RIMLS and our method obtain rel-
we perform quantitative evaluations on our reconstruction ap-
atively better results than Poisson does in terms of retaining proach. WefirstshowtheHausdorffdistancehistogramsofthe
sharp features. Note that notable defects (e.g. holes) are still
reconstructionresultsfromFigures10-13. Wealsoinvestigate
generatedfromAPSSandRIMLS.Ashighlighted,RIMLSand the robustness of our method to different levels of noise and
ours preserve more details than APSS and Poisson do. How- sparsityinaquantitativemanner. Rawscandataalwayssuffer
ever,RIMLSruinsthesmallfeatures.Comparatively,onlyours
fromvaryingsamplingresolution,noise,andsparsity. Whileit
achievesmorefavorableresults,inwhichfinedetailsandsharp isdifficulttoexactlysimulaterealscandata,weincorporatethe
featuresareproperlyrecovered.
syntheticexperimentstovalidatetherobustness.
Raw scans. Figure 12 shows the comparison results of the Hausdorff distance. To demonstrate the geometric fidelity
Clutch part with raw point data. The raw point cloud con- of the reconstructed surface relative to the original scan, the
tainssomenoise,andthepointsaredistributednon-uniformly. Hausdorff distance between the original scan and the surface
Moreover, the data on some regions are fairly sparse. Due to mesh is computed. Figure 14 shows the detailed comparisons
noise, APSS and RIMLS generate a number of artifacts (e.g. of the Hausdorff distance results of Figures 10-13. The hor-
burrs), while Poisson obtains a visually pleasing surface. As izontal axis is the absolute distance value between the recon-
10