Generating Anatomical Substructures for Physically-Based Facial Animation

206 Pages·2011·28.73 MB·English

Generating Anatomical Substructures
for
Physically-Based Facial Animation
OLUSOLA OLUMIDE AINA
A thesis submitted in partial fulfilment of the requirements
of Bournemouth University for the degree of
Doctor of Philosophy
2011
November
Bournemouth University
COPYRIGHT
Thiscopyofthethesishasbeensuppliedonconditionthatanyonewhoconsultsitisun-
derstoodtorecognizethatitscopyrightrestswiththeauthoranddueacknowledgement
mustalwaysbemadeoftheuseofanymaterialcontainedin,orderivedfrom,thisthesis.
ABSTRACT
Physically-based facial animation techniques are capable of producing realistic facial
deformations,buthavefailedtofindmeaningfuluseoutsidetheacademiccommunity
because they are notoriously difficult to create, reuse, and art-direct, in comparison
to other methods of facial animation. This thesis addresses these shortcomings and
presents a series of methods for automatically generating a skull, the superficial mus-
culoaponeuroticsystem(SMAS–alayeroffasciainvestingandinterlinkingthemimic
musclesystem),andmimicmusclesforanygiven3Dfacemodel. Thisisdonetoward
(the goal of) a production-viable framework or rig-builder for physically-based facial
animation.
Thisworkflowconsistsofthreemajorsteps. First,agenericskullisfittedtoagivenhead
modelusingthin-platesplinescomputedfromthecorrespondencebetweenlandmarks
placedonbothmodels. Second,theSMASisconstructedasavariationalimplicitorradial
basis function surface in the interface between the head model and the generic skull
fittedtoit. Lastly,musclefibresaregeneratedasboundary-valuestraightestgeodesics,
connectingmuscleattachmentregionsdefinedonthesurfaceoftheSMAS.Eachstepof
thisworkflowisdevelopedwithspeed,realismandreusabilityinmind.
CONTENTS
1 Introduction 1
1.1 Motivation 1
1.2 Thesisobjectivesandcontributions 2
1.3 Outline 4
2 FacialAnimation: TechniquesandApplications 7
2.1 FacialAnimationtechniques 7
2.1.1 BlendShapeInterpolation 7
2.1.2 Parametrization 8
2.1.3 Performance-drivenFacialAnimation 9
2.1.4 Physically-basedfacialanimation 11
2.2 FacialAnimationReuse 15
2.2.1 Expressioncloning 15
2.2.2 Rigtransfer 15
2.2.3 Rigbuilding 17
2.3 TheUncannyValley 20
2.4 Applications 22
2.4.1 Film 23
2.4.2 Videogames 23
2.4.3 Medicineandsurgery 24
2.4.4 Human-computerinteraction(HCI) 24
2.5 Summary 25
3 AnatomyoftheHumanFace 27
3.1 SkeletalAnatomy 27
3.1.1 Craniofaciallandmarks 29
3.1.2 FacialTissueDepth(FTD)Measurements 30
3.2 MuscularAnatomy 30
3.2.1 Musclesoftheupperface 31
3.2.2 Musclesofthemidface 32
3.2.3 Musclesofthelowerfaceandneck 34
3.2.4 Masticatorymuscles 37
3.3 Variationsinmuscularanatomy 37
3.3.1 Variationoffacialmusclesattheangleofthemouth 40
3.4 SoftTissueAnatomy 42
3.4.1 Layer1-skin 42
3.4.2 Layer2-subcutaneous 43
4
3.4.3 Layer3-musculoaponeurotic 43
3.4.4 Layer4-sub-SMASordeepplane 45
3.4.5 Layer5-deepfascia 46
3.5 Nasolabialfold 46
3.6 Anatomyoftheagingface 47
3.7 Facialaestheticunits 48
3.8 Summary 49
4 Theoryandapplicationsofthin-platesplines 51
4.1 Thinplatesplines 51
4.1.1 Vectorspaces 51
4.1.2 Normedvectorspaces 52
4.1.3 HilbertSpaces 55
4.1.4 ReproducingKernelHilbertSpaces(RKHS) 55
4.1.5 RadialBasisFunctions 57
4.1.6 RegularizationinReproducingKernelHilbertspaces 59
4.1.7 Regularizationwithderivativeinformation 63
4.2 BasicApplications 65
4.2.1 Height-fieldinterpolation 65
4.2.2 Patch-baseddifferentialgeometry 70
4.2.3 Implicitsurfaceconstruction 74
4.2.4 Landmark-baseddeformation 75
4.2.5 Semilandmark-baseddeformation 78
4.3 Summary 85
5 Boundary-ValueStraightestGeodesics 87
5.1 Previouswork: reviewandapplications 89
5.2 PathTracingbyVectorProjection 91
5.3 PathStraighteningbyBridging 91
5.3.1 Numberofstraightlengthsinapolygonpath 95
5.4 Pathcorrectionaroundvertices 98
5.4.1 Straightestpossiblegeodesics 101
5.5 Summary 104
5.5.1 Futurework 105
6 Alandmarkbasedmethodforskullfitting 107
6.1 Skullfittingprocess 107
6.1.1 Modelpreparation 107
6.1.2 Basicskullfittingprocess,usingsofttissuedepthdata 109
6.1.3 Incorporatingsemilandmarksandderivativeinformation 113
6.2 Summary 118
6.2.1 Futurework 118
5
7 Constructing facial muscles and the superficial musculoaponeurotic system
(SMAS) 123
7.1 SMASConstruction 124
7.1.1 FastdiscretizationoftheSMASbyacceleratingthemarchingtrian-
glesalgorithm 125
7.2 MuscleConstruction 128
7.2.1 Definingmuscleattachment(originandinsertion)regionsonthe
SMAS 128
7.2.2 Computingtheconvexhullsofmuscleattachmentregions,onthe
SMAS 135
7.2.3 Computingmutualtangents 138
7.2.4 Generatingmusclefibres 141
7.3 Summaryandrecommendationforfurtherwork 142
7.3.1 Futurework 142
8 Conclusion: summary,futurework,possibilitiesandperspectives 147
8.1 Relationshipwithcomputerizedforensicfacialreconstruction 148
a Appendix 151
a.1 FourierAnalysis 151
a.1.1 Shifttheorem 152
a.1.2 Convolutiontheorem 152
a.1.3 Fouriertransformofdifferentialoperators 153
a.1.4 Planchereltheorem 153
a.1.5 Beppo-Levisemi-norminthreedimensions 154
a.1.6 Derivativereproducingproperty 154
a.1.7 SundryrelationshipsinvolvingtermsoftheBeppo-Levisemi-norm
oforder2 154
NOTATION
- Boldfaceletters,e.g. p,indicatemulti-componentormultidimensionalquantities.
Normalletters,e.g. p,indicatesingle-componentorscalarquantities.
- squarebraces,e.g. p[i],indicateonecomponentofamulti-componentormultidi-
mensionalquantity.
- subscripts, e.g. pi or pi, indicate a single element in a collection of single or
multi-componentquantities.
7
LIST OF TABLES
Table1.1 Facialanimationtechniques: easeofuseversusreusability 2
Table2.1 Comparison of the physically-based facial animation techniques
reviewedinSection2.1.4. 16
Table7.1 Runtimestatisticsformusclegenerationonalaptopcomputerwith
a 1.6GHz (single core) processor and 512Mb main memory. (SG:
straightestgeodesic) 146
Table7.2 Runtime statistics for muscle generation on a PC with a 3.2GHz
processor (single core) and 2Gb main memory. (SG: straightest
geodesic) 146
8
LIST OF FIGURES
Figure1.1 (a)Genericskullpackage(b)typicalheadmodel(c)genericskull
fittedtoheadmodel(d)SMAS-planeshowingandmutualtangents
andconvexhullsofmuscleattachmentregions(e)musclefibresas
boundary-valuestraightestgeodesics. 3
Figure2.1 Hypotheticalplotofthehumanemotionalresponsetoincreasing
degrees of human likeness of an entity. The plot shows a dip in
thelevelofhumancomforttriggeredbytheexcessiveadditionof
human-likefeatures. (Source: theWikimediaCommons.) 21
Figure3.1 Ninemajorbonesofthehumanskull. 28
Figure3.2 Thefrontalis,corrugatorSupercilli,ProcerusandOrbicularisOculi
muscles 33
Figure3.3 TheLevatorlabiisuperiorisalaequenasi,Levatorlabiisuperioris,
LevatoranguliorisandZygomaticusminormuscles. 35
Figure3.4 The Zygomaticus major, Incisivus labii superioris, Incisivus labii
inferiorisandOrbicularisorismuscles 36
Figure3.5 TheDepressorlabiiinferioris,DepressoranguliorisandMentalis
muscles. 38
Figure3.6 TheRisorius,Temporalis,BuccinatorandMassetermuscles. 39
Figure3.7 ThePlatysmamuscle. 40
Figure3.8 Types of smiles, identified by Rubin (1974). (a) Mona Lisa smile.
(b)Caninesmile. (c)Fulldenturesmile. 41
Figure3.9 Thefivelayersofthehumanface. (FromMendelson(2009),used
withpermissionofthecopyrightholder.) 42
Figure3.10 (a) Tree-like structure of retaining ligaments (From Mendelson
(2009)–usedwithpermissionofthecopyrightholder.) (b)Subcu-
taneousfatcompartmentsofthehumanface. (FromRohrichand
Pessa(2007)–usedwithpermissionofWoltersKluwerHealth.) 44
Figure3.11 RelativestrengthofattachmentsoftheSMAStothedermisatvari-
ouspartsoftheface. Darkerstipplesindicatestrongerattachments
while lighter stipples indicate weaker attachments. (From Keller
(1997).) 45
Figure3.12 Facial aesthetic units and subunits (based on Figures 1 and 2 of
Fattahi(2003)). 49
Figure4.1 A set of nine points, a = (0.5,0.5,0.0) , b = (5.0,0.5,1.5) , c =
(9.5,0.5,0.0), d=(0.5,5.0,−1.5), e=(5.0,5.0,0.0), f=(9.5,5.0,−1.5), g=
(0.5,9.5,0.0), h=(5.0,9.5,1.5), i=(9.5,9.5,0.0). 66
9
Figure4.2 Height field interpolating the set of nine points shown in Fig-
ure4.1. 67
Figure4.3 HeightfieldinterpolatingthesetofninepointsshowninFigure4.1,
with the gradient at point e constrained to 1.0 in the direction
(1,1). 69
Figure4.4 HeightfieldinterpolatingthesetofninepointsshowninFigure4.1,
with the gradients at point e constrained to 1.0 and 0.25 in the
directions(1,1)and(−1,1)respectively. 70
Figure4.5 (a) Two and (b) three ring neighborhood vertices around a point
of interest. (c) Projecting a point of interest p on a polygonal
meshtothepointp(cid:48) onthin-platespline,byinflatinganosculating
sphere. 71
Figure4.6 Fittingmongepatches(bluemini-grids)tothen-ringneighborhood
verticesofseveralpointsofinterest(redpins). 72
Figure4.7 Reconstructing the scalar field generated by (the equation of) a
circle,giveneightsurfacepointshavingzerofieldvalues(hollow
dots), using the (a) variational implicit surface technique. Offset
points (solid dots) are assigned field values of -1. (b) Hermite
formulation, showing the gradient vector and field values at the
surfacepoints. 75
Figure4.8 Cuttingplanethroughareconstructionofthescalarfieldgenerated
by the surface points and normals of a polyhedral torus using
the (a) variational implicit surface, and (b) Hermite method. The
smoothsurfaces(c)and(d),obtainedbyraycasting,arethezero
levelsetsoftherespectivescalarfields. 76
Figure4.9 Deformationofthecircleshownininsetanditsembeddinggrid,
duetotherelocationofthelandmarksaandb. 77
Figure4.10 Gradientandnormals(red)atfourlandmarks,alongthecircum-
ferenceofacircularstructure. 78
Figure4.11 Deformationproducedbythe45degreerotationofgradientsand
normalsatthefourstationarylandmarksinFigure4.10,usingthe
parameter values (a) above: m = 1.75 (b) below: m = 2, where
d=2(seeEquation4.9). 79
Figure4.12 Deformations based on three-dimensional thin-plate splines. (a)
undeformed,sphereofunitradius,havingsixlandmarks. (b)de-
formationofspherebasedondownwarddisplacementoflandmark
at the upper pole by 0.5 units. (c) deformation due to -45 degree
rotationoftriadofvectorsatlandmarksaroundthe“equator”of
the sphere. (d) deformation due to additional twist of one of the
triadofvectors(showninblue). 80
10