3D Storyboarding for Modern Animation

162 Pages·2017·13.95 MB·English

3
D STORYBOARDING FOR MODERN ANIMATION
alexandros gouvatsos
Doctor of Engineering in Digital Media
Media School
Bournemouth University
Centre for Digital Entertainment
University of Bath
2017
September
This copy of the thesis has been supplied on condition that anyone who
consults it is understood to recognise that its copyright rests with its author
and due acknowledgement must always be made of the use of any material
contained in, or derived from, this thesis.
Alexandros Gouvatsos: 3D Storyboarding for Modern Animation
2017
© September
supervisors
:
Dr. Zhidong Xiao
Prof. Jian J. Zhang
Jerry Hibbert
Neil Marsden
2
ABSTRACT
Animation is now a classic medium that has been practiced for over a cen-
tury. While Disney arguably made it mainstream with some hand-drawn
3
classics, today’s industry is focused on Three-Dimensional ( D) animation.
3
In modern D animation productions, there have been significant leaps in
terms of optimising, automating and removing manual tasks. This has al-
lowed the artistic vision to be realised within time and budget and empow-
ered artists to do things that in the past would be technically more difficult.
However, most existing research is focused on specific tasks or processes
rather than the pipeline itself. Moreover, it is mostly focused on elements of
the animation production phase, such as modelling, animating and render-
ing.Asaresult,pre-productionpartslikestoryboardingarestilldoneinthe
traditional way, often drawn by hand. Because of this disparity between the
3
old and the new, the transition from storyboarding to D is prone to errors.
3
Dstoryboardingisanattempttoadaptthepre-productionphaseofmod-
ern animation productions. By allowing storyboard artists access to simple
3
but scale-accurate D models early on, drawing times as well as transi-
tion times between pre-production and production can be reduced. How-
3
ever, D storyboarding comes with its own shortcomings. By analysing ex-
isting pipelines, points of potential improvement are identified. Motivating
research from these points, alternative workflows, automated methods and
3
novel ideas that can be combined to make D animation pipelines more
efficient are presented. The research detailed in this thesis focuses on the
area between pre-production and production. A pipeline is presented that
consists of a portfolio of projects that aim to:
• Generate place-holder character assets from a drawn character line-up
• Create project files with scene and shot breakdowns using screenplays
3
• Empower non-experts to pose D characters using Microsoft Kinect
3 2
• Pose D assets automatically by using D drawings as input
3
CONTENTS
i research background 18
1 introduction 19
11 19
. Hibbert Ralph Animation . . . . . . . . . . . . . . . . . . . . . .
12 20
. Research focus . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13 21
. Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
14 21
. Research contributions . . . . . . . . . . . . . . . . . . . . . . . .
15 22
. Thesis structure . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2 digital media in 3d animation pre production 24
-
21 24
. An analysis of animation as a medium . . . . . . . . . . . . . .
22
. Animation as media, animation in media and media in ani-
27
mation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
23 29
. Pipeline elements . . . . . . . . . . . . . . . . . . . . . . . . . . .
24 32
. Breakdown of different media used in pre-production . . . . .
25 36
. Animation producers as consumers of media . . . . . . . . . . .
26 39
. Remediation of the pre-production phase . . . . . . . . . . . . .
3 storyboarding in 3d 42
31 3 42
. Why storyboard in D . . . . . . . . . . . . . . . . . . . . . . . .
32 3 43
. D storyboarding in theory . . . . . . . . . . . . . . . . . . . . .
33 3 44
. D storyboarding in practice . . . . . . . . . . . . . . . . . . . .
331 47
. . Storyboarding . . . . . . . . . . . . . . . . . . . . . . . . .
332 47
. . Layout . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
333 48
. . The HRA pipeline . . . . . . . . . . . . . . . . . . . . . .
34 48
. Moving forward . . . . . . . . . . . . . . . . . . . . . . . . . . . .
ii projects portfolio 51
4 automatic creation of place holder assets 53
-
41 53
. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
42 54
. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
43 55
. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
44 58
. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4
45 58
. Discussion & future work . . . . . . . . . . . . . . . . . . . . . .
5 screenplay analysis system for automatic asset import 65
51 65
. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
52 66
. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
53 68
. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
531 68
. . Parsing the screenplay . . . . . . . . . . . . . . . . . . . .
532 69
. . Linking the information . . . . . . . . . . . . . . . . . . .
533 70
. . Generating Redboard project file . . . . . . . . . . . . . .
54 71
. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
55 73
. Discussion & future work . . . . . . . . . . . . . . . . . . . . . .
6 integration of placeholder creation and screenplay
analysis 75
61 75
. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
62 75
. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
63 76
. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
64 77
. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
65 77
. Discussion & future work . . . . . . . . . . . . . . . . . . . . . .
7 cheap motion capture devices for humanoid posing in
-
pre production 80
-
71 80
. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
72 80
. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
73 83
. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
731 84
. . Mapping of storyboard timeline to Maya timeline . . . .
732 3 85
. . Mapping skeleton capture data to D rigged assets . . .
74 89
. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
75 91
. Discussion & future work . . . . . . . . . . . . . . . . . . . . . .
8 automatic 3d posing from 2d hand drawn sketches 96
-
81 96
. Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
82 97
. Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
821 98
. . Related work . . . . . . . . . . . . . . . . . . . . . . . . .
822 104
. . State-of-the-Art work . . . . . . . . . . . . . . . . . . . . .
823 109
. . Proposed Approach . . . . . . . . . . . . . . . . . . . . .
83 109
. Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
831 110
. . Overview . . . . . . . . . . . . . . . . . . . . . . . . . . .
832 112
. . PARAC-LOAPSO . . . . . . . . . . . . . . . . . . . . . . .
833 119
. . Comparing Drawings to Renders. . . . . . . . . . . . . .
5
84 122
. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
85 129
. Discussion & future work . . . . . . . . . . . . . . . . . . . . . .
iii concluding remarks 135
9 discussion future work 136
&
10 conclusion 144
references 147
glossary 162
6
LIST OF FIGURES
1
Figure Exampleflowofatraditionalpipelinewithitsdistinct
29
stages. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2
Figure Exampleofpre-vis,withoverlayofthecorresponding
31
storyboard panel at the bottom right.. . . . . . . . . . .
3
Figure Example screenplay from the movie ‘Wolf of Wall
2013 32
Street’ (Winter and Belfort ). . . . . . . . . . . . . .
4 2009 33
Figure Example storyboard (Glebas ). . . . . . . . . . . . .
5 43
Figure Screenshot of Redboard. . . . . . . . . . . . . . . . . . .
6 3 2
Figure Example of D storyboarding, with D drawing on
3 3 45
top of a D environment. The D models are unposed.
7
Figure Efficient storyboarding workflow example from Tree
Fu Tom (FremantleMedia, CBeebies, Blue-Zoo Pro-
2012 50
ductions ). (©BBC MMXVI) . . . . . . . . . . . . . .
8
Figure Example of input character drawings (© Hibbert
55
Ralph Animation).. . . . . . . . . . . . . . . . . . . . . .
9 3
Figure Automaticallygenerated DpegsasseeninAutodesk
55
Maya. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
10
Figure Examples of automatically generated character ab-
60
stractions. . . . . . . . . . . . . . . . . . . . . . . . . . . .
11 61
Figure Screenshot of the Peg Creator interface. . . . . . . . . .
12 1
Figure Step — Segmenting the character from the line-up.
2 3
Step — Trimming it based on the feet. Step — Av-
eragingthecoloursintostripestocreatea‘mosaic’ef-
4
fect. Step — Generating the rectangular cuboid and
61
adding a plane on top to convey orientation. . . . . . .
13
Figure The same character drawn differently can result in a
different bounding box that may affect how the char-
62
acter’s width is estimated. . . . . . . . . . . . . . . . . .
7
14
Figure Side by side comparison of the original character
drawing (left-most) and different mosaic stripe thick-
10 40 100
ness parameters: , and respectively from left
62
to right. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
15 67
Figure Screenshot of Final Draft software. . . . . . . . . . . . .
16
Figure Automatically populated Redboard project from ‘Q
5 2013 71
Pootle ’ (Blue-Zoo Productions ) screenplay. . . .
17 72
Figure Automatic shot breakdown as seen in Redboard. . . . .
18
Figure StructureofintegrationbetweenthePegCreationand
76
the Screenplay Analysis software. . . . . . . . . . . . . .
19
Figure A screenshot of the Peg Creation software, where
dropdown boxes allow the artist to choose the char-
79
acter’s name. . . . . . . . . . . . . . . . . . . . . . . . . .
20
Figure ThecreatedsystemrunningwithinMaya,trackingthe
pose of an actor while viewing a series of storyboard
83
panels. . . . . . . . . . . . . . . . . . . . . . . . . . . . .
21
Figure ThemainUserInterface(UI)ofthesystem,asanactor
is posing in an office environment, according to the
85
selected storyboard panel. . . . . . . . . . . . . . . . . .
22 25
Figure For frames per second, the first panel corresponds
86
to the first frame in Maya’s timeline. . . . . . . . . . . .
23 25
Figure For frames per second, the second panel corre-
25 86
sponds to the th frame in Maya’s timeline. . . . . . .
24
Figure The skeletal structure of the Kinect data. There are
20
joints on the human body tracked from the Kinect
2017 87
sensor (Microsoft ). . . . . . . . . . . . . . . . . . .
25 88
Figure KinectdataneedstobetransformedtobeusedinMaya.
26 3
Figure The skeletal structure of an example D rigged T-
posed asset in Maya. Note that there is not a one-
to-one correspondence to the skeletal structure of the
24 90
Kinect data shown in Figure . . . . . . . . . . . . . . .
8
27
Figure A screenshot during practice for the live demo se-
quence. The top left window shows the actor acting
out the pose in front of the Kinect, with the joints
overlaid on top. The bottom left window shows the
storyboardpaneltheactoriscurrentlyactingout.The
Autodesk Maya window is updated whenever the ac-
90
tor captures a new pose. . . . . . . . . . . . . . . . . . .
28 3
Figure An example of an actor posing a D character in
92
Maya, according to the selected storyboard panel. . . .
29
Figure An example of a character in a Maya scene posed us-
ing the proposed pose capture system. The actor is in
this case controlling the computer and thus does not
93
appear in front of the camera. . . . . . . . . . . . . . . .
30
Figure Anexampleoftheforward-backwardambiguityissue
3
in inferred D pose of a lamp when not using the
104
internal lines for additional information. . . . . . . . .
31
Figure Character poses created by sketching lines of action
2013 105
(Guay et al. ). . . . . . . . . . . . . . . . . . . . . . .
32
Figure Bone rotations are constrained to the viewing plane.
The bones are parameterised as a single axis-angle
component. The axis is the camera’s viewing direc-
tion projected onto the floor plane. From left to right
is the initial pose, a side view with the LOA, the re-
sult after optimisation, and a frontal view of the final
2013 106
pose (Guay et al. ). . . . . . . . . . . . . . . . . . . .
33
Figure Theposedescriptorconsistsofin-the-imageplanean-
2009 108
gles for every limb segment (Jain et al. ). . . . . . .
34
Figure Characterposesthatmatchthehandanimationmuch
more closely than the mocap animation. ‘Our result’
2012 108
refers to the work by Jain et al. ( ). . . . . . . . . . .
35 2 110
Figure Different descriptors for comparing poses in D. . . . .
36
Figure Flow chart of original Particle Swarm Optimisation
113
algorithm. . . . . . . . . . . . . . . . . . . . . . . . . . .
9
37
Figure The ring local topology of a particle swarm (on the
left) allows each particle to communicate with two
other particles, while the global topology (on the
right) allows particles to exchange information with
116
all other particles. . . . . . . . . . . . . . . . . . . . . . .
38
Figure Exampleofsilhouettecomparisontocalculatesimilar-
itybetweentwoposesXandY.Greencoloursignifies
the silhouette of pose X, red colour the silhouette of
pose Y and blue colour the overlap between the two
122
poses. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
39 3
Figure The D assets of the lamp, horse and human shown
with their joint hierarchies and controllers in Au-
123
todesk Maya. . . . . . . . . . . . . . . . . . . . . . . . . .
40
Figure Sidebysidecomparisonofthedrawingsandlesssuc-
3 123
cessfully estimated D poses (bottom). . . . . . . . . . .
41
Figure Sidebysidecomparisonofthedrawings(top)andthe
3 125
estimated D pose (bottom) for the horse model. . . . .
42
Figure Side by side comparison of a drawing (left) and the
3
estimated D pose (right) for the human model. The
3 126
estimated D pose is close to the drawing. . . . . . . .
43
Figure Sidebysidecomparisonofthedrawings(top)andthe
3 126
estimated D pose (bottom) for the lamp model. . . . .
44
Figure Frominitialstoryboardtopre-visualisation.Thelamp
in these three shots is automatically posed using the
storyboarddrawing.Ontheleftcolumn,drawingsare
3
overlaid on top of the D scene with the assets un-
posed.Ontherightcolumn,thelamphasbeenposed
127
automatically using the proposed method. . . . . . . .
45
Figure Side by side comparison of a sequence of drawings
(top) with temporal relationships and a sequence
3
from the estimated D poses (bottom) for the lamp
model. Lighter opacity and colours signify frames
earlier in the sequence while darker opacity and
128
colours signify frames later in the sequence. . . . . . .
10