Rei Toei lives!

229 Pages·2013·4.04 MB·English

by  Thomas Conner

Rei Toei lives!:

Hatsune Miku and the Design of the Virtual Pop Star

BY

THOMAS CONNER
B.A., University of Oklahoma, 1993

THESIS

Submitted as partial fulfillment of the requirements
for the degree of Master of Arts in Communication
in the Graduate College of the
University of Illinois at Chicago, 2013

Chicago, Illinois

Defense committee:

Steve Jones, Chair
Adrienne Massanari
Jason Leigh, Computer Science
This thesis would still be a virtual experience in my head without the consistent
encouragement support of my partner, Daniel, whose stern but loving tone when calling
me by an honorific I’ve not yet earned has pushed me on toward that ultimate goal.
ii
ACKNOWLEDGMENTS

All hail my cool and calm thesis advisor, Steve Jones, ever the font of
enlightenment and a refreshingly chill guide through the academic labyrinth. Deep thanks
also to my ace thesis committee, Adrienne Massanari and Jason Leigh, for their
considerable advice, wisdom, and patience.

Simply landing in the graduate communication program at the Univ. of Illinois-
Chicago was one of life’s greatest strokes of fortune thus far. Zizi Papacharissi leads a
department as intellectually vibrant as her own personality, and the energy crackling
through my feisty cohort powered the kind of nurturing, mind-expanding, and fun
environment every scholar dreams of. I never would have plugged into it without Kevin
Barnhurst’s boundless encouragement, and his graceful example of maintaining the
strength of one’s heart while working out one’s head will remain a valuable life lesson.

Additional thanks to the following: all sources contacted for interviews (my
research on this topic will continue, so those who didn’t get back to me still can!); UCSD
professor Tara Knight; Jamie Murnane, for killing an afternoon with Miku and me; all
those who kept brave faces while I perfected my elevator explanation of this
phenomenon; my mother and her beau, who dutifully sent me newspaper clippings about
robots and holograms, some of which were indeed instrumental; and the supportive
community that is the annual VisCom conference on visual communication — all my
conference experiences since have paled in comparison.

iii
For this writer, background music is essential to reining in my limbic system and
achieving something like Csikszentmihalyi’s “flow” state, and I simply must
acknowledge and thank several such sources that aided in that transition into long and,
due to their input, productive stretches of writing: a Spotify user named Keats
Handwriting, who created a fantastic playlist titled “Instrumental Music to Work To”
that, at completion of this thesis, contains 2,864 tracks of just that; plus the highly
recommended web sites Coffitivity (coffitivity.com, streaming ambient recordings of
coffee shops and similar environments in which mobile workers like me have gotten used
to producing) and especially
TABLE OF CONTENTS
CHAPTER PAGE
I. Introduction................................................................................1
A. Virtual pop stars.....................................................................1
B. Relevance and scope of study ................................................11

II. Literature review .......................................................................13
A. Histories, context and connections ........................................13
1. Dimensions of virtuality...................................................13
2. A history of holography ...................................................15
3. Simulated holography and robot actors ..........................20
4. A history of virtual pop stars ...........................................26
5. Vocaloid software ............................................................53
B. Applicable theory...................................................................69
1. Communication theory.....................................................70
2. Robotics design ................................................................82
3. Performance studies.........................................................99
4. Mixed reality ....................................................................108

III. Method ........................................................................................118
A. Mode of inquiry .....................................................................118
B. Data collection and coding.....................................................118
C. Grounded theory analysis ......................................................122

IV. Results and discussion ...............................................................124
A. The canned and the canny......................................................125
1. Stylized vs. photo-real animation.....................................126
2. Licensing vs. produsage...................................................134
3. Digitizing DNA ................................................................137
B. Invisibility and interaction .....................................................142
1. Magic and the uncanny valley .........................................142
2. Still wanted: humans........................................................150
3. Hybrid performance and interaction ...............................156
C. Evaluating the future of eversion...........................................168
1. Digital re/animation.........................................................168
2. Projection systems ...........................................................171
3. Artificial intelligence .......................................................175

V. Conclusion ..................................................................................177
A. Dimensions ............................................................................177
B. Dualisms ................................................................................179
C. Decisions................................................................................181
D. Data ........................................................................................184
E. Limitations and future research .............................................186

v
TABLE OF CONTENTS (Continued)

Cited Literature ...........................................................................................189

Appendix.......................................................................................................216

Vita ................................................................................................................220
vi
LIST OF FIGURES

FIGURE PAGE

1. Hatsune Miku.....................................................................................7
2. (a) 2.0Pac illusion design, (b) original Pepper’s ghost ......................21
3. “Let’s All Sing With the Chipmunks” (a) 1959, (b) 1961.................29
4. The Vocaloid circle of life .................................................................63
5. Steuer’s Media Technologies Classifications (with Miku)................78
6. Mori’s uncanny valley .......................................................................85
7. Auslander’s model of pop music performance ..................................103
8. Milgram & Colquhoun’s Definition of Mixed Reality scale .............110
9. Lentini’s expanding screen model .....................................................114
10. Synthesized Reality Productions’ mascot, Yume ..............................127
11. “The Blue Umbrella” .........................................................................144
vii
SUMMARY
Cyberspace is everting, and some of the first digital figures to recolonize physical
space are virtual pop stars — non-corporeal characters presented as singular musical
performers with the appearance of agency — projected as three-dimensional performance
simulations within existing concert venues. In America, these figures thus far have been
actualized as resurrections of rappers, from the Tupac “hologram” in 2012 to lifelike new
performances by the late Eazy-E and ODB in 2013. In Japan, these actualized figures are
primarily stylized animations of characters created around a vocal-synthesis computer
application, the most popular being digital software agent Hatsune Miku.

This study examines the creation and design of virtual pop star presentations,
determining what theories from visual communication, robotics design, performance
studies, and augmented/mixed realities are at play and connecting them within this new
phenomenon. Current virtual pop star presentations remain mere simulations of actual
holography, though their performances and reception have framed them as such; thus,
this study begins with a history of holography and the relation of that technique to the
emerging performance technologies, as well as a relational history of previous virtual pop
stars, from Alvin & the Chipmunks to Gorillaz. Qualitative interviews with technicians,
artists, and executives illustrate designers’ creative motives, theoretical negotiations, and
the presentations’ performance outcomes.

Findings indicate that these current virtual pop stars — whether presented as
stylized animation in Asia or photo-realistic images in America, each situated on either
viii
SUMMARY (continued)

side of the uncanny valley — are transitional figures, bridging our experience with the
normative behavior of human performers (who thus far remain as the characters’ digitally
translated antecedents) toward a future co-starring more vivid, interactive, and possibly
even intrusive digital agents and avatars. The continuing development of digital
projection technology in this performance field heralds particular doom for the hegemony
of the screen in mediating human-computer interaction. As digital presentations step off a
2D plane and into the 3D world, communication presence is complicated, the very
concept of virtuality begins to dissolve, and traditional performative tasks are distributed
across a wider network of non-performers.
ix
I. INTRODUCTION

She’s not there
But let me tell you ’bout the way she looked
The way she’d act and the colour of her hair …
— The Zombies, “She’s Not There”

A. Virtual pop stars
On April 15, 2012, the rapper Tupac Shakur (2Pac) appeared on stage at the
Coachella Valley Music and Arts Annual Festival in Indio, Calif. Shirtless but wearing
his usual jewelry and Timberland boots, and sporting his signature tattoos, Shakur
shouted, “What’s up, Coachella?!” before launching into two of his classic hip-hop
singles, “Hail Mary” and “2 of Amerikaz Most Wanted” (Kaufman, 2012). Shakur’s
performance, accompanied by a recorded backing track before a crowd of nearly 100,000
(Kaufman, 2012) and sharing the headlining slot by fellow rappers Dr. Dre and Snoop
Dogg, was a surprise for most festivalgoers (Goodale, 2012) for many reasons — not
least of which was the fact that Shakur had been shot and killed in 1996.
This resurrection was the result of new media technologies replicating a 150-year-
old theater illusion. The new, animated Shakur had been created and designed by digital
visual effects specialists and implemented at the festival by collaborating production
companies. From the point of view of the crowd, the projected two-dimensional animated
image appeared to be a three-dimensional person standing between Dr. Dre and Snoop
Dogg.
News and video of the “hologram” performance became an Internet sensation
immediately following its first presentation (the Coachella festival repeats its
programming during the following weekend, and the Shakur illusion was presented a
1