The Ontology of Digital Social Agents ANDREA L. GUZMAN BA, Truman State University MA, N

311 Pages·2015·1.27 MB·English

Imagining the Voice in the Machine:

The Ontology of Digital Social Agents

BY

ANDREA L. GUZMAN
B.A., Truman State University
M.A., Northern Illinois University

THESIS

Submitted as partial fulfillment of the requirements
for the degree of Doctor of Philosophy in Communication
in the Graduate College of the
University of Illinois at Chicago, 2015

Chicago, Illinois

Defense Committee:

Steve Jones, Chair and Advisor
Zizi Papacharissi
Adrienne Massanari
Clifford G. Christians, University of Illinois Urbana-Champaign
Jason Leigh, University of Hawai’i
This dissertation is dedicated to my husband who gave me the support and courage to
step on the train that first day of school and to keep moving forward every day since.

ii
ACKNOWLEDGMENTS

To my committee, thank you for giving your precious time to my scholarly development.
Zizi, thank you for helping me to understand that being successful is not about being perfect but
about being persistent. Adrienne, our discussions regarding navigating qualitative research have
been invaluable as have our chats regarding succeeding in academia as an emerging scholar.
Jason, you went above and beyond as an outside member and have taught me not only about AI
but also about working across disciplines. Cliff, thank you for sharing your vast knowledge and
for challenging me to think about the even bigger questions. Steve, I cannot list all of things you
have done for me as you have guided me through grad school. Above all, Steve, thank you for
being patient with me as I struggled, for continuing to push me, and for believing in me. All of
you have shown me that being a scholar is not a solitary pursuit but a path in which we all learn
from one another. I will do my best to pay forward everything you have done for me.
I also want to thank my peers and colleagues. Emilie, Adriane, Fede, Will, Jenny, and
Renee, thank you for your support. To Miao, I am glad to have made this journey with you.
Thank you, Catherine, for your help in reaching the “other side.” Sabrina, thank you for
supporting me on the bad days and helping me find the good days. Kelly, thank you for keeping
your door open. Christina and Mamie, your help is greatly appreciated.
To my participants: thank you for giving a few moments of your time to a stranger.
This dissertation would not have been possible without my family who supported me
emotionally, physically, and spiritually. It was hard not seeing you much, but knowing you were
pulling for me kept me going. Helga, I wish you were here to see this.

ALG

ii i
TABLE OF CONTENTS

CHAPTER PAGE

1. INTRODUCTION: DIGITAL SOCIAL AGENTS IN OUR WORLD ................... 1

2. BACKGROUND: DIGITAL SOCIAL AGENTS & THEIR
COMPLICATION OF COMMUNICATION .......................................................... 9
2.1 The Dual Nature of the Computer ................................................................ 9
2.2 Complicating Communication ..................................................................... 12
2.2.1 Challenging Technology’s Role in Communication Research ..................... 13
2.2.2 Challenging Conceptions of Mass and Interpersonal Communication ........ 15
2.2.3 Challenging the Definition of Communication ............................................ 20
2.3 Taking on the Challenge of Digital Social Agents ...................................... 22

3. LITERATURE REVIEW: COMING TO KNOW THE SELF AND OTHER
THROUGH COMMUNICATION .......................................................................... 24
3.1 The Meaning of Things ................................................................................ 26
3.2 The Meaning of People ................................................................................ 28
3.3 The Meaning of Digital Social Agents ........................................................ 30
3.4 A Sense of Presence ..................................................................................... 31
3.4.1 Presence, with Humans ................................................................................ 32
3.4.2 Cultural Presence, of Technology ................................................................ 35
3.4.3 Presence, of New Media .............................................................................. 37
3.4.4 The Presence of Digital Social Agents ........................................................ 43
3.5 Our Construction of Digital Social Agents .................................................. 45
3.5.1 Forming Impressions ................................................................................... 46
3.5.2 The Self and Others “in Everyday Life” ...................................................... 51
3.5.3 Seeing One Another ..................................................................................... 54
3.5.4 Impressions of Technology .......................................................................... 55
3.5.5 Impressions of Presence ............................................................................... 60
3.6 Giving Digital Social Agents a Voice .......................................................... 62
3.7 Investigating the Voice in the Machine ....................................................... 66

4. METHOD: OUR INTERPRETATIONS OF AGENTS .......................................... 70
4.1 A Qualitative Approach ............................................................................... 70
4.2 Qualitative Methods ..................................................................................... 72
4.3 Study Population .......................................................................................... 79
4.4 Observation and Interview Protocol ............................................................ 80
4.5 Documentation and Storage of Data ............................................................ 84
4.6 Coding and Analysis .................................................................................... 85
4.7 Ethics ............................................................................................................ 89
4.8 Terminating Data Collection ........................................................................ 92
4.9 Overview of Interview and Observational Data .......................................... 92

iv
TABLE OF CONTENTS (continued)

CHAPTER PAGE

PART I: VOICES IN AND OF THE MACHINE .............................................................. 97

5. SPEAKING WITH AND THROUGH THE PHONE ............................................. 98
5.1 The Voice Of the Machine ............................................................................ 99
5.2 The Voice In the Machine ............................................................................ 101
5.3 Grasping at the Disembodied ....................................................................... 104
5.4 Choosing Between Voices ........................................................................... 109
5.5 An Entity Unto Itself .................................................................................... 113

6. EPHEMERAL BEINGS AND LIFELESS THINGS .............................................. 114
6.1 Being-with .................................................................................................... 115
6.2 Unwelcome Presence ................................................................................... 116
6.3 Party Line ..................................................................................................... 117
6.4 Shifting Presence ......................................................................................... 121
6.5 Lifeless Tools ............................................................................................... 125
6.6 Social Things ............................................................................................... 126
6.7 Different States of Being ............................................................................. 130
6.8 Changing Experiences of Presence .............................................................. 132

7. THE IDENTITY OF DIGITAL VOICES ............................................................... 138
7.1 A Crowd of Voices ...................................................................................... 139
7.2 Competing Voices ........................................................................................ 140
7.3 A Known Voice ........................................................................................... 142
7.4 Merging Voices ............................................................................................ 146

PART II: THE ARTIFICIAL FRONT ............................................................................... 149

8. WHAT WE HEAR IN AGENT VOICES ............................................................... 152
8.1 Demographics of Digital Voices .................................................................. 152
8.1.1 Gender: Pervasive Female Voices ............................................................... 153
8.1.2 Nationality & Race: White, American Voices Speaking “Good English” ... 157
8.2 Human Voices .............................................................................................. 160
8.3 Artificial Voices ........................................................................................... 161
8.4 Converging Voices ....................................................................................... 163
8.5 Vocal Mash-ups ........................................................................................... 165
8.6 Schizophrenic Voices ................................................................................... 167

9. WHAT WE SEE WHEN WE HEAR AGENTS ..................................................... 169
9.1 White Women .............................................................................................. 170
9.2 Woman on the Line ...................................................................................... 172
9.3 Humanoid Robots ........................................................................................ 174
v
TABLE OF CONTENTS (continued)

CHAPTER PAGE

9.4 Talking Screens ............................................................................................ 177
9.5 The Unimagined ........................................................................................... 178
9.6 Dueling Senses, Dueling Images, Dueling Natures ..................................... 179

10. WHAT WE THINK OF AGENTS .......................................................................... 185
10.1 Alternative Tools ........................................................................................ 185
10.2 Vague Voices ............................................................................................... 187
10.3 Social Interactions ........................................................................................ 189
10.4 Personal Assistants ....................................................................................... 190
10.5 Tools and Beings at Our Service ................................................................. 192
10.6 Congruence and Contradiction .................................................................... 194
10.7 Rise of the Machine ..................................................................................... 196

PART III: OUR LIVES WITH DIGITAL SOCIAL AGENTS .......................................... 199

11. THE AWESOME WEIRDNESS OF SPEAKING WITH MACHINES ................ 200
11.1 Awesome Impressions ................................................................................. 200
11.2 Weird Encounters (of the agent kind) .......................................................... 201
11.3 Awesome Weirdness .................................................................................... 204
11.4 Typing is for Machines ................................................................................ 205
11.4.1 Old-Fashioned Googling .............................................................................. 207
11.4.2 High-Tech Gimmick .................................................................................... 209
11.4.3 Learning to Speak with Machines ................................................................ 210
11.4.4 Life On-The-Go ........................................................................................... 213
11.4.5 Convenient Laziness .................................................................................... 214
11.5 Meeting of the (Artificial & Biological) Minds ........................................... 217
11.5.1 Miscommunication ...................................................................................... 218
11.5.2 Empty Dialogue ........................................................................................... 220
11.5.3 Effective Communication ............................................................................ 221
11.5.4 Artificial Communication ............................................................................ 223
11.6 Shifting Senses of Communication .............................................................. 224

12. ONTOLOGICAL UPHEAVAL .............................................................................. 226
12.1 Confusion and Instability ............................................................................. 227
12.2 Our Sense of Things ..................................................................................... 229
12.3 Our Sense of Self ......................................................................................... 237
12.4 Humans: The Machine-Speaking Animal .................................................... 242

13. CONCLUSION: NEW WAYS FORWARD ........................................................... 249
13.1 A Multitude of Images Within the Machine ................................................ 250
13.2 A New Understanding of the Artificial ........................................................ 254
v i
TABLE OF CONTENTS (continued)

CHAPTER PAGE

13.2.1 Establishing the Artificial Front ................................................................... 254
13.2.2 An Awareness of Presence ........................................................................... 258
13.2.3 Coming into Being Through Communication ............................................. 260
13.2.4 The Importance of Talking to Humans ........................................................ 262
13.3 Limitations ................................................................................................... 264
13.4 A New Agenda for Studying Agents and HMC .......................................... 266

APPENDIX .............................................................................................................. 269

REFERENCES ........................................................................................................ 276

VITA ....................................................................................................................... 299

vi i
LIST OF ABBREVIATIONS

AI Artificial Intelligence
CMC Computer-Mediated Communication
CS Computer Science
HCI Human-Computer Interaction
HMC Human-Machine Communication
ICD Informed Consent Document
ICT Information and Communication Technology

vi ii
SUMMARY

In 2011, Apple introduced Siri, a voice-based, artificial intelligence application that
functions as a type of virtual “assistant.” Although digital and automated voices already had been
part of people’s lives in Western cultures in the form of GPS systems, phone answering services,
and public address systems, Siri was the first program available to the general public that
combined natural language vocal capabilities with AI. Since then, other major technology
companies have developed their version of a virtual assistant. These programs along with
numerous other applications developed for a variety of devices, including emerging home-based
systems, are part of a class of technologies that I label “digital social agents.” The proliferation
of digital social agents and our interactions with them are part of a larger cultural transition in
which artificial entities play a greater role in our lives. We now regularly engage in Human-
Machine Communication, or HMC, with numerous devices and programs. In this dissertation, I
focus on our conceptualizations of our new digital interlocutors, specifically agents. The
overarching question guiding this research was, “What, or who, are voice-based digital social
agents to users and non-users?” This dissertation explores the ontology of digital social agents.
My exploration of digital social agents proceeds from the epistemological position that
our understanding of our world, including objects and people, is formed in and through
communication. Consistent with this perspective, I adopted a qualitative approach and sought to
understand the ontology of digital social agents from the perspective of people and their lived
experiences. I observed people’s public interactions with digital social agents and conducted 51
field interviews, following the specific method of active interviewing (Holstein & Gubrium,
1997), with people about their use and understanding of digital social agents and themselves.
Concurrent with my observations and interviews, I qualitatively coded and analyzed the data.
ix
SUMMARY (continued)
From my findings emerge not only answers to my larger research questions but also new insights
into communication and life inside a culture populated by biological and artificial entities.
Our understanding of digital social agents is highly complex and is formed through our
direct and indirect interaction with agents. We assess agents through a concept I refer to as the
artificial front, an extension of the personal front (Goffman, 1959). We take into account their
appearance, which largely consists of how they present themselves to us through their voice and
their manner, how they communicate with us, and how they perform specified tasks. Many
people do not have a single image of who or what the agent may be but multiple
conceptualizations depending on whether the person is focusing on an agent’s voice or the agent
in its totality. Participants perceive both human and machine-like characteristics in agents, and
people’s interpretations of these characteristics both emerge from and guide their interaction with
the agent. People also take into account how they have interacted with machines previously,
usually through typing in requests, when deciding whether to engage with agents and the roles
that agents perform in their lives. Based on their interactions with agents, including the
program’s voice and how it communicates, some participants perceive themselves as being in the
presence of a life-like entity while communicating with the agent.
But agents are more than digital programs that may or may not offer us a better way of
controlling our digital devices, usually mobile, or finding information. They are a new way of
“sensing” computers and machines. We speak with and hear agents (oral-aural communication)
instead of typing with them (haptic-visual communication). This shift in the senses we use to
interact with agents has brought with it a shift in our understanding of what computers and
machines are to us. Individually and collectively we often think of agents in contradictory terms
x