Playing With Sound: A Theory of Interacting With Sound and Music in Video Games PDF
Preview Playing With Sound: A Theory of Interacting With Sound and Music in Video Games
Playing with Sound Playing with Sound A Theory of Interacting with Sound and Music in Video Games Karen Collins The MIT Press Cambridge, Massachusetts London, England © 2013 Massachusetts Institute of Technology All rights reserved. No part of this book may be reproduced in any form by any electronic or mechanical means (including photocopying, recording, or information storage and retrieval) without permission in writing from the publisher. MIT Press books may be purchased at special quantity discounts for business or sales promotional use. For information, please email [email protected] or write to Special Sales Department, The MIT Press, 55 Hayward Street, Cambridge, MA 02142. This book was set in Stone Serif and Stone Sans by Toppan Best-set Premedia Limited, Hong Kong. Printed and bound in the United States of America. Library of Congress Cataloging-in-Publication Data Collins, Karen, 1973– Playing with sound : a theory of interacting with sound and music in video games / Karen Collins. pages cm Includes bibliographical references and index. ISBN 978-0-262-01867-8 (hardcover : alk. paper) 1. Interactive multimedia. 2. Video games. I. Title. QA76.76.I59C653 2013 006.7— dc23 2012025349 10 9 8 7 6 5 4 3 2 1 Contents Preface: Game Sound Gets Heard vii Acknowledgments xi Introduction 1 How Is Interacting w ith Sound Different from Listening t o Sound? 2 The Sound of Music: Musical Sound 3 Interacting with and Listening To 4 Game Players: An Interactive Audience 13 Game Players: An Embodied Cognition Approach to Audience 15 1 Interacting with Sound: A Theory of Action, Image, and Sound 19 Schizophonia: Disembodied Sound? 23 Synchresis: Integrating Sound and Image 26 Kinesonic Synchresis: The Event-Driven Nature of Interactive Sound 32 Kinesonic Congruence and the Player 35 2 Being in the Game: A Sonic Approach 39 Extension and Incorporation 41 Self-Produced Sound 43 Sonic Game Space: Point of Audition in Games 44 Spatial Sonic Embodiment in the Game 56 3 Sound at the Borders: Enacting Game Sound 59 Posing and Playing 60 Voice and Role-Play in Games 68 Alternate-Reality Games 82 New Sonic Boundaries: Identifi cation, Performance, and Cocreativity 87 4 Embodying Game Sound in Performance: The Real and the Virtual 89 Performing Music in Games 92 Performing to Music in Games 96 vi Contents Performing Game Music 105 Creating Music from the Game 111 Interacting with the Game as Instrument 119 5 The Second Life of Game Sound: Playing with the Game 121 Sonic Modifi cation and Player-Generated Content 123 Modding Game Sound 126 Art Mods 134 Player-Generated Content: A Fourth Wall of Sound 138 Conclusions 143 Future Directions in Interactive Sound Studies 147 Notes 149 References 155 Audiovisual References 173 Index 179 Preface: Game Sound Gets Heard A few years ago, I began my previous book, Game Sound: An Introduction to the History, Theory, and Practice of Video Game Music and Sound Design (MIT Press, 2008), by discussing the importance of video games and of game sound in particular. Today, it is taken for granted (at least in some circles) that the study of sound in games is a valid pursuit. Courses are taught on campuses everywhere, and research is attracting wider attention. But in addition to recent changes in the ways in which video games are perceived publicly and in the academy, there have been a variety of changes in the industry, in the technology, within game studies, and in my own personal thinking about game sound that have inspired me to write this follow-up book. The video game industry has grown tremendously in the past half- decade, and most major game development companies have expanded their sound departments. The roles played by sound professionals have become increasingly specialized as production becomes more complex for “ triple A” games— large-scale, epic games with multimillion dollar budgets. But another significant development in the industry is the rise of indepen- dent companies, particularly new game companies that have flourished in the social media space. Zynga ’ s FarmVille game, for example, managed to attract over 80 million active monthly Facebook users at its peak. Such games have dominated the social networking world and become a ubiqui- tous form of casual gaming. 1 Multiplayer online gaming has expanded as Internet speeds have improved and social networking and smart phones (notably Apple ’ s iPhone) have gained popularity. Independent game devel- opers from all over the world have found a foothold in the industry by creating game applications ( “ apps ” ). The accessibility and affordability of apps has resulted in the growing popularity of casual gaming in general — on smart phones, tablets, netbooks, and other portable devices. To a signifi- cant extent, the success of these devices has depended on the availability viii Preface of free or cheap game apps. Social networking and mobile media have had interesting consequences for game sound, since both forms of casual games are often played with the sound turned off. The eventual outcome of this fact for game sound practitioners remains to be seen, but the history of games is repeating itself once again. Small developers have a space where they can sell and share their games, and the “ one-man band” game devel- oper has returned. Technological changes in the game industry in the past few years have also affected the development of game sound. Audio middleware engines such as Wwise and Fmod enable practitioners to demonstrate the interac- tive elements of their sound without having to program code. 2 New game sound software such as PsaiCore is opening up audio artificial intelligence and computer-aided approaches to composition in games. As artificial intel- ligence gains ground in the game development community (and as the predictability and planning of narrative and action becomes more diffi- cult), designers are returning to sound synthesis as they explore new methods in procedural generation to be more responsive to the player ’ s actions. Spatial sound has also become more important to console play as home surround-sound systems have become increasingly affordable. And more important than ever before is the increased role that gesture and the player ’ s body play in gaming. With the massive success of the Wii as well as the introduction of Microsoft ’ s Kinect and Sony ’ s Move, gaming is no longer about sitting relatively still in front of a screen but now incorporates whole-body interaction with the game. This gestural involvement is a recur- ring theme in this book as I explore the implications for the player’ s embodied experience of game sound. As the popularity of games continues to grow, scholars have entered game studies from a wide variety of disciplines, which has resulted in interest- ing interdisciplinary conferences where computer scientists both clash and collaborate with cultural theorists. The study of game sound has seen remarkable growth in just a few years as the literature in this area has increased, the scope of research has widened beyond what was previously available, and new ideas and perspectives have been brought in from dif- ferent disciplines. With this rise in scholarly interest, game sound design and composition programs and courses have formed in universities and technical colleges around the world, with a need for literature to support these courses. Finally, my own theoretical perspective has changed during my time at the University of Waterloo, a university known for its strong computer science and engineering programs. I moved out of a music department Preface ix and into a digital arts program where I now teach games and sound design. Along with my collaborative work with people from the humanities, social sciences, and natural sciences at Waterloo and beyond, this disciplinary shift has altered my own thinking about interactivity and sound through my exploration of less conventional video games (such as slot machines). One of these research collaborations resulted in several trips to Japan, where I wandered the otaku alleys of Akihabara and Harajuku and became fascinated with the role that fandom plays in Japanese video game con- sumption. This helped develop my interest in the player ’ s experience of games. All of these changes have led me to develop a new approach to thinking about interactive sound that I present here and that in some ways repre- sents a shift from my previous thinking. I still integrate recent research from music, film, and cultural studies, but I expand this disciplinary back- ground to include more work from computer science and psychology. In short, I build on my previous thinking by offering a companion book to Game Sound , which focused on the creative, production side of game sound. In this book, I seek to understand games not as texts but as sites of participation and practice where players construct meanings. Here, then, I explore game sound from the other side of the console, focusing on the player ’ s experience of sound.
The list of books you might like
