ebook img

Soma: live performance where congruent musical, visual, and proprioceptive stimuli fuse to form a PDF

245 Pages·2011·7.62 MB·English
by  BergstromI.
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview Soma: live performance where congruent musical, visual, and proprioceptive stimuli fuse to form a

Soma: live performance where congruent musical, visual, and proprioceptive stimuli fuse to form a combined aesthetic narrative Ilias Bergstrom UCL A thesis submitted for the degree of Doctor of Philosophy 2010 1 I, Ilias Bergstrom, confirm that the work presented in this thesis is my own. Where information has been derived from other sources, I confirm that this has been indicated in the thesis. 2 Abstract Artists and scientists have long had an interest in the relationship between music and visual art. Today, many occupy themselves with correlated animation and music, called ‗visual music‘. Established tools and paradigms for performing live visual music however, have several limitations: Virtually no user interface exists, with an expressivity comparable to live musical performance. Mappings between music and visuals are typically reduced to the music‘s beat and amplitude being statically associated to the visuals, disallowing close audiovisual congruence, tension and release, and suspended expectation in narratives. Collaborative performance, common in other live art, is mostly absent due to technical limitations. Preparing or improvising performances is complicated, often requiring software development. This thesis addresses these, through a transdisciplinary integration of findings from several research areas, detailing the resulting ideas, and their implementation in a novel system: Musical instruments are used as the primary control data source, accurately encoding all musical gestures of each performer. The advanced embodied knowledge musicians have of their instruments, allows increased expressivity, the full control data bandwidth allows high mapping complexity, while musicians‘ collaborative performance familiarity may translate to visual music performance. The conduct of Mutable Mapping, gradually creating, destroying and altering mappings, may allow for a narrative in mapping during performance. The art form of Soma, in which correlated auditory, visual and proprioceptive stimulus form a combined narrative, builds on knowledge that performers and audiences are more engaged in performance requiring advanced motor knowledge, and when congruent percepts across modalities coincide. Preparing and improvising is simplified, through re-adapting the Processing programming language for artists to behave as a plug-in API, thus encapsulating complexity in modules, which may be dynamically layered during performance. 3 Design research methodology is employed during development and evaluation, while introducing the additional viewpoint of ethnography during evaluation, engaging musicians, audience and visuals performers. 4 Acknowledgments Firstly, I wish to thank my supervisors, Beau Lotto and Anthony Steed, for taking on my project, and for helping me through with the work presented in this thesis. I am further grateful to have been surrounded by very interesting people at the Lottolab. Without Lefkothea Andreou, Daniel Hulme, David Malkin, David Corney, Erwan LeMartelot, Martina Wicklein, Rich Clarke, Madi Boyd, Emma Byrne, Sam Walker, and Katherine Bash, the time spent working in the lab would have been far less stimulating. Many thanks are due to Nadia Berthouze, Nick Bryan-Kinns, Alan Blackwell, Marcelo Wanderley, Katherine Stevens, Nigel Jenkins, Olivier Ruellet, Mauritius Seeger, Mat Hourteillan, Alison Hardcastle, Anne-Marie Preston, Illi Adatto, Dor Wand, the members of Accretion and of the Gathering, for their invaluable advice and assistance towards completing the work presented here. I owe my gratitude to my family, without the support of which I would never have had the chance of writing these lines. And last but not least, Giulia! 5 Table of Contents Soma: live performance where congruent musical, visual, and proprioceptive stimuli fuse to form a combined aesthetic narrative .........................................................................................1 Ilias Bergstrom ............................................................................................................................1 Abstract ..............................................................................................................................3 Acknowledgments ..............................................................................................................5 Table of Contents ...............................................................................................................6 Table of Figures ............................................................................................................... 11 1. Introduction ..................................................................................................................... 15 1.1 Limitations with current practice ............................................................................ 17 1.2 Proposed system ..................................................................................................... 21 1.2.1 Increasing expressivity ..................................................................................... 22 1.2.2 Simplifying preparation for performance ........................................................ 23 1.2.3 Increasing achievable mapping complexity and variability ............................. 24 1.2.4 Allowing for collaborative performance .......................................................... 25 1.2.5 Soma: a new artistic practice .......................................................................... 25 1.2.6 A new performer role ...................................................................................... 27 1.3 Methodology ........................................................................................................... 29 1.4 Summary of contributions, and how each is evaluated .......................................... 33 1.5 Thesis structure ....................................................................................................... 36 2 Background ...................................................................................................................... 39 2.1 Relating visual arts to music .................................................................................... 39 2.1.1 Visual Music, Intermedia, Abstract film, and Synesthetic Art ......................... 39 2.1.2 Colour Organs and Lumia ................................................................................ 43 2.1.3 Audiovisual Composition ................................................................................. 49 2.2 Musical Instruments ................................................................................................ 54 2.2.1 Electronic musical instruments and the interface using which these are played 55 6 2.2.2 Analysis of Digital Musical Instruments ........................................................... 59 2.2.3 Mapping ........................................................................................................... 61 2.2.4 Embodied / Enactive knowledge, and music virtuosity................................... 63 2.2.5 Collaborative performance, mutual engagement and group flow.................. 64 2.2.6 The mixing engineer ........................................................................................ 65 2.2.7 Performance using Conducting Gesture DMIs ................................................ 66 2.3 Musical Narrative .................................................................................................... 69 2.3.1 Absolute music ................................................................................................ 69 2.3.2 Narrative in visual music and audiovisual art .................................................. 71 2.4 Programming as art: Creative Code ........................................................................ 73 2.4.1 New Media art ................................................................................................. 74 2.4.2 Demo Scene ..................................................................................................... 75 2.4.3 Live Coding ....................................................................................................... 77 2.5 Programming for non-computer scientists ............................................................. 78 2.5.1 Visual programming languages ....................................................................... 79 2.5.2 Procedural languages, and the Processing language ...................................... 81 2.6 Review of related research and evaluation methodology ...................................... 85 2.6.1 Evaluation of Music Technologies ................................................................... 86 2.6.2 Human Computer Interaction ......................................................................... 89 2.6.3 Review of evaluation methodology employed in recent PhD theses analogous to present work ............................................................................................................. 105 2.7 Recent related work .............................................................................................. 108 3 Mutable Mapping .......................................................................................................... 112 3.1 Related work to mutable mapping ........................................................................ 114 3.2 The mutable mapping performer .......................................................................... 114 4 The Trinity system ......................................................................................................... 117 4.1 Mother ................................................................................................................... 117 4.2 Live Input Processor .............................................................................................. 119 7 4.3 Mediator ................................................................................................................ 120 4.4 Hardware ............................................................................................................... 127 4.5 Supported modes of performance ........................................................................ 129 5 Process........................................................................................................................... 131 5.1 First prototyping stage .......................................................................................... 131 5.2 Second prototyping stage ...................................................................................... 133 5.3 Third prototyping stage: the birth of Mother ....................................................... 136 5.4 Fourth prototyping stage: Mutable Mapping and the Mediator .......................... 138 5.5 Fifth prototyping stage: non-musical control ........................................................ 139 6 Experiments ................................................................................................................... 141 6.1 Online audience survey ......................................................................................... 141 6.2 Rehearsals and live performances with groups of musicians ............................... 144 6.3 Interviews with VJs and live visuals performers .................................................... 146 7 Evaluation ...................................................................................................................... 149 7.1 Results from online survey .................................................................................... 149 7.2 Results from rehearsals with musicians ................................................................ 151 2.7.1 Setup of conducted rehearsals ...................................................................... 151 7.2.1 Summary of conclusions from rehearsals ..................................................... 153 7.3 Results from interviews with VJs and live visuals performers .............................. 155 7.3.1 Olivier Ruellet, AKA Ctrl-N ............................................................................. 155 7.3.2 Mauritius Seeger, AKA Dr. Mo ....................................................................... 158 7.3.3 Mat Hourteillan, AKA Matsai, VJ Om boy ...................................................... 160 7.3.4 Nigel Jenkins, AKA Nebulus ........................................................................... 161 7.3.5 Conclusions from interviews with VJs and live visuals performers ............... 165 7.4 Further observed impact of present work ............................................................ 167 7.5 Technical evaluation .............................................................................................. 167 7.6 Evaluation discussion............................................................................................. 169 8 Further work .................................................................................................................. 172 8 8.1 Experiments ........................................................................................................... 172 8.2 Development ......................................................................................................... 173 8.3 Content .................................................................................................................. 174 9 Conclusion ..................................................................................................................... 176 9.1 Summary of contribution ...................................................................................... 176 9.2 Closing remarks ..................................................................................................... 179 Appendix A: Online survey questionnaire & responses ........................................................ 181 Appendix B: Reports from rehearsals and performances ..................................................... 195 The Gathering, St Pancras Old Church .............................................................................. 195 Guitarist 1: ..................................................................................................................... 196 Guitarist 2: ..................................................................................................................... 197 Percussion synthesizer player: ...................................................................................... 197 Accretion, EC1 Music Project ............................................................................................. 198 Guitarist: ........................................................................................................................ 199 Singer: ............................................................................................................................ 199 Drummer: ...................................................................................................................... 200 Live performance: The Gathering, St Pancras Old Church, 16/12/09 ............................... 201 Rehearsal towards live performance at Inspiral Lounge, 08/01/10 .................................. 202 Percussion synthesizer player: ...................................................................................... 203 Maschine and synthesizer player: ................................................................................. 203 Live performance: Inspiral Lounge, Camden Town, 26/01/10 .......................................... 204 Appendix C: Call for musicians .............................................................................................. 206 I am looking for musicians to accompany with live visuals!.................................................. 206 Appendix D: Non-exhaustive list of visual synthesizers created ........................................... 207 Appendix E: Example code of Processing sketch turned into visual synths for Mother ....... 215 Gradient Processing Sketch ............................................................................................... 215 Gradient Visual Synth for Mother ..................................................................................... 216 Appendix F: Presence research ............................................................................................. 219 9 Relevance to Presence field .......................................................................................... 220 Appendix G: Physiological Measurements for Detecting Enjoyment and Engagement in Video Games, Affective Computing ................................................................................................. 222 Appendix H: Allowing for multiple interpretations during evaluation process .................... 224 Appendix I: HCI, Interaction Design, and present work ........................................................ 229 References ............................................................................................................................. 230 10

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.