Dance Data Cognition and Multimodal Communication PDF
Preview Dance Data Cognition and Multimodal Communication
Routledge Advances in Theatre & Performance Studies DANCE DATA, COGNITION, AND MULTIMODAL COMMUNICATION Edited by Carla Fernandes, Vito Evola, and Cláudia Ribeiro Dance Data, Cognition, and Multimodal Communication Dance Data, Cognition, and Multimodal Communication is the result of a collaborative and transdisciplinary effort towards a first definition of “dance data”, with its complexities and contradictions, in a time where cognitive science is growing in parallel to the need of a renewed awareness of the body’s agency in our manyfold interactions with the world. It is a reflection on the observation of bodily movements in artistic settings, and one that views human social interactions, multimodal communication, and cognitive processes through a different lens—that of the close collaboration between performing artists, designers, and scholars. This collection focuses simultaneously on methods and technologies for creating, documenting, or representing dance data. The editors highlight works focusing on the dancers’ embodied minds, including research using neural, cognitive, behavioural, and linguistic data in the context of dance composition processes. Each chapter deals with dance data from an interdisciplinary perspective, presenting theoretical and methodological discussions emerging from empirical studies, as well as more experimental ones. The book, which includes digital Support Material on the volume’s Routledge website, will be of great interest to students and scholars in contemporary dance, neuro-cognitive science, intangible cultural heritage, performing arts, cognitive linguistics, embodiment, design, new media, and creativity studies. Carla Fernandes is Principal Investigator, Head of ICNOVA’s Research Group on Performance & Cognition, and Professor at FCSH – Universidade Nova de Lisboa, where she directs the ERC-funded “BlackBox LAB Arts&Cognition.” Her current research focus is in the intersection of Performing Arts and Cognitive Science, Multimodal Communication, Intangible Cultural Heritage, and New Media. She is fascinated by the complexity of the human mind and non-verbal behavior in creativity settings. She holds a PhD in Linguistics, supervises numerous PhD and MA theses, and is an author in international peer-reviewed journals and books. Vito Evola is currently a Researcher in Cognitive Linguistics and Multimodal Communication at the Universidade Nova de Lisboa, having previously lectured and conducted research at universities in Palermo, Berkeley, Cleveland, Aachen, and Geneva. His research lies at the intersection of language, culture, and cognition, and analyses data from both common and more specialised contexts, such as patient–doctor interactions, psychotherapy and forensic interviews, religious discourse, and the performing arts. Cláudia Ribeiro is a Postdoctoral Researcher at FCUL, in the research group LASIGE. She obtained her PhD in Information Systems and Computer Engineering at the University of Lisbon. Her research interests include machine learning, deep learning (focusing on emotion recognition in multimedia content), and interactive systems. Routledge Advances in Theatre & Performance Studies This series is our home for cutting-edge, upper-level scholarly studies and edited collections. Considering theatre and performance alongside topics such as religion, politics, gender, race, ecology, and the avant-garde, titles are characterized by dynamic interventions into established subjects and innovative studies on emerging topics. ASHÉ Ritual Poetics in African Diasporic Paul Cater Harrison, Michael D. Harris, and Pellom McDaniels Dancehall In/Securities Perspectives on Caribbean Expressive Life Patricia Noxolo, ‘H’ Patten, and Sonjah Stanley Niaah Circus and the Avant-Gardes History, Imaginary, Innovation Anna-Sophie Jürgens and Mirjam Hildbrand Aesthetic Collectives On the Nature of Collectivity in Cultural Performance Andrew Wiskowski Dance Data, Cognition, and Multimodal Communication Carla Fernandes, Vito Evola, and Cláudia Ribeiro Theatre and the Virtual Genesis, Touch, Gesture Zornitsa Dimitrova Old Norse Poetry in Performance Brian McMahon & Annemari Ferreira For more information about this series, please visit: https://www.routledge.com/ Routledge-Advances-in-Theatre--Performance-Studies/book-series/RATPS Dance Data, Cognition, and Multimodal Communication Edited by Carla Fernandes, Vito Evola, and Cláudia Ribeiro First published 2023 by Routledge 4 Park Square, Milton Park, Abingdon, Oxon OX14 4RN and by Routledge 605 Third Avenue, New York, NY 10158 Routledge is an imprint of the Taylor & Francis Group, an Informa business © 2023 selection and editorial matter, Carla Fernandes, Vito Evola, and Cláudia Ribeiro: individual chapters, the contributors The right of Carla Fernandes, Vito Evola, and Cláudia Ribeiro to be identified as the authors of the editorial material, and of the authors for their individual chapters, has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988. All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers. Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. British Library Cataloguing-in-Publication Data A catalogue record for this book is available from the British Library Library of Congress Cataloguing-in-Publication Data A catalog record has been requested for this book Access the digital Support Material online: https://www.routledge.com/ 9780367617455 ISBN: 978-0-367-61745-5 (hbk) ISBN: 978-0-367-62116-2 (pbk) ISBN: 978-1-003-10640-1 (ebk) DOI: 10.4324/9781003106401 Typeset in Bembo by MPS Limited, Dehradun Contents List of Illustrations viii List of Contributors xv Introduction 1 CARLA FERNANDES PART I Performance-as-Research: Dance data from the artists’ perspectives 13 1 Negotiating deliberate choice-making: Insights from an interdisciplinary and multimodal encounter during the making of a New Contemporary Dance 15 SYLVIA RIJMER 2 Dance | Data | Storytelling 38 STEPHAN JÜRGENS 3 Enabling multimodal interaction in mixed-abled dance: Insights into creating highly accessible teaching tools for inclusive cultural work 55 SUSANNE QUINTEN AND MIA SOPHIA BILITZA PART II Dance documentation and dance scores 69 4 Recording “Effect”: A case study in technical, practical, and critical perspectives on dance data creation 71 DAVID RITTERSHAUS, ANTON KOCH, SCOTT DELAHUNTA, AND FLORIAN JENETT vi Contents 5 Digital-born artworks and interactive experience: Documentation and archiving 89 PAULA VARANDA 6 Dance scoring and en-action as a creative tool for dance documentation 99 BERTHA BERMÚDEZ-PASCUAL 7 Terpsicore – dance and performing arts archive 109 DANIEL TÉRCIO, CATARINA CANELAS, AND ANA LUÍSA VALDEIRA PART III Computational dance data: Between the real and the virtual 117 8 Augmented seeing and sensing 119 ANGUS G. FORBES 9 Motion capture and the digital dance aesthetic: Using inertial sensor motion tracking for devising and producing contemporary dance performance 131 DANIEL STRUTT 10 Capturing and visualizing 3D dance data: Challenges and lessons learnt 148 CLÁUDIA RIBEIRO, RAFAEL KUFFNER, AND CARLA FERNANDES PART IV The brain’s experience of dance 161 11 The embodied neuroaesthetics of watching dance 163 EMILY S. CROSS AND REBECCA SMITH 12 Dancing neurons: Common brain activity fMRI analysis of the cerebral phenomena behind dance perception 182 SOFIA AMARAL MARTINS AND FRANK POLLICK 13 “I see something, and I like it”: Unveiling a choreographer’s decision-making process using quantitative and qualitative methods 202 ANA RITA FONSECA, RODRIGO ABRIL-DE-ABREU, AND CARLA FERNANDES Contents vii PART V Dance expertise and cognition 221 14 Dance expertise, embodied cognition, and the body in the brain 223 BETTINA BLÄSING 15 What makes dancers extraordinary? Insights from a cognitive science perspective 244 CARLA FERNANDES, VITO EVOLA, AND JOANNA SKUBISZ 16 The role of dance experience, visual processing strategies, and quantitative movement features in recognition of emotion from whole-body movements 274 REBECCA SMITH AND FRANK POLLICK PART VI Cognitive metaphor and gesture in dance and theatre 295 17 Unpeeling meaning: An analogy and metaphor identification and analysis tool for modern and post-modern dance, and beyond 297 VICKY J. FISHER 18 Understanding non-verbal metaphor: A cognitive approach to metaphor in dance 320 LACEY OKONSKI, JULIE MADDEN, AND KAITLIN TOTHPAL 19 Study on hand movements accompanied during the description of dance appreciation 333 ZI HYUN KIM AND HEDDA LAUSBERG 20 Reduction of gesticulation and information patterning strategies in acted speech 346 GIORGINA CANTALINI AND MASSIMO MONEGLIA 21 Lines of experience: Towards a research method 363 MICHAEL O’CONNOR Note about Funding 373 Author Index 374 Subject Index 382 Illustrations Figures 1.1 A crude illustration of a Body Logic filtering process (author’s artwork) 18 1.2 Diagram of the tree metaphor illustrating the dance-making creative process outlined in this Arts and Science collaboration (author’s artwork) 22 1.3 The dancers exploring Dodge Mode using Re-Dodge Perspectives in a studio rehearsal; (a) contactless; (b) contactful. Lisbon, October 12, 2018. Screenshots provided by BlackBox 24 1.4 The dancers exploring “Simple Basic Scan” using their fingertips as Prime Movers of a Con-sequential Movement Pattern (which appears as their hands and arms are lifting). Lisbon, October 12, 2018. Screenshot taken from video provided by BlackBox 25 1.5 Dancers Teresa’s (left), Elson’s (centre), and Allan’s (right) selections of graphic elements (above; details adapted from Cardew, 1967: 181, 21, 22) and annotated Re-Scored Dance Maps (below). Reprinted with permission 28 1.6 Dancer Allan fully immersed in his Treatise environment during a VR studio rehearsal exploration. Lisbon, October 2, 2018. Author’s photograph 29 1.7 Left: Dancer Allan’s selected element (detail adapted from Cardew, 1967: 20; reprinted with permission), later imported into various VR environments for the dancer’s full-body exploration. Right: Dancer’s perspective of the graphic element in the “Daytime” (above) and “Dust Particle” (below) VR environments. Screenshot taken from VR video produced by BlackBox 30 Illustrations ix 2.1 Video still from the documentary film O Que Fazer Daqui Para Tràs (What To Do With What Remains, 2017). The animated infographic shows the entrances, exits, and durations of the five dancers on the stage in a score layout 42 2.2 Video still from the 360° video recording of Fifteen Dancers and Changeable Tempo (2016), section 16, on stage view. The animated infographics provide information about the dancer’s name, the choreographic material that is danced, and the relations that are formed between the dancers 44 2.3 Video still from the VR installation on the creative process of Sylvia Rijmer and her dancers. Upon entry of the VR installation, the viewer sees three large buildings in the shape of elements from the Cornelius Cardew score 45 2.4 Video still from the 360° video recording of Fifteen Dancers and Changeable Tempo (2016), section 12, audience view. The interactive score can be used to jump to any entrance that one of the dancers on the list makes 48 2.5 Video still from the VR installation displaying an image sculpture and four panels with stereoscopic photographs 50 3.1 Research process 58 3.2 Documentation template first version 60 3.3 Overview of participants of the artistic laboratory in Greece 61 3.4 Introduction to contact improvisation: Starting position CREABILITY, © Un-Label 2019 62 3.5 Introduction to contact improvisation: Bringing the partner to the floor CREABILITY, © Un-Label 2019 63 4.1 Annotated video in Piecemaker (Motion Bank Web Systems) from the rehearsal documentation of Effect. Annotations were created live in the studio in parallel to the video recording 74 4.2 The custom data player by Motion Bank allows one to load and view a data set and to stream the data to any other application via OSC protocol, making it easier for creative coders to work with the data 79 4.3 3D visualisation of the motion capture data with a subtle way of representing the dancer’s bodies as not to steer the understanding in a certain direction through the design of the avatars. Additional graphical elements illustrate movement concepts, in this case the so-called “Blackholes” 82 4.4 Visualisations of pathways for the piece’s section “Figure 8” 84 9.1 The Rokoko Smartsuit Pro in action (rokoko.com) 132 9.2 Studio capture with Mavin Khoo in Valletta, Malta (image by Prickimage) 139
The list of books you might like
