Human-Robot Interaction: Evaluation Methods and Their Standardization PDF
Preview Human-Robot Interaction: Evaluation Methods and Their Standardization
Springer Series on Bio- and Neurosystems 12 Céline Jost · Brigitte Le Pévédic · Tony Belpaeme · Cindy Bethel · Dimitrios Chrysostomou · Nigel Crook · Marine Grandgeorge · Nicole Mirnig Editors Human-Robot Interaction Evaluation Methods and Their Standardization Springer Series on Bio- and Neurosystems Volume 12 Series Editor Nikola Kasabov, Knowledge Engineering and Discovery Research Institute, Auckland University of Technology, Penrose, New Zealand The Springer Series on Bio- and Neurosystems publishes fundamental principles and state-of-the-art research at the intersection of biology, neuroscience, informa- tionprocessingandtheengineeringsciences.Theseriescoversgeneralinformatics methods and techniques, together with their use to answer biological or medical questions.Ofinterestarebothbasicsandnewdevelopmentsontraditionalmethods such as machine learning, artificial neural networks, statistical methods, nonlinear dynamics,informationprocessingmethods,andimageandsignalprocessing.New findingsinbiologyandneuroscienceobtainedthroughinformaticsandengineering methods,topicsinsystemsbiology,medicine,neuroscienceandecology,aswellas engineering applications such as robotic rehabilitation, health information tech- nologies, and many more, are also examined. The main target group includes informaticians and engineers interested in biology, neuroscience and medicine, as well as biologists and neuroscientists using computational and engineering tools. Volumespublishedintheseriesincludemonographs,editedvolumes,andselected conference proceedings. Books purposely devoted to supporting education at the graduate and post-graduate levels in bio- and neuroinformatics, computational biologyandneuroscience,systemsbiology,systemsneuroscienceandotherrelated areas are of particular interest. The books of the series are submitted for indexing to Web of Science. More information about this series at http://www.springer.com/series/15821 é é é C line Jost Brigitte Le P v dic (cid:129) (cid:129) Tony Belpaeme Cindy Bethel (cid:129) (cid:129) Dimitrios Chrysostomou Nigel Crook (cid:129) (cid:129) Marine Grandgeorge Nicole Mirnig (cid:129) Editors Human-Robot Interaction Evaluation Methods and Their Standardization 123 Editors CélineJost Brigitte LePévédic University Paris 8 IUT deVannes—Département STID Saint-Denis, France University of SouthBrittany Vannes,France TonyBelpaeme IDLab—imec—ELIS CindyBethel Ghent, Belgium Social, Therapeutic, andRobotic Systems Lab Dimitrios Chrysostomou Mississippi State University Robotics andAutomation Group Mississippi State,MS, USA Aalborg University Aalborg, Denmark NigelCrook Research andKnowledgeExchange Marine Grandgeorge Faculty of Technology, Laboratoire Ethologie Animale et Humaine DesignandEnvironment UniversitéRennes 1 Oxford BrookesUniversity Paimpont,France Oxford,UK NicoleMirnig Centerfor Human-Computer Interaction University of Salzburg Salzburg, Austria ISSN 2520-8535 ISSN 2520-8543 (electronic) SpringerSeries onBio- andNeurosystems ISBN978-3-030-42306-3 ISBN978-3-030-42307-0 (eBook) https://doi.org/10.1007/978-3-030-42307-0 ©SpringerNatureSwitzerlandAG2020 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar methodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictionalclaimsinpublishedmapsandinstitutionalaffiliations. ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland Foreword Robotic agents and the related domains we think of spontaneously (AI, robotics, mechanics,computerscience,etc.)areintheheartofagrowingnumberofdebates inoursociety,astheindustrial,economicandtechnologicalchallengesarecrucial. However, if these stakes are obviously essential, the current or future presence of these robotic agents in all our living spaces (e.g. industry, hospital, institutions forelderlypeople,athome,atschool)raisesquestionsrelatedtothehumanfactors: what interactions with us? What are the impacts on our behaviours and our activities? What ethical dimensions must be taken into account? How to evaluate for better design? Beyond extreme opinions and positions (techno-phobia versus technophilia), it is, therefore, necessary to study and especially to anticipate expectations, uses, mental representations and human behaviours during IHR (Human-Robot Interactions). Indeed, if some technological barriers still limit robot design, the realbarriersarelinkedtohumanandsocietalbarriersintermsofacceptability,trust, emotional relationships, impact on performance and behaviour, and impact on organisations.Itisthesehumandimensionsthatthisbookproposestothereaderto address through chapters produced by experts in these different domains. Concretely, the introduction of robots in our homes or in our institutions questionsaboveallthehumanandethicaldimensionssince itprofoundlymodifies ourrelationshipsnotonlywithobjectsbutalsowithotherindividuals:indeed,these robotsare supposed toprovide monitoringfunctions (e.g. fall detection), cognitive stimulationfunctions(e.g.offeringandinitiatinginteractions),socialfunctions(e.g. prevent social isolation) or play functions. But how can these functions be recon- ciled with respect for the human individual? In addition, how do these robots impact on the relationships between individuals in these different environments? Robots are not only technical objects; they are socio-technical objects in the sensewhereontheonehand,theyinfluencetherelationshipsbetweenotheragents in the environment (whether these agents are humans or other socio-technical objects) and on the other hand, are themselves modified by these same relation- ships. In other words, robots first and foremost raise human questions. Moreover, the interactions we have or will have with robots question mainly in the case of v vi Foreword so-called“social”robots,assistanceor“companions”,i.e.inthecaseofrobotswith which physical proximity will be strong, frequent exchanges and important inter- dependencies.Thereplacementofhumansbyrobotsintasksconsidereddangerous or in extreme environments is commonly accepted (e.g. industrial robots, for underwater exploration, for space exploration); their presence in our daily and “intimate” living spaces is much less accepted a priori. The design of robots is crucial since it largely determines the acceptance and interactions that will be linked. The evaluation of interactions is equally decisive and several chapters are devoted to the methodological aspects of HMI. Recommendations and concrete approaches are also proposed to help researchers, designers and decision-makers to better take human factors into account from the early stages of robot agent design. Whether it is the collection and analysis of quantitative or qualitative data, in an ecological or laboratory context, behavioural or attitudinal, this book provides valuable insights for designing evaluation protocols and methods that will finally provide objective, verifiable, reproducible, i.e. scientific data. The User Experience domain (or UX) is a particularly relevant domain for designing interactions between robotic agents and human agents. If this domain is relevant,itisbecauseitproposestotakeintoaccountandanticipatetheexperiences (real or imagined) that the individual has with technical objects such as robots. Similarly,assomechaptersofthisbookdemonstrate,ethologyandethnographyare disciplines that need to be reintroduced into robotic systems research and devel- opment projects since they provide not only theoretical concepts to better under- standwhatisatstakeininteractionsbetweenhumansandrobots,butalsotoolsand techniques for analyzing highly relevant situations. ThefieldofHRI(Human-RobotInteraction)thereforeraisesquestionsrelatedto human nature and its relationship with specific objects (here, robots). Some ques- tionsstillseemtobesciencefiction.Forexample,canwecollaboratewitharobot, orevenbeinahierarchicalrelationshipinwhichwemustobeyarobot?Likewise, canonelovearobotorbelovedbyarobot?Finally,canweadoptorbeadoptedby a robot? The common point to these questions is that they are less technical than human (psychological, ethological, ethical, sociological, sociological, legal, etc.). This book offers, on the basis of proven theories and methodologies, to address these questions in order to anticipate our future. Lorraine, France Jérôme Dinet September 2019 Acknowledgements WethankLeontinaDiCeccowhohassupportedussincethefirstideaeverforthis book. She was immediately interested in this project. Since 2014 she has given us advice and all of her support with help while writing this book. We thank the entire Springer team for their interest in this project, their help, kindness, professionalism, support, and patience. This book exists thanks to all of them. We thank all the authors for their important engagement with this book. All of them have given their best to provide high-quality contributions, in spite of the absence of specific funding to do this job and in spite of their busy lives. Each author is indispensable to this book and we are grateful that they trusted in this project and were patient until its publication. WethankAurélieClodic,CNRSresearcherattheLAASlaboratory,whourged us to organize the first workshop which was the trigger of all this work and sup- ported us during all these years. We thank the organizers of workshops in the conferences that held our three workshops: Ginevra Castellano and François Ferland for ICSR 2015 in Paris; Fulvio Mastrogiovanni for RO-MAN 2016 and RO-MAN 2017 who also follows this work and the book; and Diego Faria for RO-MAN 2017. These workshops were essential for the authors to meet each other, to grow our team, to start reflections,tounderstandthemainconcernsandissues,todelineatethecontext,and in one word to prepare the basis for this book. WethankJérômeDinetwhoacceptedtowritetheforewordofthisbook,which demonstrates his interest in this book in spite of a busy life. We thank all the participants of the workshops and supporters who are many more people than the number of contributors to this book. Many researchers are interested in this project and have contributed to the reflections that were essential for the compilation of this book. We thank our institutions and our colleagues who supported us in this project, who gave us advice, who asked questions, who showed their interest, and who answered our questions. vii Introduction and Book Overview We are very pleased to introduce this work, which is the fruit of 5 years of exceptional collaborative work. We did not receive specific funding and each author contributed voluntarily with a real passion for our cause. Is there a nobler cause than scientific rigor according to researchers? Thisworkoriginatesfromaseriesofworkshops,whichwererichinencounters and open discussions. Our adventure began in August 2014 during the RO-MAN conference. Celine Jost, Marine Grandgeorge, and Brigitte Le Pévédic presented a paper that received an award for their multidisciplinary work in designing an evaluation for Human-Robot Interaction (HRI). It was from this common interest, that they started to exchange communications with Tony Belpaeme and Cindy Bethel. As chance would have it, Leontina Di Cecco from Springer was attending theconferenceasanexhibitorwithbookspublishedbySpringer.Afewdiscussions later,theprojectwasborn.Theorganizers’coregroupwasformed.Whereuponwe quickly started the first phase of this work and organized workshops in order to assemblealargergroupofresearchers,acommunitythatisstillgrowing.Through these workshops, we met researchers who were also organizing workshops on the same theme of standardizing metrics and measures for HRI. Our communities joined together toform aneven larger group ofmembers with this area ofinterest. To clearly understand the history and evolution of this book, General Context sumsupthecontextoftheproblemsencounteredin2014.MethodologiestoDesign Evaluations presents the workshops series we organized as well as people who contributed to this work. Some Standardization Proposals explains the structure of the book and what the reader can find within this book. Disciplinary Points of View provides a summary of each chapter. ix x IntroductionandBookOverview Initial Context Thispartpresentstheevolutionofcontributionsovertime,focusingontwoperiods. During the first period, researchers pointed out methodological issues and current practice mistakes, while during the second period, they proposed new evaluation methods. First Period: Researchers Pointed Out Practice Mistakes Inthebeginning,robotswereindustrialmachinesbuiltinordertoperformtasksin placeofhumans.However,withtheemergenceoftechnology,theyrapidlyoffered perspectives related to humans’ daily life as service robots and then companion robots.Humanstookupmorespaceinthisnewrelationshipandresearchersbegan to consider them more as robot partners. Scholtz [1] indicated, in 2002, that evaluation issues arise from a mismatch between the system’s representation and the user’s expectations. She pointed out theneed for robotsto respectsocial norms, tobe understood by people, toact ina coherent manner. She proposed to evaluate the limit of humans’ acceptance of robots and to make the results publicly available. In this way, other researchers were able to reproduce experimental designs and compare results. At the same time, Dautenhahn and Werry [2] wondered how best to analyze human-robot interactions knowing that questionnaires and interviews cannot be appliedincontextswhererobotsdonothaveanexplicittasktodo.Theyproposed ananalysistechnique“inspiredbyatechniqueusedinpsychologythatisbasedon ‘micro-behaviours’.” They highlighted the importance of objective measures to avoid biases related to influences of the experimenters or participant’s attitudes or expectations of the study outcomes. TheimportanceofobservationwasalsostressedbySabanovicetal.[3]in2006. They argued that robots should be observed, objectively and analytically in real-world environments, as it is not possible to obtain consistent results in the laboratory. They stressed the importance of testing the interaction with untrained participants because their knowledge, history, and life may influence results. The phrase “human in the loop” became obvious to researchers from various fields.In2008,Tsuietal.[4]arguedthatfocusingonahuman’sperformancegives the possibility to correlate the performance of the system with that of the human’s performance. However, they pointed out the need to choose appropriate perfor- mance measures, introducinganinterdisciplinaryapproach.Indeed, intheirminds, choosingsuitableperformancemeasuresrequiresconsultingaspecialistinthefield of evaluation. Thus, the validity of these studies rapidly became the main concern, and some researchers started to criticize the practices of other researchers. For example, in 2009, Syrdal et al. [5] analyzed the use of the Negative Attitudes Towards Robots
The list of books you might like
