Albert Ali Salah Hayley Hung Oya Aran Hatice Gunes (Eds.) 2 Human Behavior 1 2 8 S Understanding C N L 4th International Workshop, HBU 2013 Barcelona, Spain, October 2013 Proceedings 123 Lecture Notes in Computer Science 8212 CommencedPublicationin1973 FoundingandFormerSeriesEditors: GerhardGoos,JurisHartmanis,andJanvanLeeuwen EditorialBoard DavidHutchison LancasterUniversity,UK TakeoKanade CarnegieMellonUniversity,Pittsburgh,PA,USA JosefKittler UniversityofSurrey,Guildford,UK JonM.Kleinberg CornellUniversity,Ithaca,NY,USA AlfredKobsa UniversityofCalifornia,Irvine,CA,USA FriedemannMattern ETHZurich,Switzerland JohnC.Mitchell StanfordUniversity,CA,USA MoniNaor WeizmannInstituteofScience,Rehovot,Israel OscarNierstrasz UniversityofBern,Switzerland C.PanduRangan IndianInstituteofTechnology,Madras,India BernhardSteffen TUDortmundUniversity,Germany MadhuSudan MicrosoftResearch,Cambridge,MA,USA DemetriTerzopoulos UniversityofCalifornia,LosAngeles,CA,USA DougTygar UniversityofCalifornia,Berkeley,CA,USA GerhardWeikum MaxPlanckInstituteforInformatics,Saarbruecken,Germany Albert Ali Salah Hayley Hung Oya Aran Hatice Gunes (Eds.) Human Behavior Understanding 4th International Workshop, HBU 2013 Barcelona, Spain, October 22, 2013 Proceedings 1 3 VolumeEditors AlbertAliSalah Bog˘aziçiUniversity,Istanbul,Turkey E-mail:[email protected] HayleyHung DelftUniversityofTechnology,TheNetherlands E-mail:[email protected] OyaAran IdiapResearchInstitute,Martigny,Switzerland E-mail:[email protected] HaticeGunes QueenMaryUniversityofLondon,UK E-mail:[email protected] ISSN0302-9743 e-ISSN1611-3349 ISBN978-3-319-02713-5 e-ISBN978-3-319-02714-2 DOI10.1007/978-3-319-02714-2 SpringerChamHeidelbergNewYorkDordrechtLondon LibraryofCongressControlNumber:2013950178 CRSubjectClassification(1998):I.5,I.2,H.5,H.3,I.4,H.4 LNCSSublibrary:SL6–ImageProcessing,ComputerVision,Pattern Recognition,andGraphics ©SpringerInternationalPublishingSwitzerland2013 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation, broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection withreviewsorscholarlyanalysisormaterialsuppliedspecificallyforthepurposeofbeingenteredand executedonacomputersystem,forexclusiveusebythepurchaserofthework.Duplicationofthispublication orpartsthereofispermittedonlyundertheprovisionsoftheCopyrightLawofthePublisher’slocation, initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Permissionsforuse maybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violationsareliabletoprosecution undertherespectiveCopyrightLaw. Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse. Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateofpublication, neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityforanyerrorsor omissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,withrespecttothe materialcontainedherein. Typesetting:Camera-readybyauthor,dataconversionbyScientificPublishingServices,Chennai,India Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) Preface Domains where human behavior understanding is a crucial need (e.g., robotics, human–computer interaction, affective computing, and social signal processing) relyonadvancedpatternrecognitiontechniquestoautomaticallyinterpretcom- plexbehavioralpatternsgeneratedwhenhumansinteractwithmachinesorwith others. This is a challenging problem where many issues are still open, includ- ing the joint modeling of behavioral cues taking place at different time scales, theinherentuncertaintyofmachine-detectableevidencesofhumanbehavior,the mutualinfluenceofpeopleinvolvedininteractions,thepresenceoflong-termde- pendencies in observations extracted from human behavior, and the important role of dynamics in human behavior understanding. The4thWorkshoponHumanBehaviorUnderstanding(HBU),organizedasa satellitetoACMMultimedia(ACMMM2013,Barcelona),gatheredresearchers dealing with the problem of modeling human behavior under its multiple facets (expression of emotions, display of relational attitudes, performance of individ- ual or joint actions, imitation, etc.). The HBU Workshop, previously organized jointly with the International Conference on Pattern Recognition (ICPR 2010, Istanbul), International Joint Conference on Ambient Intelligence (AMI 2011, Amsterdam)andIEEE/RSJInternationalConferenceonIntelligentRobotsand Systems(IROS2012,Algarve),highlightsdifferentaspectsofthis problemsince its inception. The 4th HBU Workshop focuses on applications of human be- havior analysis for interactions in creativity, arts, entertainment, and edutain- ment. There are tremendous socialand societal implications of creatinghuman- centered, adaptive, and responsive systems in these areas. The workshop featured two invited talks. The first talk by Dr. Antonio Ca- murri (Casa Paganini InfoMus Research Centre, DIBRIS, University of Genoa) was entitled “Multimodal Systems for Embodied Experience of Music and Au- diovisual Content.” Dr. Camurri, in his talk, remarkedthat the adoption of the embodied cooperation paradigm opens new perspectives for social applications infuture user-centricmedia. Atthe sametime, mobile technologiesenable us to shareexperiencesin aneasierandfaster way,usingmultiple modalities andme- dia. By combining these aspects, Dr. Camurri and colleagues proposed mobile systems integrating real-time analysis of movement and of non-verbal expres- sive gesture and social behavior. This keynote presented an overview of recent researchresultsatCasaPaganini-InfoMusinthesedirections,includingtheEye- sWebXMIsoftwareplatform,focusingonsocio-mobileembodiedexperienceand retrieval of music and audiovisual content, and on teaching autistic children to recognizeandexpressemotionsbynon-verbalfull-bodymovementsandgestures. The researchof Dr. Camurri is partially funded by the European Projects ASC INCLUSION (7 Framework Programme ICT) and MetaBody (EU Culture). VI Preface The second keynote talk was by Dr. Pushmeet Kohli (Microsoft Research Cambridge, UK), as a part of the ACM Distinguished Speakers Program. In his talk entitled “Learning to Interact (Naturally) with (All) Users,” Dr. Kohli discussed the development of “natural” user interfaces that can understand the intent and preferences of users and are easier to use. Making machines under- standhumanintentions andpreferencesis anexceptionallychallengingproblem for the AI community. Part of this difficulty lies in capturing the large vari- ability in preferences and behavior of different users. In his talk, Dr. Kohli dis- cussedsomestepstakenbyhisresearchgroupatMicrosoftResearchCambridge to overcome this problem in the context of two application scenarios: interac- tion/activity recognition using the Kinect, and information retrieval. Dr. Kohli described their recent work on human pose estimation using the Kinect sensor and the challenges of developing a system that is supposed to work on “every- body.” He then elaborated on the problem of personalization in the context of information retrieval and discussed how traits like the personality of users can be inferred from their online behavior. This proceedings volume contains the papers presented at the workshopand a summarizing paper. We have received50 submissions in total, andeachpaper was peer-reviewed by at least two members of the Technical Program Commit- tee. Twelve papers were accepted as oral presentations, and eight papers were presented as posters. We would like to take the opportunity to thank our Program Commit- tee members and reviewers for their rigorous feedback, our authors and our keynote speakers for their contributions, Maja Pantic, Vladimir Pavlovic, and Nicu Sebe from the ACM MM Organizing Committee for their help. We thank the ACM Distinguished Speakers Program, the InfoMus Research Center, the EPSRCMAPTRAITSProject(GrantRef:EP/K017500/1),SwissNationalSci- ence Foundation (SNSF) Ambizione fellowshipprojectPZ00P2 136811,andthe BAP 6531 project of Bo˘gazi¸ci University for their support. October 2013 Albert Ali Salah Hayley Hung Oya Aran Hatice Gunes Organization Conference Co-chairs Albert Ali Salah Bog˘azic¸i University, Turkey Hayley Hung Delft University of Technology, The Netherlands Oya Aran Idiap Research Institute, Switzerland Hatice Gunes Queen Mary University of London, UK Technical Program Committee Elisabeth Andr´e University of Augsburg, Germany Nick Bryan-Kinns Queen Mary University of London, UK Carlos Busso University of Texas at Dallas, USA Rafael E. Calvo University of Sydney, Australia Antonio Camurri University of Genova, Italy Jeffrey Cohn University of Pittsburgh, USA Simon Colton Imperial College London, UK Fernando de la Torre CMU, USA Hamdi Dibeklio˘glu Delft University of Technology, The Netherlands Thierry Dutoit University of Mons, Belgium Gwenn Englebienne University of Amsterdam, The Netherlands Jordi Gonzalez UAB-CVC Barcelona, Spain Daniel Gonzalez-Jimenez Gradiant, Spain Zakia Hammal CMU, USA Christian Jacquemin LIMSI-CNRS, France Dinesh Jayagopi Idiap Research Intitute, Switzerland Kostas Karpouzis National Technical University of Athens, Greece Cem Keskin Microsoft Research Cambridge, UK Ben Kro¨se University of Amsterdam, The Netherlands Dana Kulic University of Waterloo, Canada Matei Mancas University of Mons, Belgium Louis-Philippe Morency USC, USA Florian ‘Floyd’ Mueller RMIT, Australia Frank Nack University of Amsterdam, The Netherlands Hiroshi Okuno Kyoto University, Japan Isabella Poggi Roma Tre University, Italy Thierry Pun University of Geneva, Switzerland Francis Quek Virginia Tech, USA VIII Organization Ben Schouten Eindhoven Technical University, The Netherlands Bj¨orn Schuller Technical University of Munich, Germany Alan Smeaton Dublin City University, Ireland Koray Tahiro˘glu Aalto University, Finland Reiner Wichert Fraunhofer IGD, Germany Additional Reviewers Enrique Aragones-Ru´a Derya O¨zkan Sharon Chu Fumihide Tanaka Wen-Sheng Chu Esteban Vazquez-Fernandez Raquel Dosil-Lago Kazuyoshi Yoshii Table of Contents Creative Applications of Human Behavior Understanding ............. 1 Albert Ali Salah, Hayley Hung, Oya Aran, and Hatice Gunes Behaviour and Affect in Arts, Creativity, Entertainment, and Edutainment Applications A Behavioral Study on the Effects of Rock Music on Auditory Attention ....................................................... 15 Letizia Marchegiani and Xenofon Fafoutis Human Nonverbal Behaviour Understanding in the Wild for New Media Art ...................................................... 27 Evan Morgan and Hatice Gunes Creative Dance: An Approach for Social Interaction between Robots and Children ............................................. 40 Raquel Ros and Yiannis Demiris Stylistic Features for Affect-Based Movie Recommendations ........... 52 Jussi Tarvainen, Stina Westman, and Pirkko Oittinen Actions and Activities Dynamic Feature Selection for Online Action Recognition ............. 64 Victoria Bloom, Vasileios Argyriou, and Dimitrios Makris A Fully Unsupervised Approach to Activity Discovery ................ 77 Umut Avci and Andrea Passerini Transfer Learning of Human Poses for Action Recognition............. 89 Mario Rodr´ıguez, Carlos Medrano, Elias Herrero, and Carlos Orrite ATTENTO: ATTENTion Observed for Automated Spectator Crowd Analysis ........................................................ 102 Davide Conigliaro, Francesco Setti, Chiara Bassetti, Roberta Ferrario, and Marco Cristani Human Behavior Understanding with Wide Area Sensing Floors ....... 112 Martino Lombardi, Augusto Pieracci, Paolo Santinelli, Roberto Vezzani, and Rita Cucchiara X Table of Contents Facial Behavior Efficient Graph Construction for Label Propagation Based Multi-observation Face Recognition ................................ 124 Fadi Dornaika, Alireza Bosaghzadeh, and Bogdan Raducanu Multiple Local Curvature Gabor Binary Patterns for Facial Action Recognition ..................................................... 136 Anıl Yu¨ce, Nuri Murat Arar, and Jean-Philippe Thiran A Dense Deformation Field for Facial Expression Analysis in Dynamic Sequences of 3D Scans............................................ 148 Hassen Drira, Boulbaba Ben Amor, Mohamed Daoudi, and Stefano Berretti Social Signals NovA: Automated Analysis of Nonverbal Signals in Social Interactions ..................................................... 160 Tobias Baur, Ionut Damian, Florian Lingenfelser, Johannes Wagner, and Elisabeth Andr´e Social Behavior Modeling Based on Incremental Discrete Hidden Markov Models .................................................. 172 Alaeddine Mihoub, G´erard Bailly, and Christian Wolf MMLI: Multimodal Multiperson Corpus of Laughter in Interaction..... 184 Radoslaw Niewiadomski, Maurizio Mancini, Tobias Baur, Giovanna Varni, Harry Griffin, and Min S.H. Aung Real-Time Comprehensive Sociometrics for Two-PersonDialogs........ 196 Umer Rasheed, Yasir Tahir, Shoko Dauwels, Justin Dauwels, Daniel Thalmann, and Nadia Magnenat-Thalmann VIP: A Unifying Framework for Computational Eye-Gaze Research .... 209 Keng-Teck Ma, Terence Sim, and Mohan Kankanhalli Affective Signals Head, Shoulders and Hips Behaviors during Turning.................. 223 Nesrine Fourati and Catherine Pelachaud