Multimodal Interaction with W3C Standards: Toward Natural User Interfaces to Everything PDF
Preview Multimodal Interaction with W3C Standards: Toward Natural User Interfaces to Everything
Deborah A. Dahl Editor Multimodal Interaction with W3C Standards Toward Natural User Interfaces to Everything Multimodal Interaction with W3C Standards Deborah A. Dahl Editor Multimodal Interaction with W3C Standards Toward Natural User Interfaces to Everything Editor DeborahA.Dahl ConversationalTechnologies PlymouthMeeting,PA,USA ISBN978-3-319-42814-7 ISBN978-3-319-42816-1 (eBook) DOI10.1007/978-3-319-42816-1 LibraryofCongressControlNumber:2016952224 ©SpringerInternationalPublishingSwitzerland2017 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilarmethodologynowknownorhereafterdeveloped. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthis book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained hereinorforanyerrorsoromissionsthatmayhavebeenmade. Printedonacid-freepaper ThisSpringerimprintispublishedbySpringerNature TheregisteredcompanyisSpringerInternationalPublishingAG Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland To the memory of my parents, Robert and Marilyn Dahl Foreword Theworldofcomputingischangingrapidly,aswemovefromdesktopcomputers to tablets, mobile phones, watches, rings, and other things such as devices containingembedded computersorsensorsconnectedtotheInternet. New modes of input to computers are evolving from keyboarding and clicking to speaking, gesturing, seeing, and sensing our actions and behaviors. Future applications will integrate information fromone ormore user inputs anddetermine the appropriate response. Standardsareoneapproachtotamingtheexponentialgrowthofthiscomplexity andopportunity.StandardAPIshidethecomplexityofhowhardwareandsoftware platformswork.These APIsenableapplication developerstoaccess thefunctions providedbyplatformsandtocreatenewandexcitingapplications.Givenastandard API,differentplatformdeveloperswilloptimizeinternalprocessesforspecialized hardware or software, enabling application developers to (1) choose the best platform available for their application; (2) implement software on multiple plat- forms, using the same API; and (3) switch from one platform to another as new platformsbecomeavailableandoldplatformsbecomeobsolete. Thisbookintroducesexistingandpotentialstandardsformultimodaltechnolo- giesandprovidesexamplesofhowthesestandardscanintegratemultiplemodesof input for user interaction with existing applications. It also provides suggestions aboutthefeatures,functions,andcapabilitiesthatnewplatformsmightprovideto applicationdevelopers. If you are a platform developer, this book will help you determine which new features you might add to existing platforms or include in future platforms. Stan- dardAPIsenableplatformdeveloperstoenterthemarketplacewithnewplatforms thatcompetewithestablishedplatforms. If you are an application developer, this book will help you understand which newfeatureswillbeavailableonfuturehardwareandsoftwareplatformsandwhich APIswillbeavailabletoaccessthosefeatures.Withthisinsight,youcanplanhow tousethesenewfeaturesandfunctionswhencreatingnewapplications. vii viii Foreword If you are a visionary and plan road maps for exciting innovative applications, thisbookwillhelpyouconceptualize,design,andimplementfutureapplications. V.P.,LarsonTechnicalServices, JamesA.Larson Co-programChair, SpeechTEKConference Preface From tiny fitness trackers to huge industrial robots, we are interacting today with devices in shapes, sizes, and capabilities that would have been hard to imagine when the traditional graphical user interface (GUI) first became popular in the 1980s. As we move further and further away from the classic desktop paradigm, with input limited to mouse and keyboard, and a large screen as the only output modality,itisbecomingincreasinglyapparentthatthedecades-oldGUIinterfaceis a poor fit for today’s computer-human interactions. While the growth of touch interfaceshasbeenespeciallydramatic,wearenowalsostartingtoseeapplications that make use of many other forms of interaction, including voice, handwriting, emotionrecognition,naturallanguageunderstanding,andobjectrecognition. As these forms of interaction (modalities) are combined into systems, the importance of having standard ways for them to communicate with each other andwithapplicationlogicisapparent.Thesheervarietyandcomplexityofmulti- modal technologies makes it impractical for most implementers tohandle the full rangeofpossiblemodalities(currentandfuture)withproprietaryAPIs. Toaddressthisneed,theWorldWideWebConsortium(W3C)hasdevelopeda comprehensivesetofstandardsformultimodalinteractionwhicharewell-suitedas thebasisofinteroperablemultimodalapplications.However,mostoftheinforma- tion about these standards is currently available only in the formal standards documents, conference presentations, and a few academic journal papers. All of these can be hard to find and are not very accessible to most technologists. In addition,papersonapplicationsthatusethestandardsaresimilarlyscatteredamong manydifferentresources. Thisbookwilladdressthisgapwithclearlypresentedoverviewsofthefullsuite of W3C multimodal standards, as well as some relevant standards from other standards bodies. In addition, to illustrate the standards in use, it also includes ix x Preface descriptionsofimplementedplatformsandcasestudiesofapplicationsthatusethe standards. Finally, a future directions section discusses new ideas for other stan- dardsaswellasfornewapplications. PlymouthMeeting,PA,USA DeborahA.Dahl Acknowledgments The standards described in this book would not have been possible without the visionandsupportoftheWorldWideWebConsortium.FromthefirstW3CVoice Workshop in 1998 in Cambridge, Massachusetts where the idea of a standard language for voice applications was initially discussed, to the present, the W3C has consistently supported the goal of providing powerful, comprehensive, and usable standard languages to support voice and multimodal application develop- ment.Despitethefactthatitiseasytothinkofthewebasjustfocusedongraphical interaction, the W3C has supported work that takes a broader perspective on human-computer interaction than simply responding to graphical interaction events.Thisbroaderperspectiveincludesvoice,ink,andgesture-basedinteraction as well as the more abstract idea of interaction management. As the Chair of the Multimodal Interaction Working Group, I have had the privilege of working with manytalentedanddedicatedindividualsovertheyears.Formanyyearsofsupport, IwouldliketothankPhilippHoschka,theW3CdomainleaderfortheUbiquitous WebthroughoutthehistoryofboththeVoiceBrowserandMultimodalInteraction Working Groups. In addition, we would also like to thank Judy Brewer, domain leader fortheWeb AccessibilityInitiative,fordiscussionsofhowthesestandards canbeusedforenhancingaccessibility.WearealsoofcourseverygratefultoSir TimBerners-Lee,thedirectoroftheW3C,forinventingthewebandfoundingthe W3Cinthefirstplace. Our W3C team contacts have helped steer the Voice Browser and Multimodal Working Group chairs through the W3C process. They’ve helped the groups prepare documents, understand the stages of a W3C standard, and helped us find relevantpointsofinteractionwithotherW3Cworkinggroupsandotherstandards bodies. We are very grateful to Dave Raggett, Max Froumentin, Kazuyuki Ashimura, and Matt Wormer for their work over the years as team contacts for the Voice Browser Working Group and the Multimodal Interaction Working Group. The Voice Browser Working Group was the first W3C working group that workedonuserinterfacestandardsbeyondthegraphicaluserinterface,specifically xi
The list of books you might like
