Towards Practical Brain-Computer Interfaces: Bridging the Gap from Research to Real-World Applications

423 Pages·2013·8.4 MB·English

by  Brendan Z. Allison

biological and medical physics,
biomedical engineering
Forfurthervolumes:
http://www.springer.com/series/3740
biological and medical physics,
biomedical engineering
Thefieldsofbiologicalandmedicalphysicsandbiomedicalengineeringarebroad,multidisciplinaryand
dynamic.Theylieatthecrossroadsoffrontierresearchinphysics,biology,chemistry,andmedicine.The
BiologicalandMedicalPhysics,BiomedicalEngineeringSeriesisintendedtobecomprehensive,coveringa
broadrangeoftopicsimportanttothestudyofthephysical,chemicalandbiologicalsciences.Itsgoalisto
providescientistsandengineerswithtextbooks,monographs,andreferenceworkstoaddressthegrowing
needforinformation.
Booksintheseriesemphasizeestablishedandemergentareasofscienceincludingmolecular,membrane,
andmathematicalbiophysics;photosyntheticenergyharvestingandconversion;informationprocessing;
physicalprinciplesofgenetics;sensorycommunications;automatanetworks,neuralnetworks,andcellu-
larautomata.Equallyimportantwillbecoverageofappliedaspectsofbiologicalandmedicalphysicsand
biomedicalengineeringsuchasmolecularelectroniccomponentsanddevices,biosensors,medicine,imag-
ing,physicalprinciplesofrenewableenergyproduction,advancedprostheses,andenvironmentalcontroland
engineering.
Editor-in-Chief:
EliasGreenbaum,OakRidgeNationalLaboratory,OakRidge,Tennessee,USA
EditorialBoard:
MasuoAizawa,DepartmentofBioengineering, MarkS.Humayun,DohenyEyeInstitute,
TokyoInstituteofTechnology,Yokohama,Japan LosAngeles,California,USA
OlafS.Andersen,DepartmentofPhysiology, PierreJoliot,InstitutedeBiologie
Biophysics&MolecularMedicine, Physico-Chimique,FondationEdmond
CornellUniversity,NewYork,USA deRothschild,Paris,France
RobertH.Austin,DepartmentofPhysics, LajosKeszthelyi,InstituteofBiophysics,Hungarian
PrincetonUniversity,Princeton,NewJersey,USA AcademyofSciences,Szeged,Hungary
JamesBarber,DepartmentofBiochemistry, RobertS.Knox,DepartmentofPhysics
andAstronomy,UniversityofRochester,Rochester,
ImperialCollegeofScience,Technology
NewYork,USA
andMedicine,London,England
AaronLewis,DepartmentofAppliedPhysics,
HowardC.Berg,DepartmentofMolecular
HebrewUniversity,Jerusalem,Israel
andCellularBiology,HarvardUniversity,
Cambridge,Massachusetts,USA StuartM.Lindsay,DepartmentofPhysics
VictorBloomfield,DepartmentofBiochemistry, andAstronomy,ArizonaStateUniversity,
UniversityofMinnesota,St.Paul,Minnesota,USA Tempe,Arizona,USA
RobertCallender,DepartmentofBiochemistry, DavidMauzerall,RockefellerUniversity,
NewYork,NewYork,USA
AlbertEinsteinCollegeofMedicine,
Bronx,NewYork,USA EugenieV.Mielczarek,DepartmentofPhysics
andAstronomy,GeorgeMasonUniversity,Fairfax,
StevenChu,LawrenceBerkeleyNational
Virginia,USA
Laboratory,Berkeley,California,USA
MarkolfNiemz,MedicalFacultyMannheim,
LouisJ.DeFelice,DepartmentofPharmacology,
UniversityofHeidelberg,Mannheim,Germany
VanderbiltUniversity,Nashville,Tennessee,USA
V.AdrianParsegian,PhysicalScienceLaboratory,
JohannDeisenhofer,HowardHughesMedical
NationalInstitutesofHealth,Bethesda,
Institute,TheUniversityofTexas,Dallas,
Maryland,USA
Texas,USA
LindaS.Powers,UniversityofArizona,
GeorgeFeher,DepartmentofPhysics,
Tucson,Arizona,USA
UniversityofCalifornia,SanDiego,LaJolla,
California,USA EarlW.Prohofsky,DepartmentofPhysics,
PurdueUniversity,WestLafayette,Indiana,USA
HansFrauenfelder,
LosAlamosNationalLaboratory, AndrewRubin,DepartmentofBiophysics,Moscow
LosAlamos,NewMexico,USA StateUniversity,Moscow,Russia
IvarGiaever,RensselaerPolytechnicInstitute, MichaelSeibert,NationalRenewableEnergy
Troy,NewYork,USA Laboratory,Golden,Colorado,USA
SolM.Gruner,CornellUniversity, DavidThomas,DepartmentofBiochemistry,
Ithaca,NewYork,USA UniversityofMinnesotaMedicalSchool,
Minneapolis,Minnesota,USA
JudithHerzfeld,DepartmentofChemistry,
BrandeisUniversity,Waltham,Massachusetts,USA
Brendan Z. A llison • Stephen Dunne
Robert Leeb • José del R. Millán
Anton Nijholt
Editors
Towards Practical
Brain-Computer
Interfaces
Bridging the Gap from Research to
Real-World Applications
With 107 Figures
123
Editors
Brendan Z . Allison Anton Nijholt
Graz University of Technology University of Twente
Austria Enschede, The Netherlands
Stephen Dunne
StarLab Barcelona
Spain
Robert Leeb
José del R. Millán
Swiss Federal Institute of Technology Lausanne
Switzerland
BiologicalandMedicalPhysics,BiomedicalEngineering ISSN1618-7210
ISBN978-3-642-29745-8 ISBN978-3-642-29746-5 (eBook)
DOI10.1007/978-3-642-29746-5
SpringerHeidelbergNew York DordrechtLondon
Library of Congress Control Number: 2012944560
© Springer-Verlag Berlin Heidelberg 2012
This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
or information storage and retrieval, electronic adaptation, computer software, or by similar or
dissimilar methodology now known or hereafter developed. Exempted from this legal reservation
are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically
for the purpose of being entered and executed on a computer system, for exclusive use by the
purchaser of the work. Duplication of this publication or parts thereof is permitted only under the
provisions of the Copyright Law of the Publisher’s location, in its current version, and permission
for use must always be obtained from Springer. Permissions for use may be obtained through
RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the
respective Copyright Law.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publication does not imply, even in the absence of a specific statement, that such names are exempt
from the relevant protective laws and regulations and therefore free for general use.
While the advice and information in this book are believed to be true and accurate at the date of
publication, neither the authors nor the editors nor the publisher can accept any legal responsibility
for any errors or omissions that may be made. The publisher makes no warranty, express or implied,
with respect to the material contained herein.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)
Preface
Brain–computerinterface (BCI) research is advancing rapidly. The last few years
have seen a dramatic rise in journal publications, academic workshops and con-
ferences, books, new products aimed at both healthy and disabled users, research
funding from different sources, and media attention. This media attention has
included both BCI fi (BCI-based science fiction) and stories in mainstream mag-
azinesandtelevisionnewsprograms.
Despite this progress and attention, most people still do not use BCIs, or even
knowwhattheyare.Whiletheauthorsofthisbookgenerallyhaveaccesstothebest
BCI equipment,and they know how to use it, the chapters are written in the old-
fashioned way, with keyboardsand mice instead of BCIs. This may be surprising
because BCIs are generally presented inaccurately in the popular media, where
undeservedhypeandsloppyreportingoftencreateagapbetweenexpectationsand
reality.
This book aims to bridge that gap by educating readers about BCIs, with
emphasisonmakingBCIspracticalinreal-worldsettings.ExpertsinBCIresearch
widely agree that one of the major challenges in the field is moving BCIs from
laboratory gadgets that work with some healthy users to tools that are reliable,
straightforward,andusefulinfieldsettingsforwhoeverneedsthem.Manyofthese
expertsdiscussthestateoftheartandmajorchallengesacrossfoursections.Three
of the sections address the three main components of BCIs: sensors, signals, and
signalprocessing;devicesand applications;and interfacesand environments.The
lastsectionsummarizesotherchallengesthatrelatetocompleteBCIsystemsinstead
ofonecomponent.
BCI research is inherently interdisciplinary, requiring contributions from neu-
roscience,psychology,medicine,human–computerinteraction(HCI),manyfacets
of engineering,and other disciplines. Similarly, manysectors are involvedin BCI
research, including academia, small and large businesses, government, medicine,
and different types of nonprofit institutions. The authors who contributed to this
book represent an eclectic mix of these disciplines and sectors. This breadth of
contributorsprovidesdifferentperspectivesandshouldmakethisbookrelevanttoa
widevarietyofreaders.
v
vi Preface
However, while this book could be useful for different specialists in the BCI
community, we also made a strong effort to keep the chapters practical and
readablefor peoplewhodonothavea backgroundin BCI researchor anyrelated
discipline. Chapters are written in plain English, without unnecessary technical
detail, and acronyms and special terms are defined within chapters and in our
glossary. Ample references are provided in case readers want more information.
Hence, many readers outside of the conventionalBCI community may enjoy this
book for different reasons. Nurses, doctors, therapists, caretakers, and assistive
technologypractitionersmay want to learn more about what real-world BCIs can
(and cannot) do, which may help them decide whether a BCI is viable as an
assistive technology. Other readers may instead be curious about BCIs for other
user groups, including healthy users. Students might use this book to learn about
BCIs, andteachersmightassignchaptersinrelevantclasses. Businessexpertsand
policymakersmaywant to learn moreaboutwhetherBCIs are promisingenough
to merit additional funding through commercial investment or grants. Journalists,
writers, or other people interested in developing articles, documentaries, or other
shows might find helpful backgroundinformation or inspiration here. Finally, we
hopeourbookappealsto peoplewhoare justcuriousabouta technologythathas
longcapturedthe humanimaginationandcouldrevolutionizehow peopleinteract
witheachotherandtheirenvironments.
Acknowledgements:Theeditorsgratefullyacknowledgethehelpofthefollowing
chapterreviewers:TomCarlson,Gu¨nterEdlinger,JanvanErp,ShangkaiGao,Gary
Garcia Molina, GangadharGaripelli, Cuntai Guan, David Iban˜ez, Andrea Ku¨bler,
BramVandeLaar,FabienLotte,MassimilianoMalavasi,BehnamMolaee,Roder-
ickMurray-Smith,TimMullen,FemkeNijboer,DaniPerezMarcos,MannesPoel,
Aureli Soria-Frisch, Olga Sourina, Michael Tangermann, Aleksander Va¨ljama¨e,
YijunWang, TomasWard,andThorstenZander.Theiroftenextensiveandalways
careful comments certainly helped the authors in improving the chapters. The
editorsalsowanttoexpresstheirgratitudetotheir“technicaleditor,”HendriHon-
dorp from the HMI group of the University of Twente, for improvinguniformity,
consistency,andcompletenessofthebook.Finally,preparationofmanychaptersin
thisbookhasbenefitedfromfundingfromtheEuropeanUnionSeventhFramework
Programme(FP7/2007-2013).Inparticulartheeditorsgratefullyacknowledgethe
supportoftheFutureBNCIproject(ProjectnumberICT-248320).
Enschede BrendanZ.Allison
November2011 (GrazUniversityofTechnology,Austria)
StephenDunne
(StarLabBarcelona,Spain)
RobertLeeb
(E´colePolytechniqueFe´de´raledeLausanne,Switserland)
Jose´ delR.Milla´n
(E´colePolytechniqueFe´de´raledeLausanne,Switserland)
AntonNijholt
(UniversityofTwente,TheNetherlands)
Contents
1 RecentandUpcomingBCIProgress:Overview,Analysis,
andRecommendations..................................................... 1
BrendanZ.Allison,StephenDunne,RobertLeeb,Jose´ delR.
Milla´n,andAntonNijholt
1.1 Introduction......................................................... 1
1.2 OverviewofThisBook............................................. 2
1.2.1 OverviewofSectionOne................................. 3
1.2.2 OverviewofSectionTwo................................. 4
1.2.3 OverviewofSectionThree ............................... 6
1.2.4 OverviewofSectionFour................................. 7
1.3 PredictionsandRecommendations................................. 8
1.4 Summary............................................................ 11
References.................................................................... 12
PartI Sensors,SignalsandSignalProcessing
2 HybridOptical–ElectricalBrainComputerInterfaces,
PracticesandPossibilities.................................................. 17
TomasE.Ward
2.1 Introduction......................................................... 17
2.2 TheUnderlyingPhysiologicalOriginsofEEGandfNIRS....... 17
2.2.1 OriginoftheEEG......................................... 18
2.2.2 OriginoffNIRSResponses............................... 19
2.3 SignalModels....................................................... 28
2.3.1 ModellingtheVascularResponse........................ 28
2.3.2 SpectrophotometricTranslation .......................... 30
2.3.3 SyntheticSignalGeneration.............................. 31
vii
viii Contents
2.4 CombinedEEG-fNIRSMeasurementsin Overt
andImaginedMovementTasks .................................... 33
2.4.1 fNIRS/EEGSensor........................................ 33
2.4.2 ExperimentalDescription................................. 33
2.4.3 SignalProcessing ......................................... 34
2.4.4 Results..................................................... 35
2.5 Conclusion .......................................................... 37
References.................................................................... 38
3 ACriticalReviewontheUsageofEnsemblesforBCI................. 41
AureliSoria-Frisch
3.1 Introduction......................................................... 41
3.2 TheoreticalBackground............................................ 43
3.2.1 PatternRecognitionEnsembleDefinition
andContext................................................ 43
3.2.2 PatternRecognitionPerspectiveonFusion.............. 44
3.2.3 GroundingtheSuperiorityofEnsembles ................ 46
3.3 IntegrationandFusionLevel....................................... 47
3.3.1 FeatureConcatenation .................................... 47
3.3.2 ClassificationConcatenation.............................. 48
3.3.3 ClassificationFusion...................................... 49
3.3.4 DecisionFusion ........................................... 50
3.4 EnsembleType...................................................... 51
3.4.1 ClassifierEnsembles...................................... 51
3.4.2 StackedEnsemble......................................... 52
3.4.3 Multi-ChannelEnsemble ................................. 52
3.4.4 MultimodalEnsemble..................................... 52
3.5 ResamplingStrategies.............................................. 52
3.5.1 DataSetPartitioning...................................... 53
3.5.2 FeatureSpacePartitioning................................ 56
3.5.3 SignalPartitioning ........................................ 57
3.6 FusionOperators.................................................... 57
3.6.1 SampleBasedFusion ..................................... 58
3.6.2 TimeDomainFusionOperators.......................... 59
3.7 SummaryofEnsemblesObtainedResults......................... 59
3.8 FinalRemarks....................................................... 60
References.................................................................... 62
4 Improving Brain–Computer Interfaces Using
IndependentComponentAnalysis........................................ 67
YijunWangandTzyy-PingJung
4.1 Introduction......................................................... 67
4.2 ICAinEEGSignalProcessing..................................... 68
4.3 ICAinBCISystems................................................ 69
4.3.1 ArtifactRemoval.......................................... 71
4.3.2 SNREnhancementofTask-RelatedEEGSignals....... 72
4.3.3 ElectrodeSelection........................................ 73
Contents ix
4.4 ICA-BasedZero-Training-TrainingBCI........................... 75
4.4.1 ExperimentandDataRecording.......................... 75
4.4.2 Method..................................................... 76
4.4.3 Results..................................................... 78
4.5 DiscussionandConclusion......................................... 80
References.................................................................... 81
5 TowardsElectrocorticographicElectrodesforChronic
UseinBCIApplications ................................................... 85
ChristianHenle,MartinSchuettler,Jo¨rnRickert,andThomas
Stieglitz
5.1 Introduction:From Presurgical Diagnostics
toMovementDecoding............................................. 85
5.2 ApproachesandTechnologiesforECoG-Electrodes ............. 88
5.3 ECoGRecordingsinBCIStudies.................................. 91
5.4 HighChannelECoGArraysforBCI .............................. 92
5.4.1 ManufacturingofLaserStructuredElectrodes .......... 93
5.4.2 BiologicalEvaluation/ResultsfromFirstStudies ....... 95
5.5 TowardsChronicWirelessSystems................................ 97
References.................................................................... 100
PartII Devices,ApplicationsandUsers
6 IntroductiontoDevices,ApplicationsandUsers:Towards
PracticalBCIsBasedonSharedControlTechniques .................. 107
RobertLeebandJose´ d.R.Milla´n
6.1 Introduction......................................................... 107
6.2 CurrentandEmergingUserGroups ............................... 109
6.3 BCIDevicesandApplicationScenarios........................... 109
6.3.1 CommunicationandControl.............................. 110
6.3.2 MotorSubstitution:GraspRestoration................... 111
6.3.3 EntertainmentandGaming ............................... 113
6.3.4 MotorRehabilitationandMotorRecovery .............. 113
6.3.5 MentalStateMonitoring.................................. 114
6.3.6 HybridBCI................................................ 114
6.4 PracticalBCIsBasedonSharedControlTechniques:
TowardsControlofMobility....................................... 115
6.4.1 Tele-Presence Robot Controlled by
Motor-DisabledPeople ................................... 116
6.4.2 BCIControlledWheelchair............................... 118
6.5 AdaptationofGestureRecognitionSystemsUsing
EEGErrorPotentials ............................................... 120
6.6 Conclusion .......................................................... 122
References.................................................................... 123