Toward Brain-Computer Interfacing

520 Pages·2007·9.086 MB·English

by  Guido Dornhege, José del R. Millán, Thilo Hinterberger, Dennis J. McFarland, Klaus-Robert Müller

computer science/neuroscience D T
o o
rn w
h
a
e
g r
e d
,

Of related interest M B
il r
lá a
n i
New Directions in Statistical Signal Processing , n

From Systems to Brains H -
i C
n
Guido Dornhege is a Postdoctoral Researcher in the edited by Simon Haykin, José C. Príncipe, Terrence J. Sejnowski, and John McWhirter t o Toward
e
Intelligent Data Analysis Group at the Fraunhofer Insti- Signal processing and neural computation have separately and signiicantly inluenced many disciplines, but the cross- rb m Brain-Computer
tute for Computer Architecture and Software Technol- e
fertilization of the two ields has begun only recently. Research now shows that each has much to teach the other, as we p
r
ogy in Berlin. José del R. Millán is a Senior Researcher see highly sophisticated kinds of signal processing and elaborate hierarchical levels of neural computation performed side ge u Interfacing
at the IDIAP Research Institute in Martigny, Switzer- r t
by side in the brain. In New Directions in Statistical Signal Processing, leading researchers from both signal processing and , e
M
land, and Adjunct Professor at the Swiss Federal Insti- neural computation present new work that aims to promote interaction between the two disciplines. r edited by Guido Dornhege, José del R. Millán,
c
tute of Technology in Lausanne. Thilo Hinterberger is F I Thilo Hinterberger, Dennis J. McFarland, and
a n
with the Institute of Medical Psychology at the Univer- r Klaus-Robert Müller
Dynamical Systems in Neuroscience l t
sity of Tübingen and is a Senior Researcher at the Uni- a e
The Geometry of Excitability and Bursting n r foreword by Terrence J. Sejnowski
versity of Northampton. Dennis J. McFarland is a Re- d
Eugene M. Izhikevich , fa
search Scientist with the Laboratory of Nervous System a
In order to model neuronal behavior or to interpret the results of modeling studies, neuroscientists must call upon methods n c
Disorders, Wadsworth Center, New York State Depart- d i
of nonlinear dynamics. This book offers an introduction to nonlinear dynamical systems theory for researchers and graduate n
ment of Health. Klaus-Robert Müller is Head of the M
g
Intelligent Data Analysis Group at the Fraunhofer In- students in neuroscience. It also provides an overview of neuroscience for mathematicians who want to learn the basic facts ü
l
of electrophysiology. l
stitute and Professor in the Department of Computer e
r Toward
,
Science at the Technical University of Berlin.
e
Bayesian Brain d
i
Probabilistic Approaches to Neural Coding to
r Interest in developing an effective communication in-
edited by Kenji Doya, Shin Ishii, Alexandre Pouget, and Rajesh P. N. Rao s Brain-Computer
terface connecting the human brain and a computer has
A Bayesian approach can contribute to an understanding of the brain on multiple levels, by giving normative predictions
grown rapidly over the past decade. A brain-computer
about how an ideal sensory system should combine prior knowledge and observation, by providing mechanistic interpre-
interface (BCI) would allow humans to operate comput-
tation of the dynamic functioning of the brain circuit, and by suggesting optimal ways of deciphering experimental data.
Interfacing ers, wheelchairs, prostheses, and other devices, using
Bayesian Brain brings together contributions from both experimental and theoretical neuroscientists that examine the brain
brain signals only. BCI research may someday provide
mechanisms of perception, decision making, and motor control according to the concepts of Bayesian estimation.
a communication channel for patients with severe physi-
cal disabilities but intact cognitive functions, a working
tool in computational neuroscience that contributes to
a better understanding of the brain, and a novel inde-
Neural Information Processing series The MIT Press edited by pendent interface for human-machine communication
Massachusetts Institute of Technology that offers new options for monitoring and control. This
Cambridge, Massachusetts 02142
Guido Dornhege, José del R. Millán, volume presents a timely overview of the latest BCI
http://mitpress.mit.edu
research, with contributions from many of the impor-
Thilo Hinterberger, Dennis J. McFarland, tant research groups in the ield.
The book covers a broad range of topics, describing
and Klaus-Robert Müller work on both noninvasive (that is, without the implanta-
978-0-262-04244-4
0-262-04244-4 tion of electrodes) and invasive approaches. Other chap-
ters discuss relevant techniques from machine learning
and signal processing, existing software for BCI, and pos-
foreword by Terrence J. Sejnowski
sible applications of BCI research in the real world.
Toward Brain-Computer Interfacing
NeuralInformationProcessingSeries
MichaelI.JordanandThomasDietterich,editors
Advances in Large Margin Classifiers, Alexander J. Smola, Peter L. Bartlett, Bernhard
Scho¨lkopf,andDaleSchuurmans,eds.,2000
Advanced Mean Field Methods: Theory and Practice, Manfred Opper and David Saad,
eds.,2001
Probabilistic Models of the Brain: Perception and Neural Function, Rajesh P. N. Rao,
BrunoA.Olshausen,andMichaelS.Lewicki,eds.,2002
ExploratoryAnalysisandDataModelinginFunctionalNeuroimaging,FriedrichT.Som-
merandAndrzejWichert,eds.,2003
AdvancesinMinimumDescriptionLength:TheoryandApplications,PeterD.Grunwald,
InJaeMyung,andMarkA.Pitt,eds.,2005
Nearest-Neighbor Methods in Learning and Vision: Theory and Practice, Gregory
Shakhnarovich,PiotrIndyk,andTrevorDarrell,eds.,2006
NewDirectionsinStatisticalSignalProcessing:FromSystemstoBrains,SimonHaykin,
Jose´ C.Pr´ıncipe,TerrenceJ.Sejnowski,andJohnMcWhirter,eds.,2007
PredictingStructuredData,Go¨khanBakir,ThomasHofmann,BernardScho¨lkopf,Alexan-
derJ.Smola,BenTaskar,S.V.N.Vishwanathan,eds.,2007
Toward Brain-Computer Interfacing, Guido Dornhege, Jose´ del R. Milla´n, Thilo Hinter-
berger,DennisJ.McFarland,Klaus-RobertMu¨ller,eds.,2007
LargeScaleKernelMachines,Le´onBottou,OlivierChapelle,DenisDeCoste,JasonWest-
man,eds.,2007
Toward Brain-Computer Interfacing
editedby
GuidoDornhege
Jose´ delR.Milla´n
ThiloHinterberger
DennisJ.McFarland
Klaus-RobertMu¨ller
forewordby
TerrenceJ.Sejnowski
ABradfordBook
TheMITPress
Cambridge,Massachusetts
London,England
(cid:2)c 2007MassachusettsInstituteofTechnology
Allrightsreserved.Nopartofthisbookmaybereproducedinanyformbyanyelectronicormechanicalmeans
(includingphotocopying,recording,orinformationstorageandretrieval)withoutpermissioninwritingfromthe
publisher.
Forinformationaboutspecialquantitydiscounts,[e-mail].
ThisbookwassetinLaTexbytheauthors.
PrintedandboundintheUnitedStatesofAmerica.
LibraryofCongressCataloging-in-PublicationData
Towardbrain-computerinterfacing/editedbyGuidoDornhege...[etal.];forewordbyTerrenceJ.Sejnowski.
p.; cm. –(Neuralinformationprocessingseries)
”ABradfordbook.”
Includesbibliographicalreferencesandindex.
ISBN978-0-262-04244-4(hardcover:alk.paper)
1.Brain-computerinterfaces.I.Dornhege,Guido.II.Series.
[DNLM: 1. Brain Mapping. 2. User-Computer Interface. 3. Brain–physiology. 4. Psychomotor Performance.
5.Rehabilitation–instrumentation.WL335T7372007]
QP360.7.T692007
612.8’2–dc22
2007000517
10987654321
Contents
Foreword . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
TerrenceJ.Sejnowski
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
1 AnIntroductiontoBrain-ComputerInterfacing . . . . . . . . . . . . . . 1
AndreaKu¨blerandKlaus-RobertMu¨ller
I BCISystemsandApproaches 27
2 NoninvasiveBrain-ComputerInterfaceResearchattheWadsworthCenter 31
EricW.Sellers,DeanJ.Krusienski,DennisJ.McFarland,
andJonathanR.Wolpaw
3 Brain-Computer Interfaces for Communication in Paralysis: A Clinical
ExperimentalApproach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
ThiloHinterberger,FemkeNijboer,AndreaKu¨bler,
TamaraMatuz,AdrianFurdea,UrsulaMochty,MiguelJordan,
ThomasNavinLal,N.JeremyHill,Ju¨rgenMellinger,
MichaelBensch,MichaelTangermann,GuidoWidman,
ChristianE.Elger,WolfgangRosenstiel,Bernhard
Scho¨lkopf,andNielsBirbaumer
4 Graz-Brain-ComputerInterface:StateofResearch . . . . . . . . . . . . 65
GertPfurtscheller,GernotR.Mu¨ller-Putz,AloisSchlo¨gl,
BernhardGraimann,ReinholdScherer,RobertLeeb,
ClemensBrunner,ClaudiaKeinrath,GeorgeTownsend,
CarmenVidaurre,MuhammadNaeem,FelixY.Lee,
SelinaWriessnegger,DorisZimmermann,EvaHo¨fler,
andChristaNeuper
vi Contents
5 The Berlin Brain-Computer Interface: Machine Learning-Based Detection
ofUserSpecificBrainStates . . . . . . . . . . . . . . . . . . . . . . . . . 85
BenjaminBlankertz,GuidoDornhege,MatthiasKrauledat,
VolkerKunzmann,FlorianLosch,GabrielCurio,
andKlaus-RobertMu¨ller
6 The IDIAP Brain-Computer Interface: An Asynchronous Multiclass
Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
Jose´ delR.Milla´n,PierreW.Ferrez,andAnnaButtfield
7 BrainInterfaceDesignforAsynchronousControl . . . . . . . . . . . . . 111
JaimieF.Borisoff,SteveG.Mason,andGaryE.Birch
II InvasiveBCIApproaches 123
8 ElectrocorticogramasaBrain-ComputerInterfaceSignalSource . . . . 129
JaneE.Huggins,BernhardGraimann,SeYoungChun,
JeffreyA.Fessler,andSimonP.Levine
9 Probabilistically Modeling and Decoding Neural Population Activity in
MotorCortex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
MichaelJ.BlackandJohnP.Donoghue
10 TheImportanceofOnlineErrorCorrectionandFeed-ForwardAdjustments
inBrain-MachineInterfacesforRestorationofMovement . . . . . . . . . 161
DawnM.Taylor
11 Advances in Cognitive Neural Prosthesis: Recognition of Neural Data with
anInformation-TheoreticObjective . . . . . . . . . . . . . . . . . . . . . 175
ZoranNenadic,DanielS.Rizzuto,RichardA.Andersen,
andJoelW.Burdick
12 A Temporal Kernel-Based Model for Tracking Hand Movements from
NeuralActivities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
LaviShpigelman,KobyCrammer,RonyPaz,EilonVaadia,
andYoramSinger
III BCITechniques 203
13 GeneralSignalProcessingandMachineLearningToolsforBCIAnalysis 207
GuidoDornhege,MatthiasKrauledat,Klaus-RobertMu¨ller,
andBenjaminBlankertz
Contents vii
14 Classifying Event-Related Desynchronization in EEG, ECoG, and MEG
Signals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
N.JeremyHill,ThomasNavinLal,MichaelTangermann,
ThiloHinterberger,GuidoWidman,ChristianE.Elger,
BernhardScho¨lkopf,andNielsBirbaumer
15 Classification of Time-Embedded EEG Using Short-Time Principal
ComponentAnalysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
CharlesW.Anderson,MichaelJ.Kirby,DouglasR.Hundley,
andJamesN.Knight
16 Noninvasive Estimates of Local Field Potentials for Brain-Computer
Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
RolandoGravedePeraltaMenendez,SaraGonzalezAndino,
PierreW.Ferrez,andJose´ delR.Milla´n
17 Error-RelatedEEGPotentialsinBrain-ComputerInterfaces . . . . . . . 291
PierreW.FerrezandJose´ delR.Milla´n
18 AdaptationinBrain-ComputerInterfaces . . . . . . . . . . . . . . . . . . 303
Jose´ delR.Milla´n,AnnaButtfield,CarmenVidaurre,
MatthiasKrauledat,AloisSchlo¨gl,PradeepShenoy,
BenjaminBlankertz,RajeshP.N.Rao,RafaelCabeza,
GertPfurtscheller,andKlaus-RobertMu¨ller
19 EvaluationCriteriaforBCIResearch . . . . . . . . . . . . . . . . . . . . 327
AloisSchlo¨gl,JulienKronegg,JaneE.Huggins,andSteveG.Mason
IV BCISoftware 343
20 BioSig:AnOpen-SourceSoftwareLibraryforBCIResearch . . . . . . . 347
AloisSchlo¨gl,ClemensBrunner,ReinholdScherer,
andAndreasGlatz
21 BCI2000:AGeneral-PurposeSoftwarePlatformforBCI . . . . . . . . . 359
Ju¨rgenMellingerandGerwinSchalk
V Applications 369
22 Brain-Computer Interfaces for Communication and Motor
Control—PerspectivesonClinicalApplications . . . . . . . . . . . . . . . 373
AndreaKu¨bler,FemkeNijboer,andNielsBirbaumer
viii Contents
23 CombiningBCIandVirtualReality:ScoutingVirtualWorlds . . . . . . 393
RobertLeeb,ReinholdScherer,DoronFriedman,
FelixY.Lee,ClaudiaKeinrath,HorstBischof,
MelSlater,andGertPfurtscheller
24 Improving Human Performance in a Real Operating Environment through
Real-TimeMentalWorkloadDetection . . . . . . . . . . . . . . . . . . . 409
JensKohlmorgen,GuidoDornhege,MikioL.Braun,
BenjaminBlankertz,Klaus-RobertMu¨ller,GabrielCurio,
KonradHagemann,AndreasBruns,MichaelSchrauf,
andWilhelmE.Kincses
25 Single-TrialAnalysisofEEGduringRapidVisualDiscrimination:Enabling
CorticallyCoupledComputerVision . . . . . . . . . . . . . . . . . . . . . 423
PaulSajda,AdamD.Gerson,MariosG.Philiastides,
andLucasC.Parra
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 491
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 503
Foreword
The advances in brain-computer interfaces in this book could have far-reaching conse-
quences for how we interact with the world around us. A communications channel that
bypasses the normal motor outflow from the brain will have an immediate benefit for
paraplegic patients. Someday the same technology will allow humans to remotely con-
trolagentsinexoticenvironments,whichwillopennewfrontiersthatwecanonlydimly
imaginetoday.
The earliest systems to be developed were based on noninvasive electroencephalo-
graphic(EEG)recordings.Becausethesesystemsdonotrequireinvasivesurgicalimplants,
they can be used for a wide range of applications. The disadvantage is the relatively low
rate of signaling that can be achieved. Nonetheless, advances in signal processing tech-
niquesandthedevelopmentofdryelectrodesmakethisanattractiveapproach.
Three separate research areas have contributed to major advances in invasive brain-
computer interfaces. First, the neural code for motor control was uncovered based on
recordingsfromsingleneuronsindifferentcorticalareasofalertprimates.Thesecondwas
the development of mathematical algorithms for converting the train of spikes recorded
from populations of these neurons to an intended action, called the decoding problem.
Third,itwasnecessarytoachievestable,long-termrecordingsfromsmall,corticalneurons
inaharshaqueousenvironment.
For both invasive and noninvasive BCIs interdisciplinary teams of scientists and engi-
neersneededtoworkcloselytogethertocreatesuccessfulsystems.
Successinbrain-computerinterfaceshasalsodependedontheremarkableabilityofthe
braintoadapttounusualtasks,nonemorechallengingthan“mindcontrol”ofextracorpo-
real space. We are still at an early stage of development, but the field is moving forward
rapidlyandwecanconfidentlyexpectfurtheradvancesinthenearfuture.
TerrenceJ.Sejnowski
LaJolla,CA