Table Of ContentSPRINGER BRIEFS IN STATISTICS
JSS RESEARCH SERIES IN STATISTICS
Gareth William Peters
Tomoko Matsui Editors
Modern
Methodology and
Applications in
Spatial-Temporal
Modeling
SpringerBriefs in Statistics
JSS Research Series in Statistics
Editors-in-Chief
Naoto Kunitomo
Akimichi Takemura
Series editors
Genshiro Kitagawa
Tomoyuki Higuchi
Nakahiro Yoshida
Yutaka Kano
Toshimitsu Hamasaki
Shigeyuki Matsui
Manabu Iwasaki
ThecurrentresearchofstatisticsinJapanhasexpandedinseveraldirectionsinline
with recent trends in academic activities in the area of statistics and statistical
sciences over the globe. The core of these research activities in statistics in Japan
has been the Japan Statistical Society (JSS). This society, the oldest and largest
academicorganization for statistics inJapan, was founded in1931by ahandful of
pioneerstatisticiansandeconomistsandnowhasahistoryofabout80years.Many
distinguished scholars have been members, including the influential statistician
Hirotugu Akaike, who was a past president of JSS, and the notable mathematician
Kiyosi Itô, who was an earlier member of the Institute of Statistical Mathematics
(ISM), which has been a closely related organization since the establishment of
ISM. The society has two academic journals: the Journal of the Japan Statistical
Society (English Series) and the Journal of the Japan Statistical Society (Japanese
Series). The membership of JSS consists of researchers, teachers, and professional
statisticians in many different fields including mathematics, statistics, engineering,
medical sciences, government statistics, economics, business, psychology, educa-
tion, and many other natural, biological, and social sciences.
The JSS Series of Statistics aims to publish recent results of current research
activities in the areas of statistics and statistical sciences in Japan that otherwise
wouldnotbeavailableinEnglish;theyarecomplementarytothetwoJSSacademic
journals, both English and Japanese. Because the scope of a research paper in
academicjournalsinevitablyhasbecomenarrowlyfocusedandcondensedinrecent
years,thisseriesisintendedtofillthegapbetweenacademicresearchactivitiesand
the form of a single academic paper.
The series will be of great interest to a wide audience of researchers, teachers,
professional statisticians, and graduate students in many countries who are
interested in statistics and statistical sciences, in statistical theory, and in various
areas of statistical applications.
More information about this series at http://www.springer.com/series/13497
Gareth William Peters Tomoko Matsui
(cid:129)
Editors
Modern Methodology
and Applications
in Spatial-Temporal
Modeling
123
Editors
Gareth William Peters TomokoMatsui
Department ofStatistical Science TheInstitute of Statistical Mathematics
University CollegeLondon Tachikawa, Tokyo
London Japan
UK
ISSN 2191-544X ISSN 2191-5458 (electronic)
SpringerBriefs inStatistics
ISSN 2364-0057 ISSN 2364-0065 (electronic)
JSSResearch Series in Statistics
ISBN978-4-431-55338-0 ISBN978-4-431-55339-7 (eBook)
DOI 10.1007/978-4-431-55339-7
LibraryofCongressControlNumber:2015953790
SpringerTokyoHeidelbergNewYorkDordrechtLondon
©TheAuthor(s)2015
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpart
of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission
orinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodologynowknownorhereafterdeveloped.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfrom
therelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
The publisher, the authors and the editors are safe to assume that the advice and information in this
book are believed to be true and accurate at the date of publication. Neither the publisher nor the
authorsortheeditorsgiveawarranty,expressorimplied,withrespecttothematerialcontainedhereinor
foranyerrorsoromissionsthatmayhavebeenmade.
Printedonacid-freepaper
SpringerJapanKKispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Theideatocreatethisbookaroseasaresponsetothediscussionsandpresentations
thattookplaceinthefirstandsecondannualinternationalworkshopsonspatialand
temporal modeling (STM2013 and STM2014) and the first workshop on complex
systems modeling and estimation challenges in big data (CSM2014), all of which
were held in the Institute of Statistical Mathematics (ISM), Tokyo, Japan. These
workshopswerecohostedbyProf.TomokoMatsui(ISM)andDr.GarethW.Peters
(UCL). It was apparent after these workshops were completed that the wide range
of participants from various backgrounds including probability, statistics, applied
mathematics, physics, engineering, and signal processing as well as speech and
audioprocessinghadbeenrecentlydevelopingavarietyofnewtheory,models,and
methodsfordealingwithspatialandtemporalproblemsthatwouldbebeneficialto
document for a wider scientific audience.
Therefore,thisbookisintendedtobringtogetherarangeofnewinnovationsin
the area of spatial and temporal modeling in the form of self-contained tutorial
chapters on recent areas of research innovations. Since it is based around contri-
butions from a selection of world experts in spatial and temporal modeling who
participatedintheworkshop,itreflectsacrosssectionofspecialistinformationona
rangeofimportanttopicsinspatialandtemporalmodelingandapplication.Itisthe
aim ofsuchatext toprovideameans tomotivatefurtherresearch, discussion,and
cross fertilization of research ideas and directions among the different research
fields representative of the authors who contributed.
Whilst this book covers more of the practical and methodological aspects of
spatial-temporal modeling, its companion book, also in the Springer Briefs series,
titled Theoretical Aspects of Spatial-Temporal Modeling, complements this book
for theoreticians as it covers a range of new innovations in theoretical aspects of
modeling. The chapters in this book cover the topics summarized in the figure.
This book aims to provide a modern introductory tutorial on specialized
methodological and applied aspects of spatial and temporal modeling. The areas
covered involve a range of topics which reflect the diversity of this domain of
research across a number of quantitative disciplines. For instance, the first chapter
v
vi Preface
covers nonparametric Bayesian inference via a recently developed framework
known as kernel mean embedding that has had a significant influence in machine
learning disciplines. The second chapter covers nonparametric statistical methods
for spatial field reconstruction and exceedance probability estimation based on
Gaussianprocess-basedmodelsinthecontextofwirelesssensornetworkdata.The
third chapter covers signal processing methods applied to acoustic mood analysis
basedonmusicsignalanalysis.Thefinalchaptercoversmodelsthatareapplicable
to time series modeling in the domain of speech and language processing. This
includes aspects of factor analysis, independent component analysis in an unsu-
pervised learning setting. Then it moves to cover more advanced topics on gen-
eralized latent variable topic models based on hierarchical Dirichlet processes
which have been developed recently in nonparametric Bayesian literature.
We first note that each chapter of this book is intended to be a self-contained
research-level tutorial on modern approaches to the practical and methodological
study of some aspect of spatial and temporal statistical modeling. However, to
guide the reader in considering the sections of this book, we note the following
relationshipsbetweenchapters.Thefirstandsecondchapterscoverrecentadvances
inmachinelearning-basedmethodologiesfornonparametricestimationprocedures.
The first chapter addresses the recent topic of kernel mean embedding methods,
which are now becoming popular approaches to performing high-dimensional
state-space modeling problems as well as addressing problems with intractable
likelihood in filtering applications. These recent nonparametric inference methods
with positive definite kernels have been developed to utilize the kernel mean
expression of distributions. In this approach, the distribution of a variable is rep-
resented by the kernel mean, which is the mean element of the random feature
vectordefinedbythekernelfunction,andtherelationamongvariablesisexpressed
Preface vii
by covariance operators. This general methodology is starting to have important
applications in many spatial and temporal modeling settings.
Thesecondandthirdchaptersalsoconsidernonparametricmodels,focussingon
theclassofGaussianprocessmodels,thesecondchapterlookingatspatialmodels,
and the third chapter looking at state-space models. In the second chapter new
methods to model spatial data via combinations of Gaussian process models with
observations of mixed type, discrete, and continuous. It develops a framework for
spatial field reconstruction and establishes efficient spatial best linear unbiased
estimators for this spatial field estimation given observations. In addition, an esti-
mationframeworkbasedonacovarianceregressionmodelisestablishedtoperform
parameter estimation and introduce covariates into the spatial covariance function
structure. In the third chapter state-space models with Gaussian process state or
observation equations are considered in the application of speech and music
emotion recognition.
Thefinalchapteralsostudiesspeechandlanguageprocessing,thistimefocusing
on topic models for structural learning and temporal modeling from unlabeled
sequential patterns. The nonparametric models developed in this chapter are based
on the family of hierarchical Dirichlet processes and are considered in a Bayesian
formulation.Thechapteralsodiscusses,inadditiontoconstructionofsuchmodels,
the variational Bayes- and MCMC-based estimation procedures for such models.
Tokyo, Japan Gareth William Peters
August 2015 Tomoko Matsui
Acknowledgments
WeareverygratefultoResearchOrganizationofInformationandSystems(ROIS),
The Institute of Statistical Mathematics (ISM), UK Royal Society International
Exchange Grant, and Ministry of Education, Culture, Sports, Science and
Technology (MEXT) undertake project “Cooperation with Math Program” to
supportthefirstandsecondannualinternationalworkshopsonspatialandtemporal
modeling (STM2013 and STM2014) and the first workshop on complex systems
modelingandestimationchallengesinbigdata(CSM2014).Theideatocreatethis
bookaroseasaresponsetothediscussionsandpresentationsthattookplaceinthe
workshops.
Wewouldliketoexpressoursincerethankstoallthefollowingpresentersinthe
workshops.
(cid:129) Prof. Nourddine Azzaoui (Université Blaise Pascal)
(cid:129) Prof. Jen-Tzung Chien (National Chiao Tung University)
(cid:129) Prof. Arnaud Doucet (Oxford University)
(cid:129) Prof. Norikazu Ikoma (KIT)
(cid:129) Prof. Kenji Fukumizu (ISM)
(cid:129) Prof. Konstatin Markov (Aizu University)
(cid:129) Prof. Daichi Mochihashi (ISM)
(cid:129) Prof. Pierre Del Moral (UNSW)
(cid:129) Prof. Tor Andre Myrvoll (SINTEF)
(cid:129) Dr. Ido Nevat (Institute for Infocomm Research, A-Star)
(cid:129) Prof. Yosihiko Ogata (ERI, University of Tokyo and ISM, ROIS)
(cid:129) Dr. Takashi Owada (Technion)
(cid:129) Prof. Daniel P. Palomar (HKUST)
(cid:129) Prof. François Septier (Telecom Lille 1)
(cid:129) Prof. Taiji Suzuki (Tokyo Institute of Technology)
(cid:129) Prof. Kazuya Takeda (Nagoya University)
(cid:129) Prof. Mario Wüthrich (ETH Zurich)
ix
Contents
1 Nonparametric Bayesian Inference with Kernel
Mean Embedding. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Kenji Fukumizu
2 How to Utilize Sensor Network Data to Efficiently Perform
Model Calibration and Spatial Field Reconstruction . . . . . . . . . . . 25
Gareth W. Peters, Ido Nevat and Tomoko Matsui
3 Speech and Music Emotion Recognition
Using Gaussian Processes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
Konstantin Markov and Tomoko Matsui
4 Topic Modeling for Speech and Language Processing . . . . . . . . . . 87
Jen-Tzung Chien
xi
