Artificial Neural Networks and Machine Learning – ICANN 2011: 21st International Conference on Artificial Neural Networks, Espoo, Finland, June 14-17, 2011, Proceedings, Part I

408 Pages·2011·10.377 MB·English

by  Jyri J. Kivinen, Christopher K. I. Williams (auth.), Timo Honkela, Włodzisław Duch, Mark Girolami, Samuel Kaski (eds.)

Lecture Notes in Computer Science 6791
CommencedPublicationin1973
FoundingandFormerSeriesEditors:
GerhardGoos,JurisHartmanis,andJanvanLeeuwen
EditorialBoard
DavidHutchison
LancasterUniversity,UK
TakeoKanade
CarnegieMellonUniversity,Pittsburgh,PA,USA
JosefKittler
UniversityofSurrey,Guildford,UK
JonM.Kleinberg
CornellUniversity,Ithaca,NY,USA
AlfredKobsa
UniversityofCalifornia,Irvine,CA,USA
FriedemannMattern
ETHZurich,Switzerland
JohnC.Mitchell
StanfordUniversity,CA,USA
MoniNaor
WeizmannInstituteofScience,Rehovot,Israel
OscarNierstrasz
UniversityofBern,Switzerland
C.PanduRangan
IndianInstituteofTechnology,Madras,India
BernhardSteffen
TUDortmundUniversity,Germany
MadhuSudan
MicrosoftResearch,Cambridge,MA,USA
DemetriTerzopoulos
UniversityofCalifornia,LosAngeles,CA,USA
DougTygar
UniversityofCalifornia,Berkeley,CA,USA
GerhardWeikum
MaxPlanckInstituteforInformatics,Saarbruecken,Germany
Timo Honkela Włodzisław Duch
Mark Girolami Samuel Kaski (Eds.)
Artificial Neural Networks
and Machine Learning –
ICANN 2011
21st International Conference
on Artificial Neural Networks
Espoo, Finland, June 14-17, 2011
Proceedings, Part I
1 3
VolumeEditors
TimoHonkela
AaltoUniversitySchoolofScience
DepartmentofInformationandComputerScience
P.O.Box15400,00076Aalto,Finland
E-mail:timo.honkela@aalto.fi
WłodzisławDuch
NicolausCopernicusUniversity
SchoolofPhysics,AstronomyandInformatics
DepartmentofInformatics
ul.Grudziadzka5,87-100Torun,Poland
E-mail:[email protected]
MarkGirolami
UniversityCollegeLondon
DepartmentofStatisticalScience
1-19TorringtonPlace,London,WC1E7HB,UK
E-mail:[email protected]
SamuelKaski
AaltoUniversitySchoolofScience
DepartmentofInformationandComputerScience
P.O.Box15400,00076Aalto,Finland
E-mail:samuel.kaski@aalto.fi
ISSN0302-9743 e-ISSN1611-3349
ISBN978-3-642-21734-0 e-ISBN978-3-642-21735-7
DOI10.1007/978-3-642-21735-7
SpringerHeidelbergDordrechtLondonNewYork
LibraryofCongressControlNumber:2011929230
CRSubjectClassification(1998):I.2,F.1,I.4,I.5,J.3,H.3
LNCSSublibrary:SL1–TheoreticalComputerScienceandGeneralIssues
©Springer-VerlagBerlinHeidelberg2011
Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis
concerned,specificallytherightsoftranslation,reprinting,re-useofillustrations,recitation,broadcasting,
reproductiononmicrofilmsorinanyotherway,andstorageindatabanks.Duplicationofthispublication
orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,1965,
initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer.Violationsareliable
toprosecutionundertheGermanCopyrightLaw.
Theuseofgeneraldescriptivenames,registerednames,trademarks,etc.inthispublicationdoesnotimply,
evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevantprotectivelaws
andregulationsandthereforefreeforgeneraluse.
Typesetting:Camera-readybyauthor,dataconversionbyScientificPublishingServices,Chennai,India
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
The International Conference on Artificial Neural Networks (ICANN) is the
annual flagship conference of the European Neural Network Society (ENNS).
TheideaofICANNistobringtogetherresearchersfromtwoworlds:information
sciences and neurosciences. The scope is wide, ranging from machine learning
algorithmstomodelsofrealnervoussystems.Theaimistofacilitatediscussions
andinteractionsintheefforttowarddevelopingmoreintelligentartificialsystems
andincreasingourunderstandingofneuralandcognitiveprocessesinthe brain.
In2011,ICANNreturnedtoitsrootsafter20years(formoreinformation,see
theENNSwebsitewww.e-nns.org).TheveryfirstICANNin1991wasorganized
on the premises of Helsinki University of Technology on its beautiful campus
in Espoo, Finland. For ICANN 2011, we invited all neural network researchers
worldwide to join us in celebrating this 20th anniversary of ICANN and to see
the latest advancements in our fast progressing field.
ICANN 2011 had two basic tracks: brain-inspired computing and machine
learningresearch,withProgramCommitteechairsfrombothworldsandathor-
oughreviewingsystem.The conferencestructurewasbuiltaroundplenarytalks
given by renowned scientists described briefly in the following section.
– ThomasGriffiths(UniversityofCalifornia,Berkeley,USA)istheDirectorof
theComputationalCognitiveScienceLabandtheInstituteofCognitiveand
BrainSciencesattheUniversityofCalifornia,Berkeley.Theydevelop,forin-
stance,mathematicalmodelsofhigherlevelcognition,includingprobabilistic
reasoning, learning causal relationships, acquiring and using language, and
inferring the structure of categories.
– Riitta Hari (Aalto University, Finland) is an internationally recognized and
respected neuroscientist. She was newly appointed as Academician of Sci-
ence, a title that can be held by only 12 scientists at a time in Finland.
She has developed methods and applications of human brain imaging and
contributed decisively to the progress of this branch of science. Prof. Hari’s
current focus is on the brain basis of social interaction.
– Geoffrey Hinton (University of Toronto, Canada), the first winner of the
David E. Rumelhart Prize, has provided many influential contributions to
theareaofartificialneuralnetworksandadaptivesystems.Anon-exhaustive
list of the areas where he has contributed substantial inventions includes
back-propagation algorithm, Boltzmann machines, distributed representa-
tions, time-delay neural networks, and mixtures of experts. Prof. Hinton
was the founding director of the Gatsby Computational Neuroscience Unit
at University College London.
– Aapo Hyva¨rinen (University of Helsinki, Finland) is widely known for his
contributionstothetheoryandapplicationsofindependentcomponentanal-
ysis.Hisrecentworkalsoincludesresearchonnaturalimagestatistics.Hehas
VI Preface
publishedinthe majorjournalsinthe areasofneuralnetworksandmachine
learning and his books have been translated into Japanese and Chinese.
– JohnShawe-Taylor(UniversityCollegeLondon,UK)isHeadofDepartment
ofComputerScience,andthescientificcoordinatoroftheNetworkofExcel-
lenceinPatternAnalysis,StatisticalModellingandComputationalLearning
(PASCAL).Hismainresearchareaisstatisticallearningtheory,buthiscon-
tributionsrangefromneuralnetworksandmachinelearningtographtheory.
He is the co-author of two very successful books on the theory of support
vector machines and kernel methods.
– JoshuaTenenbaum(MassachusettsInstituteofTechnology,USA)isapromi-
nent researcher in the area of computational cognitive science. With his
research group, he explores topics such as learning concepts, judging simi-
larity, inferring causal connections, forming perceptual representations, and
inferring mental states of other people.
Aspecialplenarytalk,sharedwiththeco-locatedWSOM2011,Workshopon
Self-OrganizingMaps,wasgivenbyTeuvoKohonen,AcademicianofScience.He
has introduced several new concepts to neural computing including theories of
distributed associative memory and optimal associative mappings, the learning
subspace method, the self-organizing maps (SOM), the learning vector quanti-
zation(LVQ), the adaptive-subspaceSOM (ASSOM) in which invariant-feature
filters emerge. Academician Teuvo Kohonen was the initiator and Chair of the
first ICANN conference in 1991.
The technical program of ICANN 2011 consisted of 106 oral or poster pre-
sentations that highlighted key advances in the areas of neural networks and
statistical machine learning research. The overall quality of the contributions
can be considered high, also due to the high rejection rate. Approximately only
every fourth submission was accepted to be presented orally in the conference.
Inadditionto theregularconferencesessions,onedaywasdevotedtofivework-
shops on topics related to theory and applications of brain-inspired computing
and statistical machine learning. Two of the workshops were related to special
challenges. A mind reading competition on MEG data was sponsored by the
PASCAL network of excellence. The META-NET network of excellence spon-
sored a workshopon the use of context in machine translation.
The organizers had a chance to welcome the participants to the new Aalto
University School of Science. From the beginning of 2010, the 100-year-old
Helsinki University of Technology changed its name and form. It merged with
Helsinki School of Economics and University of Art and Design Helsinki into
Aalto University, becoming the second largest university in Finland. The con-
ference was organized at Aalto University School of Science and the nearby
Dipoli CongressCenter.Bothare centrallylocatedin Otaniemi,Espoo,15 min-
utes west of Helsinki. Otaniemi features a unique mix of world-class research
organizations, academic institutions and over 800 companies from start-ups to
multinational corporations operating around a compact campus. Otaniemi has
beentwiceselectedbytheEUasoneofthemostinnovativeregionsinEurope.It
Preface VII
isacommunityofover32,000peoplewith16,000studentsand16,000technology
professionals.
We warmly thank all the authors of the contributed papers, workshop or-
ganizers and presenters. We also gratefully acknowledge the contribution of the
plenary speakers whose presentations formed the backbone of the conference.
We express our gratitude to the highly respected international Area Chairs and
members of the Program Committee whose role was instrumental for the suc-
cess of the conference. The Area Chairs, ProgramCommittee members and the
reviewers ensured a timely and thorough evaluation of the papers.
We are grateful to the members of the Executive Committee whose contri-
butions were essential in ensuring the successful organization of the conference.
Erkki Oja as the General Chair led the conference organizations with his great
experience. Amaury Lendasse, the Local Chair, kept all details of the organi-
zation under control. Mari-Sanna Paukkeri committed a lot of work to compile
the proceedings. Ilari Nieminen took care of numerous details in the arrange-
ments, especially related to the review process and compilation of the proceed-
ings. Laura Kainulainen efficiently handled the matters related to registrations
in collaboration with the Aalto University staff. Jaakko Peltonen, the Public-
ity Chair, made sure that the conference was announced in all major forums.
Alexander Ilin took goodcare of the workshoporganizations.Francesco Corona
as the Finance Chair ensured that the budget stayed in balance. Yoan Miche
contributed in several practical areas in the arrangements including the Web.
Ricardo Viga´rio was responsible for the social program and, in particular, the
arrangementsof the conference dinner. Tommi Vatanen organizedthe activities
of the conference assistants and helped in preparing the evening program.
We are grateful to Microsoft Research whose representatives provided us
with free access to their Conference Management Tool and helped in setting
up the system. Last but not least, we would like to thank Springer for their
co-operation in publishing the proceedings in the prestigious Lecture Notes in
Computer Science series.
April 2011 Timo Honkela
Wl(cid:4)odzis(cid:4)law Duch
Mark Girolami
Samuel Kaski
Organization
ICANN 2011 was held during June 14–17, 2011, organized by the Department
of Information and Computer Science, Aalto University School of Science. The
head of the department is Professor Olli Simula, who is a long-term member of
the ENNS Executive Committee.
Executive Committee
General Chair Erkki Oja (Aalto University, Finland)
ProgramChairs Wl(cid:4)odzis(cid:4)law Duch (Nicolaus Copernicus
University, Poland)
Mark Girolami (University College
London, UK)
Timo Honkela (Aalto University,
Finland)
Samuel Kaski (Aalto University,
Finland)
Local Chair Amaury Lendasse (Aalto University,
Finland)
Publicity Chair Jaakko Peltonen (Aalto University,
Finland)
Area Chairs
Peter Auer (Austria) Jan Peters (Germany)
Christian Bauckhage (Germany) Marios Polycarpou (Cyprus)
Wray Buntine (Australia) Jose Principe (USA)
Vince Calhoun (USA) Volker Roth (Switzerland)
Antonius Coolen (UK) Craig Saunders (UK)
Fernando Morgado Dias (Portugal) Masashi Sugiyama (Japan)
Barbara Hammer (Germany) Alan Stocker (USA)
Giulio Jacucci (Finland) Ron Sun (USA)
Kristian Kersting (Germany) Alfred Ultsch (Germany)
Neil Lawrence (UK) Peter Tino (UK)
Te-Won Lee (USA) Koen Van Leemput (USA)
Mikko Kurimo (Finland) Michel Verleysen (Belgium)
Hiroshi Mamitsuka (Japan) Jean-Philippe Vert (France)
Klaus-Robert Mu¨ller (Germany) Ole Winther (Denmark)
Klaus Obermayer (Germany) Chang Yoo (Korea)
Cheng Soon Ong (Switzerland)
X Organization
Program Committee and Referees
Luis Alexandre (Portugal) Gernot Fink (Germany)
Paul Aljabar (UK) Benoit Frenay (Belgium)
Tanel Alumae (Estonia) Artu Garcez (UK)
Mauricio A´lvarez (UK) Phil Garner (Switzerland)
Elisabeth Andre (Germany) Shuzhi Sam Ge (Singapore)
Alessia Annibale (UK) Mohammad Ghavamzadeh (France)
Ana Antunes (Portugal) Fausto Giunchiglia (Italy)
Bruno Apolloni (Italy) Tobias Glasmachers (Switzerland)
Ebru Arisoy (Turkey) Geoff Goodhill (Australia)
Beghdadi Azeddine (France) Dan Grollman (Switzerland)
Vineeth Balasubramaniam(USA) Moritz Grosse-Wentrup (Germany)
Peter Battaglia (USA) Hirotaka Hachiya (Japan)
Luba Benuskova (New Zealand) David Hardoon (UK)
Ulysses Bernardet (Spain) Kohei Hatano (Japan)
Monica Bianchini (Italy) Sebastien Helie (USA)
Ginestra Bianconi (USA) Ricardo Henao (Denmark)
Michael Biehl (The Netherlands) James Hensman (UK)
Benjamin Blankertz (Germany) Tom Heskes (The Netherlands)
Jamie Blundel (UK) Antti Honkela (Finland)
Antoine Bordes (Canada) Naira Hovakimyan(USA)
Abdeslam Boularias (Canada) Matthew Howard (UK)
Mikio Braun (Germany) Guang-Bin Huang (Singapore)
Sebastien Bubeck (France) Zakria Hussain (UK)
Samuel Bulo (Italy) Alexander Ilin (Finland)
Guido Caldarelli (Italy) Robert Jacob (USA)
Cristina Campi (Finland) Matjaz Jogan (USA)
Tonatiuh Pen˜a Centeno (UK) Shivaram Kalyanakrishnan(USA)
Michal Cernansky (Slovakia) Kittipat Kampa (USA)
Jung-Hsien Chiang (Taiwan) Bert Kappen (The Netherlands)
Jens Christian Claussen (Germany) Juha Karhunen (Finland)
James Cussens (UK) Matthias Kaschube (USA)
Jesse David (Belgium) Dietrich Klakow (Germany)
Lieven De Lathauwer (Belgium) Stefanos Kollias (Greece)
Andrea De Martino (Italy) Vera Kurkova (Hungary)
Yannick Deville (France) Ville Kyrki (Finland)
Dotan Di Castro (Israel) Jorma Laaksonen (Finland)
Jan Drugowitsch (France) Harri Lahdesm¨aki (Finland)
Lan Du (Australia) St´ephane Lallich (France)
Carl Ek (Sweden) Georg Langs (France)
Deniz Erdogmus (USA) Jan Larsen (Denmark)
Igor Farkas (Slovakia) Robert Leeb (Switzerland)
Jason Farquhar (The Netherlands) Amaury Lendasse (Finland)
Santo Fortunato (Italy) Weifeng Liu (USA)
Organization XI
Jyrki Lo¨tjo¨nen (Finland) Mikkel Schmidt (Denmark)
Mantas Lukosevicius (Germany) Gerwin Schalk (USA)
Zhiyuan Luo (UK) Markus Schla¨pfer (Switzerland)
Mathew Magimai Doss (Switzerland) Benoit Scherrer (USA)
Amir Madany Mamlouk (Germany) Mikkel Schmidt (Denmark)
Danilo Mandic (UK) Gabriele Schweikert (Germany)
Naoki Masuda (Japan) Sambu Seo (Germany)
Bjoern Menze (USA) Sohan Seth (USA)
Alessio Micheli (Italy) Vesa Siivola (Finland)
Kazushi Mimura (Japan) So¨ren Sonnenburg (Germany)
Tetsuro Morimura (Japan) Emilio Soria-Olivas (Spain)
Morten Mørup (Denmark) Alessandro Sperduti (Italy)
Alexander Mozeika (UK) Jochen Steil (Germany)
Christian Mu¨ller (Germany) Marc Strickert (Germany)
Shinichi Nakajima (Japan) Taiji Suzuki (Japan)
Sriraam Natarajan (USA) Csaba Szepesvari (Canada)
Tim Nattkemper (Germany) Ichiro Takeuchi (Japan)
Yizhao Ni (UK) Michael Tangermann (Germany)
Matthias Nickles (UK) Franck Thollard (France)
Nikolay Nikolaev (UK) Christian Thurau (Germany)
Ann Nowe (Belgium) Michalis Titsias (UK)
Ronald Ortner (Austria) Jussi Tohka (Finland)
Erhan Oztop (Japan) Ryota Tomioka (Japan)
Sebastian Pannasch (Germany) Antonios Tsourdos (UK)
Ulrich Paquet (UK) Koji Tsuda (Japan)
Il Park (USA) Laurens van der Maaten
(The Netherlands)
Emilio Parrado(Spain) Carmen Vidaurre (Germany)
Elzbieta Pekalska (UK) Ricardo Viga´rio (Finland)
Jaakko Peltonen (Finland) Silvia Villa (Italy)
Andrew Philippides (UK) Nathalie Villa-Vialaneix (France)
Justus Piater (Austria) Draguna Vrabie (USA)
Alex Pouget (USA) Shinji Watanabe (Japan)
Novi Quadrianto (Australia) Markus Weimer (Germany)
Sabrina Rabello (USA) Stefan Wermter (Germany)
Achim Rettinger (USA) Heiko Wersing (Germany)
Alexis Roche (Switzerland) Daan Wierstra (Switzerland)
Michael Rohs (Germany) Nick Wilson (UK)
Fabrice Rossi (France) Zhao Xu (Germany)
Volker Roth (Switzerland) Zenglin Xu (Germany)
Mert Sabuncu (USA) Makoto Yamada (Japan)
Jun Sakuma (Japan) Yoshihiro Yamanishi (Japan)
Scott Sanner (Australia) Zhirong Yang (Finland)
Ignacio Santamaria (Spain) Massimo Zancanaro (Italy)
Murat Saraclar (Turkey) Xinhua Zhang (Canada)
Jagannathan Sarangapani(USA) Shanfeng Zhu (China)
Table of Contents – Part I
Transformation Equivariant Boltzmann Machines .................... 1
Jyri J. Kivinen and Christopher K.I. Williams
Improved Learning of Gaussian-BernoulliRestricted Boltzmann
Machines ....................................................... 10
KyungHyun Cho, Alexander Ilin, and Tapani Raiko
A Hierarchical Generative Model of Recurrent Object-Based Attention
in the Visual Cortex.............................................. 18
David P. Reichert, Peggy Series, and Amos J. Storkey
(cid:2) -Penalized Linear Mixed-Effects Models for BCI.................... 26
1
Siamac Fazli, Ma´rton Dan´oczy, Ju¨rg Schelldorfer, and
Klaus Robert Mu¨ller
Slow Feature Analysis - A Tool for Extraction of Discriminating
Event-Related Potentials in Brain-Computer Interfaces ............... 36
Sven D¨ahne, Johannes Ho¨hne, Martijn Schreuder, and
Michael Tangermann
Transforming Auto-Encoders ...................................... 44
Geoffrey E. Hinton, Alex Krizhevsky, and Sida D. Wang
Stacked ConvolutionalAuto-Encoders for HierarchicalFeature
Extraction ...................................................... 52
Jonathan Masci, Ueli Meier, Dan Cire¸san, and Ju¨rgen Schmidhuber
Error-Backpropagationin Networks of Fractionally Predictive Spiking
Neurons ........................................................ 60
Sander M. Bohte
ESN Intrinsic Plasticity versus Reservoir Stability.................... 69
Petia Koprinkova-Hristova and Guenther Palm
Adaptive Routing Strategies for Large Scale Spiking Neural Network
Hardware Implementations........................................ 77
Snaider Carrillo, Jim Harkin, Liam McDaid, Sandeep Pande,
Seamus Cawley, and Fearghal Morgan
Self-Organizing Map for the Multi-Goal Path Planning with Polygonal
Goals........................................................... 85
Jan Faigl and Libor Pˇreuˇcil