Table Of Content

Lecture Notes in Artificial Intelligence 1572 Subseries ofLectureNotesin ComputerScience Editedby J.G. Carbonelland J. Siekmann Lecture Notes in Computer Science Editedby G.Goos,J. Hartmanisand J.van Leeuwen 3 Berlin Heidelberg NewYork Barcelona HongKong London Milan Paris Singapore Tokyo Paul Fischer Hans Ulrich Simon (Eds.) Computational Learning Theory 4th European Conference, EuroCOLT’99 Nordkirchen, Germany, March 29-31, 1999 Proceedings 1 3 SeriesEditors JaimeG.Carbonell, CarnegieMellonUniversity,Pittsburgh,PA,USA Jo¨rgSiekmann, UniversityofSaarland,Saarbru¨cken,Germany VolumeEditors PaulFischer Lehrstuhlfu¨rInformatikII,Universita¨tDortmund D-44221Dortmund,Germany E-mail:[email protected] HansUlrichSimon Fakulta¨tfu¨rMathematik,RuhrUniversita¨tBochum D-44780Bochum,Germany E-mail:[email protected] Cataloging-in-Publicationdataappliedfor DieDeutscheBibliothek-CIP-Einheitsaufnahme Computationallearningtheory:4thEuropeanconference; proceedings/EuroCOLT’99,Nordkirchen,Germany,March29-31, 1999.PaulFischer;HansUlrichSimon(ed.).-Berlin;Heidelberg; NewYork;Barcelona;HongKong;London;Milan;Paris; Singapore;Tokyo:Springer,1999 (Lecturenotesincomputerscience;1572:Lecturenotesinartificial intelligence) ISBN3-540-65701-0 CRSubjectClassification(1998):I.2.6,I.2.3,F.4.1,F.1.1,F.2 ISBN3-540-65701-0Springer-VerlagBerlinHeidelbergNewYork Thisworkissubjecttocopyright.Allrightsarereserved,whetherthewholeorpartofthematerialis concerned,specificallytherightsoftranslation,reprinting,re-useofillustrations,recitation,broadcasting, reproductiononmicrofilmsorinanyotherway,andstorageindatabanks.Duplicationofthispublication orpartsthereofispermittedonlyundertheprovisionsoftheGermanCopyrightLawofSeptember9,1965, initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer-Verlag.Violationsare liableforprosecutionundertheGermanCopyrightLaw. (cid:1)c Springer-VerlagBerlinHeidelberg1999 PrintedinGermany Typesetting:Camera-readybyauthor SPIN10703032 06/3142–543210 Printedonacid-freepaper Preface This volume contains papers presented at the Fourth European Conference on ComputationalLearningTheory,whichwasheldatNordkirchenCastle,inNord- kirchen, NRW, Germany, from March 29 to 31, 1999. This conference is the fourth in a series of bi-annual conferences established in 1993. TheEuroCOLTconferencesarefocusedontheanalysisoflearningalgorithms and the theory of machine learning, and bring together researchersfrom a wide varietyofrelated(cid:12)elds.Someofthe issuesandtopicsthatareaddressedinclude the sample and computational complexity of learning speci(cid:12)c model classes, frameworksmodeling the interaction between the learner, teacher and the envi- ronment (such as learning with queries, learning control policies and inductive inference),learningwithcomplexmodels(suchasdecisiontrees,neuralnetworks, and support vector machines), learning with minimal prior assumptions (such as mistake-bound models, universal prediction, and agnostic learning), and the study of model selection techniques. We hope that these conferences stimulate an interdisciplinary scienti(cid:12)c interaction that will be fruitful in all represented (cid:12)elds. Thirty-(cid:12)ve papers were submitted to the program committee for considera- tion, and twenty-one of these were accepted for presentation at the conference and publication in these proceedings. In addition, Robert Schapire (AT & T Labs), and Richard Sutton (AT & T Labs) were invited to give lectures and contributea writtenversionto theseproceedings.Therewereanumber ofother joint events including a banquet and an excursion to Mu¨nster. The IFIP WG 1.4 Scholarship was awarded to Andra(cid:19)s Antos for his paper \Lower bounds on the rate of convergence of nonparametric pattern recognition". The EuroCOLT’99 conference was sponsored by the DFG (Deutsche For- schungsgemeinschaft) and by IFIP through WG 1.4. We want to thank everybody who helped to make this meeting possible: the authors for submitting papers, the Program Committee and referees for their e(cid:11)ort in composing the program, the Steering Committee, the sponsors, the local organizers, and Springer-Verlag. Thanks to Norbert Klasner and Karsten Tinnefeld for their help in preparing the proceedings, and thanks to Margret Vaupelfor heradministrativesupport.The programcommittee wishesto thank everybody who acted as subreferee. January 1999 Paul Fischer and Hans Simon Conference Chairs EuroCOLT’99 Organization EUROCOLT’99isorganizedbythedepartmentofComputerScience,University of Dortmund. Organizers Paul Fischer (University of Dortmund, Germany) Hans Simon (Ruhr-Universita¨t Bochum, Germany) Steering Commitee Shai Ben-David (Chair) Technion, Haifa Nicol(cid:18)o Cesa-Bianchi,University of Milano Paul Fischer, University of Dortmund Gabor Lugosi, Pompeu Fabra University, Barcelona Arun Sharma, University of New South Wales, Sydney Hans Simon, Ruhr-Universita¨t Bochum Program Commitee Martin Anthony, London School of Economics Peter Auer, Technical University of Graz Shai Ben-David, Technion, Haifa Nader Bshouty, University of Calgary John Case, University of Delaware Nicol(cid:18)o Cesa-Bianchi,University of Milano Paul Fischer, University of Dortmund Jyrki Kivinen, University of Helsinki Nick Littlestone, NEC Research Institute, Princeton Ga(cid:19)bor Lugosi, Pompeu Fabra University, Barcelona Arun Sharma, University of New South Wales, Sydney Hans Simon, Ruhr-Universita¨t Bochum (Chair) Organization VII Subreferees Peter Bartlett Claudio Gentile Mark Pendrith Gianluca Della Vedova Tibor Hegedu}s Dale Schuurmans Nadav Eiron Christian Killin Frank Stephan Tapio Elomaa Juha Kärkkaïnen Henry Tirri Ran El-Yaniv Eric Martin Jaak Vilo Yoav Freund Petri Myllymäki Sponsoring Institutions Deutsche Forschungsgemeinsschaft(DFG) and IFIP WG 1.4. Table of Contents Invited Lectures Theoretical Views of Boosting ::::::::::::::::::::::::::::::::::::::: 1 Robert E. Schapire Open Theoretical Questions in Reinforcement Learning ::::::::::::::::: 11 Richard S. Sutton Learning from Random Examples A Geometric Approach to Leveraging Weak Learners ::::::::::::::::::: 18 Nigel Du(cid:11)y, David Helmbold Query by Committee, Linear Separation and Random Walks :::::::::::: 34 Ran Bachrach, Shai Fine, Eli Shamir Hardness Results for Neural Network Approximation Problems :::::::::: 50 Peter Bartlett, Shai Ben-David Learning from Queries and Counterexamples Learnability of Quanti(cid:12)ed Formulas :::::::::::::::::::::::::::::::::: 63 V(cid:19)(cid:16)ctor Dalmau, Peter Jeavons Learning Multiplicity Automata from Smallest Counterexamples::::::::: 79 Ju¨rgen Forster Exact Learning when Irrelevant Variables Abound ::::::::::::::::::::: 91 David Guijarro, V(cid:19)(cid:16)ctor Lav(cid:19)(cid:16)n, Vijay Raghavan An Application of Codes to Attribute{E(cid:14)cient Learning :::::::::::::::: 101 Thomas Hofmeister Learning Range Restricted Horn Expressions ::::::::::::::::::::::::: 111 Roni Khardon Reinforcement Learning On the Asymptotic Behavior of a Constant Stepsize Temporal-Di(cid:11)erence Learning Algorithm :::::::::::::::::::::::::::::::::::::::::::::::: 126 Vladislav Tadi(cid:19)c X Table of Contents On-line Learning and Expert Advice Direct and Indirect Algorithms for On-line Learning of Disjunctions ::::: 138 D. P. Helmbold, S. Panizza, M. K. Warmuth Averaging Expert Predictions ::::::::::::::::::::::::::::::::::::::: 153 Jyrki Kivinen, Manfred K. Warmuth Teaching and Learning On Teaching and Learning Intersection-ClosedConcept Classes :::::::::: 168 Christian Kuhlmann Inductive Inference Avoiding Coding Tricks by Hyperrobust Learning :::::::::::::::::::::: 183 Matthias Ott, Frank Stephan Mind Change Complexity of Learning Logic Programs:::::::::::::::::: 198 Sanjay Jain, Arun Sharma Statistical Theory of Learning and Pattern Recognition Regularized Principal Manifolds ::::::::::::::::::::::::::::::::::::: 214 Alex J. Smola, Robert C. Williamson, Sebastian Mika, Bernhard Scho¨lkopf Distribution-Dependent Vapnik-Chervonenkis Bounds :::::::::::::::::: 230 Nicolas Vayatis, Robert Azencott Lower Bounds on the Rate of Convergence of Nonparametric Pattern Recognition :::::::::::::::::::::::::::::::::::::::::::::::::::::: 241 Andr(cid:19)as Antos On Error Estimation for the Partitioning Classi(cid:12)cation Rule::::::::::::: 253 Ma(cid:19)rta Horva(cid:19)th Margin Distribution Bounds on Generalization::::::::::::::::::::::::: 263 John Shawe-Taylor, Nello Cristianini Generalization Performance of Classi(cid:12)ers in Terms of Observed Covering Numbers:::::::::::::::::::::::::::::::::::::::::::::::::::::::::: 274 John Shawe-Taylor, Robert C. Williamson Entropy Numbers, Operators and Support Vector Kernels::::::::::::::: 285 Robert C. Williamson, Alex J. Smola, Bernhard Scho¨lkopf Author Index ::::::::::::::::::::::::::::::::::::::::::::::::: 301