INR-Cover.qxd 10/11/2006 8:41 AM Page 1 F n T IN Foundations and Trends® in R 1:1 Information Retrieval Music Retrieval: A Tutorial and Review M 1:1 (2006) u s ic Nicola Orio R e tr ie Music Retrieval: A Tutorial and Review is the first comprehensive survey of the vast new field v a of Music Information Retrieval (MIR). It describes a number of issues which are peculiar to the l: A language of music – including forms, formats, and dimensions of music – together with the T u Music Retrieval: typologies of users and their information needs. To fulfil these needs a number of approaches to r are discussed, from direct search to information filtering and clustering of music documents. ial a A Tutorial and Review The emphasis is on tools, techniques, and approaches for content-based MIR, rather than on n d the systems that implement them. The interested reader can, however, find descriptions of R e more than 35 systems for music retrieval with links to their Web sites. v ie w Nicola Orio Music Retrieval: A Tutorial and Review can be used as both a guide for beginners who are embarking on research in this relatively new area, and a useful reference for established N ic researchers in this field. o la O r io This book is originally published as Foundations and Trends®in Information Retrieval, Volume 1 Issue 1 (2006), ISSN: 1554-0669. n now o w the essence of knowledge Music Retrieval: A Tutorial and Review Music Retrieval: A Tutorial and Review Nicola Orio Department of Information Engineering University of Padova Via Gradenigo, 6/b, Padova 35131, Italy [email protected] Boston – Delft Foundations and Trends(cid:13)R in Information Retrieval Published, sold and distributed by: now Publishers Inc. PO Box 1024 Hanover, MA 02339 USA Tel. +1-781-985-4510 www.nowpublishers.com [email protected] Outside North America: now Publishers Inc. PO Box 179 2600 AD Delft The Netherlands Tel. +31-6-51115274 Library of Congress Control Number: 2006935708 ThepreferredcitationforthispublicationisNicolaOrio,MusicRetrieval:ATutorial andReview,FoundationandTrends(cid:13)R inInformationRetrieval,vol1,no1,pp1–96, November 2006 Printed on acid-free paper ISBN: 1-933019-39-5 (cid:13)c November 2006 Nicola Orio All rights reserved. No part of this publication may be reproduced, stored in a retrieval system,ortransmittedinanyformorbyanymeans,mechanical,photocopying,recording orotherwise,withoutpriorwrittenpermissionofthepublishers. Photocopying. In the USA: This journal is registered at the Copyright Clearance Cen- ter, Inc., 222 Rosewood Drive, Danvers, MA 01923. Authorization to photocopy items for internal or personal use, or the internal or personal use of specific clients, is granted by now Publishers Inc for users registered with the Copyright Clearance Center (CCC). The ‘services’foruserscanbefoundontheinternetat:www.copyright.com For those organizations that have been granted a photocopy license, a separate system of payment has been arranged. Authorization does not extend to other kinds of copy- ing, such as that for general distribution, for advertising or promotional purposes, for creating new collective works, or for resale. In the rest of the world: Permission to pho- tocopy must be obtained from the copyright owner. Please apply to now Publishers Inc., PO Box 1024, Hanover, MA 02339, USA; Tel. +1 781 871 0245; www.nowpublishers.com; [email protected] nowPublishersInc.hasanexclusivelicensetopublishthismaterialworldwide.Permission tousethiscontentmustbeobtainedfromthecopyrightlicenseholder.Pleaseapplytonow Publishers,POBox179,2600ADDelft,TheNetherlands,www.nowpublishers.com;e-mail: [email protected] Foundations and Trends(cid:13)R in Information Retrieval Volume 1 Issue 1, November 2006 Editorial Board Editors-in-Chief: Fabrizio Sebastiani Consiglio Nazionale delle Ricerche [email protected] Jamie Callan Carnegie Mellon University [email protected] Editors Alan Smeaton (Dublin City University) Andrei Z. Broder (Yahoo! Research) Bruce Croft (University of Massachusetts, Amherst) Charles L.A. Clarke (University of Waterloo) Ellen Voorhees (National Institute of Standards and Technology) Ian Ruthven (University of Strathclyde, Glasgow) James Allan (University of Massachusetts, Amherst) Justin Zobel (RMIT University, Melbourne) Karen Sparck Jones (University of Cambridge) Maarten de Rijke (University of Amsterdam) Marcello Federico (ITC-irst) Norbert Fuhr (University of Duisburg-Essen) Soumen Chakrabarti (Indian Institute of Technology) Susan Dumais (Microsoft Research) Wei-Ying Ma (Microsoft Research Asia) William W. Cohen (CMU) Editorial Scope Foundations and Trends(cid:13)R in Information Retrievalwillpublish survey and tutorial articles in the following topics: • Applications of IR • Metasearch, rank aggregation and • Architectures for IR data fusion • Collaborative filtering and • Natural language processing for IR recommender systems • Performance issues for IR systems, • Cross-lingual and multilingual IR including algorithms, data structures, optimization • Distributed IR and federated techniques, and scalability search • Question answering • Evaluation issues and test collections for IR • Summarization of single documents, multiple documents, • Formal models and language and corpora models for IR • Text mining • IR on mobile platforms • Topic detection and tracking • Indexing and retrieval of structured documents • Usability, interactivity, and visualization issues in IR • Information categorization and clustering • User modelling and user studies for IR • Information extraction • Web search • Information filtering and routing Information for Librarians Foundations and Trends(cid:13)R in Information Retrieval, November 2006, Vol- ume1,4issues.ISSNpaperversion1554-0669.ISSNonlineversion1554-0677. Also available as a combined paper and online subscription. FoundationsandTrends(cid:13)R in InformationRetrieval Vol.1,No1(November2006)1–96 (cid:13)c November2006NicolaOrio DOI:10.1561/1500000002 Music Retrieval: A Tutorial and Review Nicola Orio Department of Information Engineering, University of Padova Via Gradenigo, 6/b, Padova 35131, Italy, [email protected], Abstract The increasing availability of music in digital format needs to be matched by the development of tools for music accessing, filtering, classification, and retrieval. The research area of Music Information Retrieval (MIR) covers many of these aspects. The aim of this paper is to present an overview of this vast and new field. A number of issues, which are peculiar to the music language, are described—including forms, formats, and dimensions of music—together with the typologies of users and their information needs. To fulfil these needs a number of approaches are discussed, from direct search to information filtering and clustering of music documents. An overview of the techniques for music processing, which are commonly exploited in many approaches, is also presented. Evaluation and comparisons of the approaches on a common benchmark are other important issues. To this end, a descrip- tionoftheinitialeffortsandevaluationcampaignsforMIRisprovided. Contents 1 Introduction 1 1.1 Review of Music Concepts 4 2 Characteristics of the Music Language 9 2.1 Which Content Does Music Convey? 10 2.2 Dimensions of the Music Language 11 2.3 Music Forms 14 2.4 Formats of Musical Documents 21 3 The Role of the User 29 3.1 Casual Users 30 3.2 Professional Users 34 3.3 Music Theorists, Musicologists, and Musicians 36 3.4 Interaction with the System 37 4 Music Processing 41 4.1 Symbolic Form: Melody 42 4.2 Symbolic Form: Harmony 46 4.3 Audio Form: Timbre 47 4.4 Audio Form: Orchestration 49 4.5 Audio Form: Rhythm 50 4.6 Audio Form: Melody 52 ix