PERSONAL CONTENT EXPERIENCE Personal Content Experience: Managing Digital Life in the Mobile Age J. Lehikoinen, A. Aaltonen, P. Huuskonen and I. Salminen © 2007 John Wiley & Sons, Ltd. ISBN: 978-0-470-03464-4 PERSONAL CONTENT EXPERIENCE Managing Digital Life in the Mobile Age Juha Lehikoinen Antti Aaltonen Pertti Huuskonen Ilkka Salminen Copyright © 2007 John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England Telephone (+44) 1243 779777 Email (for orders and customer service enquiries): [email protected] Visit our Home Page on www.wiley.com All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988 or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher. Requests to the Publisher should be addressed to the Permissions Depart- ment, John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or emailed to [email protected], or faxed to (+44) 1243 770620. Designations used by companies to distinguish their products are often claimed as trade- marks. All brand names and product names used in this book are trade names, service marks, trademarks or registered trademarks of their respective owners. The Publisher is not associated with any product or vendor mentioned in this book. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold on the understanding that the Publisher is not engaged in rendering professional services. If professional advice or other expert assistance is required, the services of a competent professional should be sought. Other Wiley Editorial Offi ces John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA Jossey-Bass, 989 Market Street, San Francisco, CA 94103-1741, USA Wiley-VCH Verlag GmbH, Boschstr. 12, D-69469 Weinheim, Germany John Wiley & Sons Australia Ltd, 42 McDougall Street, Milton, Queensland 4064, Australia John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore 129809 John Wiley & Sons Canada Ltd, 6045 Freemont Blvd, Mississauga, ONT, L5R 4J3, Canada Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Anniversary Logo Design: Richard J. Pacifi co British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library ISBN 978-0-470-03464-4 (PB) Typeset by SNP Best-set Typesetter Ltd., Hong Kong Printed and bound in Great Britain by Antony Rowe Ltd, Chippenham, England. This book is printed on acid-free paper responsibly manufactured from sustainable forestry in which at least two trees are planted for each one used for paper production. Contents Foreword ......................................................................... xiii Acknowledgements ......................................................... xv List of Abbreviations ....................................................... xvii Prologue .......................................................................... xxi Chapter 1: Digital Memories and the Personal Content Explosion ........................................................... 1 1.1 Digital Us ............................................................ 2 1.2 You and This Book .............................................. 5 1.3 Contents at a Glance ......................................... 7 Chapter 2: Trends Towards Mobility ............................. 11 2.1 The New Nomads ............................................... 12 2.1.1 Five-Second Attention Span ................................. 13 2.1.2 Continuous and Nomadic Mobile Use ................. 15 2.2 Mobile Device Categories ................................... 16 2.2.1 Dedicated Media Devices .................................... 17 2.2.2 Swiss Army Knives................................................ 20 2.2.3 Toolbox Devices .................................................. 21 2.2.4 Accessories and Other Devices ............................ 24 2.3 Mobile Compromises .......................................... 25 2.3.1 Teeny Weeny UIs ................................................ 30 2.4 Because it Can! ................................................... 30 2.5 Convergence ........................................................ 31 2.6 Wireless Revolution ............................................ 33 vi Personal Content Experience: Managing Digital Life in the Mobile Age 2.6.1 Broadcast Networks ............................................. 35 2.6.2 Short-Range Wireless ........................................... 36 2.7 Case Study: Mobile Music .................................. 38 2.8 References ........................................................... 43 Chapter 3: Mobile Personal Content Uncovered ........... 45 3.1 First there were Files........................................... 48 3.1.1 From File Management to Content Management ........................................................ 49 3.1.2 Creation and Usage make Content Personal ........ 52 3.2 Categorization ..................................................... 54 3.3 Characteristics of Personal Content ................... 56 3.3.1 Content Explosion ................................................ 58 3.3.2 Personal Content is Invaluable ............................. 59 3.3.3 Personal Content is Familiar . . . or Not ................ 59 3.3.4 Favourites ............................................................ 61 3.3.5 Sharing and Communities .................................... 61 3.3.6 Relations and Associations ................................... 61 3.3.7 Privacy and Security Requirements ...................... 62 3.4 Mobile Personal Content .................................... 63 3.4.1 Mobile Personal Content is Distributed ................ 63 3.4.2 Mobile Content is Tied to Creation and Usage Context ..................................................... 64 3.4.3 The Same Content Types, New Usage Patterns ... 64 3.4.4 Totally New Content Types, or Extended Use of Existing Content Types ..................................... 65 3.4.5 New Behavioural Patterns .................................... 66 3.4.6 New Challenges ................................................... 67 3.5 Content Wants to be Free? ................................. 67 3.6 GEMS, a Tool for Modelling Personal Content Experience ............................................. 72 3.7 References ........................................................... 76 Contents vii Chapter 4: Metadata Magic ............................................ 79 4.1 Metadata for Consumers: A Brief Introduction ........................................................ 80 4.1.1 Metadata Semantics ............................................. 82 4.1.2 Metadata – For Managing or Enjoying? ................ 83 4.2 Metadata Creation .............................................. 84 4.3 Metadata Maintenance ....................................... 90 4.4 Relations Give Meaning ...................................... 91 4.4.1 People as First-Class Metadata ............................. 92 4.4.2 Derived Metadata ................................................ 93 4.5 How does Metadata Benefi t the User? ............... 93 4.5.1 Tracing and Recall ............................................... 94 4.5.2 Searching ............................................................. 94 4.5.3 Organizing: Sorting, Grouping and Filtering ......... 95 4.5.4 Automatic Summarizing ....................................... 96 4.5.5 Enhancing Privacy and Security ............................ 97 4.5.6 Constructing Views .............................................. 97 4.5.7 Better Recommendations ..................................... 98 4.5.8 Reusing / Remixing / Reconstructing .................... 98 4.5.9 Smoother Transition Between Applications .......... 99 4.6 Existing Approaches ............................................ 99 4.6.1 MARC .................................................................. 100 4.6.2 Dublin Core Metadata Initiative ........................... 101 4.6.3 XMP ..................................................................... 103 4.6.4 ID3v2 .................................................................. 104 4.6.5 Acidized Wav ...................................................... 105 4.6.6 DCF and EXIF ...................................................... 107 4.6.7 Quicktime ............................................................ 107 4.6.8 MPEG-7 ............................................................... 108 4.6.9 RSS ...................................................................... 109 4.6.10 Summary ........................................................... 110 viii Personal Content Experience: Managing Digital Life in the Mobile Age 4.7 The PCE Trinity: Mobility, Context and Metadata ...................................................... 111 4.7.1 File Context ......................................................... 112 4.7.2 Elements of Context ............................................. 114 4.7.3 Context is Essential for Communication ............... 115 4.8 The Challenges: Universal Metadata, Extensibility, Abuse ............................................. 116 4.9 Yet Another Challenge: Interoperability ............. 118 4.9.1 Personal Content Device Ecosystem ..................... 118 4.9.2 Application Interoperability .................................. 120 4.9.3 Existing Solutions for Interoperability .................... 120 4.10 The Dream: When Metadata Really Works ................................................................ 122 4.11 References ......................................................... 124 Chapter 5: Realizing a Metadata Framework ................. 127 5.1 Metadata is a Solution . . . and a Problem ............................................................... 128 5.2 Challenges in Distributed Mobile Content Management ....................................................... 130 5.2.1 Storage ................................................................. 130 5.2.2 Synchronization ................................................... 133 5.2.3 Version Control .................................................... 134 5.2.4 Backing Up .......................................................... 135 5.2.5 Content Adaptation .............................................. 136 5.2.6 Locating the Desired Piece of Content ................. 138 5.3 Different Types of Metadata ............................... 140 5.3.1 Tags ..................................................................... 141 5.3.2 Context Capture .................................................. 142 5.3.3 Relationships ........................................................ 145 5.3.4 Usage History and Events ..................................... 147 Contents ix 5.4 From Content Management to Metadata Management ....................................................... 148 5.4.1 Cross Media Challenge and Metadata Ownership ........................................................... 150 5.4.2 Separating Metadata from Content Binaries ......... 151 5.4.3 Preparing for the Future ....................................... 153 5.5 Overall Architecture ............................................ 154 5.6 Our Metadata Ontology ...................................... 157 5.6.1 Instance Metadata and the Schema ..................... 160 5.6.2 Initializing the Framework .................................... 160 5.6.3 Our Default Ontology .......................................... 161 5.6.4 Namespace .......................................................... 161 5.6.5 Metadata Schema Objects ................................... 162 5.6.6 The Most Typical Metadata Schema Objects and Attributes ...................................................... 163 5.6.7 Events .................................................................. 164 5.6.8 Relationships ........................................................ 165 5.6.9 How to Handle Composite Objects ..................... 166 5.6.10 URIs for Fragments ............................................ 167 5.6.11 Extending the Ontology ..................................... 169 5.7 Making a Prototype Implementation .................. 171 5.7.1 Metadata Engine .................................................. 172 5.7.2 Managing Schemas .............................................. 174 5.7.3 Why Use SQL and Especially SQLite as Persistent Storage ................................................. 178 5.7.4 Harvester Manager ............................................... 183 5.7.5 Context Engine .................................................... 185 5.8 Facing Real Life ................................................... 189 5.8.1 Memory Consumption ......................................... 190 5.8.2 Speed .................................................................. 191 5.8.3 Example Usage of Metadata Engine ..................... 192 x Personal Content Experience: Managing Digital Life in the Mobile Age 5.9 Metadata Processors ........................................... 194 5.10 Summary ........................................................... 195 5.11 References ......................................................... 196 Chapter 6: User Interfaces for Mobile Media ................ 199 6.1 Human in the Loop ............................................. 202 6.1.1 Searching ............................................................. 204 6.1.2 User-Centred Design ............................................ 205 6.2 Interacting with Mobile Personal Content ......... 206 6.2.1 Music ................................................................... 207 6.2.2 Photos .................................................................. 209 6.2.3 Video ................................................................... 211 6.3 Interfaces for Mobile Media Devices ................. 213 6.3.1 Why not Speech User Interfaces for Mobiles? ...... 214 6.3.2 Graphical User Interfaces ..................................... 215 6.3.3 Interaction Technologies and Techniques ............ 216 6.3.4 UI structure and Navigation ................................. 223 6.3.5 Basic UI Components for Mobile Media .............. 230 6.4 Designing a Mobile User Interface ..................... 242 6.4.1 Common UI Design Guidelines ........................... 243 6.4.2 The UI Design Process and Methods ................... 246 6.4.3 Validating the Design ........................................... 252 6.5 Performing the GEMS Tasks ............................... 254 6.5.1 Cross-GEMS Tasks: Browse and Search ................ 254 6.5.2 Get Content ......................................................... 260 6.5.3 Enjoy Content ...................................................... 266 6.5.4 Maintain Content ................................................. 270 6.5.5 Share Content ...................................................... 273 6.5.6 Multi-Tasking in GEMS ......................................... 276 6.6 The Effect of Device Category on UI .................. 277 6.7 Summary ............................................................. 279 6.8 References ........................................................... 280 Contents xi Chapter 7: Application Outlook ..................................... 285 7.1 General Characteristics of Mobile Applications ......................................................... 288 7.2 Location-Based Applications ............................... 289 7.2.1 Point of Interest ................................................... 290 7.2.2 Wayfi nding .......................................................... 291 7.2.3 Annotations .......................................................... 293 7.2.4 Location as Metadata ........................................... 294 7.2.5 Location and Communities .................................. 295 7.2.6 Other Applications ............................................... 298 7.2.7 Discussion ............................................................ 298 7.3 Sharing and Communities .................................. 300 7.3.1 Content Sharing ................................................... 300 7.3.2 Content Rating ..................................................... 302 7.3.3 Self-Expression ..................................................... 303 7.3.4 YouTube .............................................................. 303 7.3.5 Video Sharing Cornes of Age ............................... 304 7.4 Games ................................................................. 307 7.4.1 Mobile Games ..................................................... 308 7.4.2 Personal Content Types Related to Games ........... 311 7.4.3 Modding .............................................................. 313 7.4.4 Discussion ............................................................ 314 7.5 Other Domains ................................................... 317 7.5.1 Personal Training ................................................. 317 7.5.2 Movie Subtitles .................................................... 319 7.5.3 Flash, Comics, Animations ................................... 320 7.5.4 Discussion ............................................................ 321 7.6 References ........................................................... 321 Chapter 8: Timeshifting Life ........................................... 323 8.1 Metadata in the Years to Come .......................... 324 8.1.1 Metadata Enablers................................................ 326