S60 Smartphone Quality Assurance: A Guide for Mobile Engineers and Developers

207 Pages·2007·3.773 MB·English

by  Saila Laitinen(auth.)

S60
SMARTPHONE
QUALITY
ASSURANCE
S60 Smartphone Quality Assurance: A Guide for Mobile Engineers(cid:13) and Developers
Saila Laitinen © 2007 John Wiley & Sons, Ltd. ISBN: 0-470-05685-1
S60 SMARTPHONE
QUALITY ASSURANCE
A Guide for Mobile Engineers
and Developers
Saila Laitinen
Nokia, Finland
Copyright © 2007 John Wiley & Sons Ltd, The Atrium, Southern Gate, Chichester,
West Sussex PO19 8SQ, England
Telephone [phone]
Email (for orders and customer service enquiries): [email protected]
Visit our Home Page on www.wiley.com
All Rights Reserved. No part of this publication may be reproduced, stored in a retrieval system
or transmitted in any form or by any means, electronic, mechanical, photocopying, recording,
scanning or otherwise, except under the terms of the Copyright, Designs and Patents Act 1988
or under the terms of a licence issued by the Copyright Licensing Agency Ltd, 90 Tottenham
Court Road, London W1T 4LP, UK, without the permission in writing of the Publisher.
Requests to the Publisher should be addressed to the Permissions Department, John Wiley &
Sons Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, England, or emailed
to [email protected], or faxed to ([phone].
Designations used by companies to distinguish their products are often claimed as trademarks.
All brand names and product names used in this book are trade names, service marks, trade-
marks or registered trademarks of their respective owners. The Publisher is not associated with
any product or vendor mentioned in this book.
This publication is designed to provide accurate and authoritative information in regard to the
subject matter covered. It is sold on the understanding that the Publisher is not engaged in
rendering professional services. If professional advice or other expert assistance is required,
the services of a competent professional should be sought.
Other Wiley Editorial Offi ces
John Wiley & Sons Inc., 111 River Street, Hoboken, NJ 07030, USA
Jossey-Bass, 989 Market Street, San Francisco, CA 94103–1741, USA
Wiley-VCH Verlag GmbH, Boschstr. 12, D-69469 Weinheim, Germany
John Wiley & Sons Australia Ltd, 42 McDougall Street, Milton, Queensland 4064, Australia
John Wiley & Sons (Asia) Pte Ltd, 2 Clementi Loop #02-01, Jin Xing Distripark, Singapore
129809
John Wiley & Sons Canada Ltd, 6045 Freemont Blvd, Mississauga, ONT, L5R 4J3, Canada
W iley also publishes its books in a variety of electronic formats. Some content that appears
in print may not be available in electronic books.
Anniversary Logo Design: Richard J. Pacifi co
British Library Cataloguing in Publication Data
A catalogue record for this book is available from the British Library
ISBN 978-0-470-05685-1 (PB)
Typeset in 11/13pt Zapf Humanist 601 by SNP Best-set Typesetter Ltd., Hong Kong.
Printed and bound in Great Britain by Antony Rowe Ltd, Chippenham, England.
This book is printed on acid-free paper responsibly manufactured from sustainable forestry
in which at least two trees are planted for each one used for paper production.
Table of Contents
About the Author ........................................................ xiii
Preface ......................................................................... xv
Chapter 1: Introduction to S60 ..................................... 1
1.1 T he Competitive Advantage of the S60
Platform .............................................................. 3
1.2 S60 Architecture ................................................. 5
1.2.1 T he Symbian Operating System
(Symbian OS) ...................................................... 6
1.2.2 Domestic Operating System (DOS) .................... 6
1.2.3 User Interface (UICon) ........................................ 7
1.3 Summary ............................................................. 7
Chapter 2: Selecting the Baseline ................................. 9
2.1 Manny Lehman’s Law ......................................... 10
2.2 What is so Challenging about Selecting
the Best Baseline? .............................................. 11
2.3 How should the Baseline be Selected? .............. 12
2.3.1 Baseline Maturity .............................................. 13
2.3.2 Customization Maturity ...................................... 13
2.3.3 Least Stable Sub-system ..................................... 14
2.3.4 Program Timing ................................................ 14
2.4 Summary ........................................................... 14
vi S60 Smartphone Quality Assurance: A Guide for Mobile Engineers and Developers
Chapter 3: Release Management ................................ 17
3.1 The Build Cycle .................................................. 19
3.2 Required Testing Activities ................................ 23
3.3 Summary ........................................................... 23
Chapter 4: Binary Compatibility ................................. 25
4.1 API Categorization ............................................. 28
4.2 Maintaining Compatibility ................................. 30
4.2.1 Platform Compatibility ....................................... 30
4.2.2 Platform-based Phone Compatibility ................. 31
4.2.3 Application Compatibility ................................... 32
4.2.4 Compatibility Dimensions ................................. 32
4.3 Binary Compatibility Scenario ........................... 33
4.4 Binary Compatibility Verifi cation ...................... 35
4.4.1 The Binary Compatibility Verifi cation Process .... 35
4.4.2 The Binary Compatibility Verifi cation Suite ......... 36
4.4.2.1 The SDK Analyser ................................................. 36
4.4.2.2 The Source Analyser ............................................. 38
4.4.2.3 The Binary Analyser .............................................. 39
4.4.2.4 The Application Launcher ..................................... 39
4.4.2.5 Binary Compatibility Applications ......................... 40
4.4.2.6 Third-Party Applications ....................................... 40
4.5 Possible Future Tools ......................................... 40
4.5.1 DepInfo Tool ...................................................... 41
4.5.2 Header Checker Tool ......................................... 41
4.5.3 Ordinal Checker ............................................... 42
4.6 Summary ............................................................ 42
Chapter 5: Certifi cates and Standards ...................... 43
5.1 Technology Certifi cates ..................................... 44
5.1.1 Java/TCK ........................................................... 44
5.1.2 Bluetooth ........................................................... 47
5.1.2.1 BT Certifi cation Areas ........................................... 47
Table of Contents vii
5.1.3 Other Technology Licences ............................... 48
5.1.4 Security Certifi cates ............................................ 49
5.1.5 Universal Serial Bus ........................................... 50
5.1.6 Infrared Connectivity ......................................... 50
5.1.7 Multimedia Cards (MMC) ................................... 51
5.2 The Open Mobile Alliance (OMA) ................... 52
5.2.1 Process and Principles ........................................ 52
5.3 Cellular Standards and Operators .................... 55
5.3.1 Government and Quality Certifi cates ................ 56
5.3.1.1 Mandatory ............................................................ 56
5.3.2 Optional ............................................................ 58
5.4 Summary ............................................................ 61
Chapter 6: What Quality Means ................................ 63
6.1 Quality Culture .................................................. 64
6.2 Quality Standards ............................................. 66
6.2.1 ISO 9000 .......................................................... 66
6.2.2 Six Sigma ........................................................... 67
6.3 Quality in a Product ......................................... 68
6.3.1 Quality in Manufacturing ................................... 69
6.3.2 Quality in Service .............................................. 70
6.3.3 Getting Better Quality ........................................ 71
6.4 Quality in the S60 Platform and
S60-based Phones ............................................. 73
6.4.1 Choosing the Process ........................................ 73
6.4.2 The Waterfall Process ....................................... 73
6.4.3 The Incremental Process .................................... 74
6.4.4 Agile Software Development ............................. 75
6.4.5 Concurrent Engineering ..................................... 76
6.4.6 Other Things to Consider ................................... 77
6.5 Summary ........................................................... 78
viii S60 Smartphone Quality Assurance: A Guide for Mobile Engineers and Developers
Chapter 7: Stumbling Blocks ..................................... 79
7.1 S tumbling Blocks General to All Projects ....... 79
7.2 Stumbling Blocks Specifi c to a Software
Program ........................................................... 80
7.2.1 Contradictory, Overwhelming or Too
Many Requirements ............................................ 83
7.2.2 Unstable, Incomplete and Informal
Requirements ..................................................... 83
7.2.3 Poor Planning and Project Management ............ 84
7.2.4 Unrealistic Estimates and Unjustifi ed
Expectations ...................................................... 84
7.2.5 Lack of Knowledge on New
Technologies .................................................... 84
7.2.6 Lack of Proper Risk Management ...................... 85
7.2.7 Lack of Organisational Integrity .......................... 86
7.3 W ays to Avoid Stumbling Blocks in a
Software Program ............................................ 87
7.3.1 Overview of COCOMO .................................... 87
7.4 S tumbling Blocks Specifi c to a S60-based
Phone Program ................................................ 88
7.4.1 Program-level Risks ............................................. 89
7.4.1.1 Integration Competence ....................................... 89
7.4.1.2 Testing Environment ............................................. 90
7.4.1.3 Amount of Differentiation .................................... 90
7.4.1.4 Baseline Selection ................................................ 91
7.4.1.5 Defect Fixing Order ............................................... 91
7.4.2 Component-level Risks ....................................... 91
7.4.2.1 Coding Style and Culture ..................................... 92
7.4.3 Fixing Speed .................................................... 94
7.4.3.1 Testing Activities and Extent ................................. 94
7.4.3.2 Insuffi ciency of the Specifi cation ........................... 96
7.4.3.3 Adaptation Layer Implementation ........................ 96
7.5 Provider Components ...................................... 97
7.6 Summary .......................................................... 97
Table of Contents ix
Chapter 8: Platform Testing versus
Platform-based Phone Testing .................. 99
8.1 The S60 Testing Process .................................. 100
8.1.1 Platform Test Execution Process ........................ 100
8.1.1.1 Module/Component Testing ................................. 102
8.1.1.2 Sub-system Integration Testing ............................. 102
8.1.1.3 Basic Acceptance Testing (BAT) ............................ 102
8.1.1.4 Functional Testing ............................................... 103
8.1.1.5 System, Localization, Binary Compatibility
and Interoperability Testing ................................. 103
8.1.1.6 Release Testing ................................................... 104
8.1.1.7 Regression Testing .............................................. 104
8.1.1.8 Maintenance Testing ........................................... 104
8.1.1.9 S60-based Phone Testing ..................................... 104
8.1.1.10 Planning based on Baseline Maturity Analysis ..... 107
8.1.1.11 Planning based on Fix Analysis .......................... 108
8.2 Summary ......................................................... 108
Chapter 9: Testing as a Tool ..................................... 109
9.1 Testing in Different Processes ......................... 111
9.1.1 Testing in an Iterative Process ........................... 113
9.1.2 Testing in an Incremental Process ..................... 114
9.1.3 Testing in an Agile Process ............................... 114
9.1.4 Testing in an Extreme Programming
Process ............................................................ 115
9.2 Testing Techniques .......................................... 115
9.3 Testing Phases ................................................. 120
9.3.1 Documentation Testing ................................... 121
9.3.2 Module Testing ................................................ 121
9.3.3 Integration Testing in the Small ....................... 124
9.3.4 Functional Testing ............................................ 126
9.3.5 Non-functional Testing ..................................... 126
9.3.6 Integration Testing in the Large ........................ 127
9.3.7 The Real User Experience (TRUE) ................... 128
x S60 Smartphone Quality Assurance: A Guide for Mobile Engineers and Developers
9.4 What then? ...................................................... 131
9.5 Summary .......................................................... 132
Chapter 10: The Testing Environment ...................... 133
10.1 Module Testing .............................................. 134
10.2 Integration Testing in the Small .................... 135
10.3 Functional Testing ......................................... 135
10.3.1 Common ........................................................ 136
10.3.2 UI Customization and Personalization ............ 137
10.3.3 Local Connectivity ........................................ 138
10.3.4 Networking and Data Bearers ........................ 138
10.3.5 Telephony ...................................................... 139
10.3.6 Multimedia ................................................... 140
10.3.7 Personal Information Management (PIM) ........ 141
10.3.8 Messaging ...................................................... 141
10.3.9 Browsing ........................................................ 143
10.4 Performance Testing ...................................... 144
10.5 Interoperability Testing ................................. 145
10.6 Miscellaneous Testing Activities .................... 146
10.6.1 Certifi cation .................................................. 147
10.6.2 Usability ......................................................... 147
10.7 Summary ........................................................ 148
Chapter 11: Defect Analysis ...................................... 149
11.1 Focused Testing .............................................. 152
11.2 Defect Analysis and Reporting ...................... 153
11.2.1 Defect Database ............................................. 154
11.2.2 The Defect Management Process .................. 154
11.2.3 Defect Priority ................................................ 156
11.2.3.1 Show Stopper ................................................... 156
11.2.3.2 Critical .............................................................. 158
Table of Contents xi
11.2.3.3 Major ............................................................... 159
11.2.3.4 Minor ............................................................... 159
11.2.4 Defect Reporting ............................................ 160
11.3 Summary ........................................................ 161
Chapter 12: Integration and Build
Environment .......................................... 163
12.1 Software Confi guration Management ............ 163
12.2 Changing the Code ........................................ 164
12.2.1 Confi guration Management ............................. 166
12.3 Build Environment ......................................... 167
12.3.1 Delivery Structure ........................................... 167
12.3.2 Build Process ................................................. 168
12.3.3 Build Tools ..................................................... 169
12.4 S60 Integration ............................................. 171
12.4.1 Stage 1 ........................................................... 171
12.4.1.1 Step 1: Successful Boot to Textshell ................... 171
12.4.1.2 Step 2: Simple Application and Launch
via WSER .......................................................... 172
12.4.1.3 Step 3: Starter Integration and Calculator
Launch ............................................................. 172
12.4.1.4 Step 4: Complete the S60 Boot ........................ 172
12.4.2 Stage 2 ........................................................... 172
12.4.3 Stage 3 ........................................................... 173
12.4.4 Stage 4 ........................................................... 173
12.4.5 Stage 5 ........................................................... 173
12.5 Summary ........................................................ 173
Appendix A: Examples of S60 Devices ..................... 175
Appendix B: Glossary ................................................ 179