Software Test Attacks to Break Mobile and Embedded Devices PDF
Preview Software Test Attacks to Break Mobile and Embedded Devices
Software Engineering CHAPMAN & HALL/CRC INNOVATIONS IN SOFTWARE ENGINEERING AND SOFTWARE DEVELOPMENT Software Test Attacks Software Test Attacks M to Break Mobile and S o o ft b Embedded Devices to Break Mobile and iw l e a ar e n Embedded Devices dT “Using the framework of attacks popularized by James Whittaker’s books, Jon Duncan Ee s Hagar describes those that are relevant here and extends the approach with new attacks mt specifically for mobile and embedded systems. He provides detailed information and A Jon Duncan Hagar b guidance on how to test more effectively and efficiently in the mobile and embedded tt e world.” da —From the Foreword by Dorothy Graham, Software Testing Consultant dc k e “Every tester who wants to keep current needs to read this book, and you can read with s d confidence knowing you are being guided by the best in this business.” t Do —From the Foreword by Lisa Crispin, Agile Testing Coach and Practitioner e B Using a mix-and-match approach, Software Test Attacks to Break Mobile and v r Embedded Devices presents an attack basis for testing mobile and embedded systems. ice The book focuses on attack-based testing that can be used by individuals and teams. ea sk The numerous test attacks show you when a software product does not work (i.e., has bugs) and provide you with information about the software product under test. The book guides you step by step starting with the basics. It explains patterns and techniques ranging from simple mind mapping to sophisticated test labs. Features • Provides over 30 specific software test attacks needed to find bugs/errors in today’s mobile and smart devices • Explains how to create test labs, facilities, and tools to aid in conducting the attacks • Addresses white and black box test approaches as well as risk-based testing, H a model-driven tests, agile testing, and mathematical approaches g • Details the error taxonomy upon which the attacks are based a r • Presents checklists for user interface and game evaluations • Includes practice exercises in each chapter K16700 K16700_Cover.indd 1 8/8/13 12:48 PM Software Test Attacks to Break Mobile and Embedded Devices Chapman & Hall/CRC Innovations in Software Engineering and Software Development Series Editor Richard LeBlanc Chair, Department of Computer Science and Software Engineering, Seattle University AIMS AND SCOPE This series covers all aspects of software engineering and software development. Books in the series will be innovative reference books, research monographs, and textbooks at the undergraduate and graduate level. Coverage will include traditional subject matter, cutting-edge research, and current industry practice, such as agile software development methods and service-oriented architectures. We also welcome proposals for books that capture the latest results on the domains and conditions in which practices are most ef- fective. PUBLISHED TITLES Software Development: An Open Source Approach Allen Tucker, Ralph Morelli, and Chamindra de Silva Building Enterprise Systems with ODP: An Introduction to Open Distributed Processing Peter F. Linington, Zoran Milosevic, Akira Tanaka, and Antonio Vallecillo Software Engineering: The Current Practice Václav Rajlich Fundamentals of Dependable Computing for Software Engineers John Knight Introduction to Combinatorial Testing D. Richard Kuhn, Raghu N. Kacker, and Yu Lei Software Test Attacks to Break Mobile and Embedded Devices Jon Duncan Hagar Software Designers in Action: A Human-Centric Look at Design Work André van der Hoek and Marian Petre CHAPMAN & HALL/CRC INNOVATIONS IN SOFTWARE ENGINEERING AND SOFTWARE DEVELOPMENT Software Test Attacks to Break Mobile and Embedded Devices Jon Duncan Hagar MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MATLAB® software or related products does not constitute endorsement or sponsorship by The MathWorks of a particular pedagogical approach or particular use of the MATLAB® software. CRC Press Taylor & Francis Group 6000 Broken Sound Parkway NW, Suite 300 Boca Raton, FL 33487-2742 © 2014 by Taylor & Francis Group, LLC CRC Press is an imprint of Taylor & Francis Group, an Informa business No claim to original U.S. Government works Version Date: 20130611 International Standard Book Number-13: 978-1-4665-7531-8 (eBook - PDF) This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid- ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may rectify in any future reprint. Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti- lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy- ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the publishers. For permission to photocopy or use material electronically from this work, please access www.copyright.com (http:// www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged. Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe. Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com and the CRC Press Web site at http://www.crcpress.com Contents Foreword by Dorothy Graham, xi Foreword by Lisa Crispin, xiii Preface, xv Acknowledgments, xvii Copyright and Trademarks Declaration Page, xix Introduction, xxi Author, xxxiii Chapter 1 ◾ Setting the Mobile and Embedded Framework 1 OBJECTIVES OF TESTING MOBILE AND EMBEDDED SOFTWARE SYSTEMS 1 WHAT IS EMBEDDED SOFTWARE? 2 WHAT ARE “SMART” HANDHELD AND MOBILE SYSTEMS? 3 WHY MOBILE AND EMBEDDED ATTACKS? 5 FRAMEWORK FOR ATTACKS 6 BEGINNING YOUR TEST STRATEGY 6 ATTACKS ON MOBILE AND EMBEDDED SOFTWARE 8 IF YOU ARE NEW TO TESTING 9 AN ENLIGHTENED TESTER MAKES A BETTER TESTER 10 Chapter 2 ◾ Developer Attacks: Taking the Code Head On 13 ATTACK 1: STATIC CODE ANALYSIS 14 ATTACK 2: FINDING WHITE-BOX DATA COMPUTATION BUGS 21 ATTACK 3: WHITE-BOX STRUCTURAL LOGIC FLOW COVERAGE 25 TEST COVERAGE CONCEPTS FOR WHITE-BOX STRUCTURAL TESTING 28 NOTE OF CONCERN IN MOBILE AND EMBEDDED ENVIRONMENTS 29 v vi ◾ Contents Chapter 3 ◾ Control System Attacks 33 ATTACK 4: FINDING HARDWARE–SYSTEM UNHANDLED USES IN SOFTWARE 33 ATTACK 5: HARDWARE-TO-SOFTWARE AND SOFTWARE-TO-HARDWARE SIGNAL INTERFACE BUGS 39 ATTACK 6: LONG-DURATION CONTROL ATTACK RUNS 45 ATTACK 7: BREAKING SOFTWARE LOGIC AND/OR CONTROL LAWS 49 ATTACK 8: FORCING THE UNUSUAL BUG CASES 54 Chapter 4 ◾ Hardware Software Attacks 59 ATTACK 9: BREAKING SOFTWARE WITH HARDWARE AND SYSTEM OPERATIONS 59 SUB-ATTACK 9.1: BREAKING BATTERY POWER 65 ATTACK 10: FINDING BUGS IN HARDWARE–SOFTWARE COMMUNICATIONS 66 ATTACK 11: BREAKING SOFTWARE ERROR RECOVERY 69 ATTACK 12: INTERFACE AND INTEGRATION TESTING 74 SUB-ATTACK 12.1: CONFIGURATION INTEGRATION EVALUATION 80 ATTACK 13: FINDING PROBLEMS IN SOFTWARE–SYSTEM FAULT TOLERANCE 80 Chapter 5 ◾ Mobile and Embedded Software Attacks 89 ATTACK 14: BREAKING DIGITAL SOFTWARE COMMUNICATIONS 89 ATTACK 15: FINDING BUGS IN THE DATA 94 ATTACK 16: BUGS IN SYSTEM-SOFTWARE COMPUTATION 97 ATTACK 17: USING SIMULATION AND STIMULATION TO DRIVE SOFTWARE ATTACKS 101 Chapter 6 ◾ Time Attacks: “It’s about Time” 107 ATTACK 18: BUGS IN TIMING INTERRUPTS AND PRIORITY INVERSIONS 108 STATE MODELING EXAMPLE 114 ATTACK 19: FINDING TIME-RELATED BUGS 116 ATTACK 20: TIME-RELATED SCENARIOS, STORIES, AND TOURS 121 ATTACK 21: PERFORMANCE TESTING INTRODUCTION 125 SUPPORTING CONCEPTS 139 COMPLETING AND REPORTING THE PERFORMANCE ATTACK 140 WRAPPING UP 140 Contents ◾ vii Chapter 7 ◾ H uman User Interface Attacks: “The Limited (and Unlimited) User Interface” 143 HOW TO GET STARTED—THE UI 144 ATTACK 22: FINDING SUPPORTING (USER) DOCUMENTATION PROBLEMS 146 SUB-ATTACK 22.1: CONFIRMING INSTALL-ABILITY 149 ATTACK 23: FINDING MISSING OR WRONG ALARMS 149 ATTACK 24: FINDING BUGS IN HELP FILES 153 Chapter 8 ◾ Smart and/or Mobile Phone Attacks 159 GENERAL NOTES AND ATTACK CONCEPTS APPLICABLE TO MOST MOBILE–EMBEDDED DEVICES 159 ATTACK 25: FINDING BUGS IN APPS 161 ATTACK 26: TESTING MOBILE AND EMBEDDED GAMES 165 ATTACK 27: ATTACKING APP–CLOUD DEPENDENCIES 170 Chapter 9 ◾ Mobile/Embedded Security 177 THE CURRENT SITUATION 178 REUSING SECURITY ATTACKS 178 ATTACK 28: PENETRATION ATTACK TEST 180 ATTACK 28.1: PENETRATION SUB-ATTACKS: AUTHENTICATION—PASSWORD ATTACK 186 ATTACK 28.2: SUB-ATTACK FUZZ TEST 188 ATTACK 29: INFORMATION THEFT—STEALING DEVICE DATA 189 ATTACK 29.1: SUB-ATTACK—IDENTITY SOCIAL ENGINEERING 193 ATTACK 30: SPOOFING ATTACKS 194 ATTACK 30.1: LOCATION AND/OR USER PROFILE SPOOF SUB-ATTACK 199 ATTACK 30.2: GPS SPOOF SUB-ATTACK 200 ATTACK 31: ATTACKING VIRUSES ON THE RUN IN FACTORIES OR PLCS 201 Chapter 10 ◾ Generic Attacks 209 ATTACK 32: USING COMBINATORIAL TESTS 209 ATTACK 33: ATTACKING FUNCTIONAL BUGS 215 Chapter 11 ◾ Mobile and Embedded System Labs 221 INTRODUCTION TO LABS 221 TO START 222 TEST FACILITIES 223 viii ◾ Contents WHY SHOULD A TESTER CARE? 224 WHAT PROBLEM DOES A TEST LAB SOLVE? 225 STAGED EVOLUTION OF A TEST LAB 227 SIMULATION ENVIRONMENTS 227 PROTOTYPE AND EARLY DEVELOPMENT LABS 228 DEVELOPMENT SUPPORT TEST LABS 228 INTEGRATION LABS 230 PRE-PRODUCT AND PRODUCT RELEASE (FULL TEST LAB) 230 FIELD LABS 230 OTHER PLACES LABS CAN BE REALIZED 232 DEVELOPING LABS: A PROJECT INSIDE OF A PROJECT 233 PLANNING LABS 233 REQUIREMENT CONSIDERATIONS FOR LABS 234 FUNCTIONAL ELEMENTS FOR A DEVELOPER SUPPORT LAB 234 FUNCTIONAL ELEMENTS FOR A SOFTWARE TEST LAB 235 TEST LAB DESIGN FACTORS 236 LAB IMPLEMENTATION 238 LAB CERTIFICATION 238 OPERATIONS AND MAINTENANCE IN THE LAB 239 LAB LESSONS LEARNED 240 AUTOMATION CONCEPTS FOR TEST LABS 241 TOOLING TO SUPPORT LAB WORK 241 TEST DATA SET-UP 243 TEST EXECUTION: FOR DEVELOPER TESTING 244 TEST EXECUTION: GENERAL 245 PRODUCT AND SECURITY ANALYSIS TOOLS 247 TOOLS FOR THE LAB TEST RESULTS RECORDING 247 PERFORMANCE ATTACK TOOLING 248 BASIC AND GENERIC TEST SUPPORT TOOLS 250 AUTOMATION: TEST ORACLES FOR THE LAB USING MODELING TOOLS 251 SIMULATION, STIMULATION, AND MODELING IN THE LAB TEST BED 253 CONTINUOUS REAL-TIME, CLOSED-LOOP SIMULATIONS TO SUPPORT LAB TEST ENVIRONMENTS 256 KEYWORD-DRIVEN TEST MODELS AND ENVIRONMENTS 259 Contents ◾ ix DATA COLLECTION, ANALYSIS, AND REPORTING 260 POSTTEST DATA ANALYSIS 262 POSTTEST DATA REPORTING 265 WRAP UP: N-VERSION TESTING PROBLEMS IN LABS AND MODELING 267 FINAL THOUGHTS: INDEPENDENCE, BLIND SPOTS, AND TEST LAB STAFFING 268 Chapter 12 ◾ Some Parting Advice 273 ARE WE THERE YET? 273 WILL YOU GET STARTED TODAY? 273 ADVICE FOR THE “NEVER EVER” TESTER 273 BUG DATABASE, TAXONOMIES, AND LEARNING FROM YOUR HISTORY 274 LESSONS LEARNED AND RETROSPECTIVES 275 IMPLEMENTING SOFTWARE ATTACK PLANNING 275 REGRESSION AND RETEST 277 WHERE DO YOU GO FROM HERE? 278 APPENDIX A: M OBILE AND EMBEDDED ERROR TAXONOMY: A SOFTWARE ERROR TAXONOMY (FOR TESTERS), 279 APPENDIX B: MOBILE AND EMBEDDED CODING RULES, 289 APPENDIX C: Q UALITY FIRST: “DEFENDING THE SOURCE CODE SO THAT ATTACKS ARE NOT SO EASY,” 293 APPENDIX D: BASIC TIMING CONCEPTS, 299 APPENDIX E: DETAILED MAPPING OF ATTACKS, 303 APPENDIX F: UI/GUI AND GAME EVALUATION CHECKLIST, 307 APPENDIX G: RISK ANALYSIS, FMEA, AND BRAINSTORMING, 313 REFERENCES 319 GLOSSARY, 323
The list of books you might like
