Software engineering for science

300 Pages·2017·17.163 MB·English

by  Carver, Jeffrey, Hong, Neil P. Chue, Thiruvathukal, George Kuriakose

SOFTWARE
ENGINEERING
FOR SCIENCE
Chapman & Hall/CRC
Computational Science Series
SERIES EDITOR
Horst Simon
Deputy Director
Lawrence Berkeley National Laboratory
Berkeley, California, U.S.A.
PUBLISHED TITLES
COMBINATORIAL SCIENTIFIC COMPUTING

Edited by Uwe Naumann and Olaf Schenk
CONTEMPORARY HIGH PERFORMANCE COMPUTING: FROM PETASCALE
TOWARD EXASCALE
Edited by Jeffrey S. Vetter
CONTEMPORARY HIGH PERFORMANCE COMPUTING: FROM PETASCALE
TOWARD EXASCALE, VOLUME TWO
Edited by Jeffrey S. Vetter
DATA-INTENSIVE SCIENCE
Edited by Terence Critchlow and Kerstin Kleese van Dam
THE END OF ERROR: UNUM COMPUTING
John L. Gustafson
FROM ACTION SYSTEMS TO DISTRIBUTED SYSTEMS: THE REFINEMENT APPROACH
Edited by Luigia Petre and Emil Sekerinski
FUNDAMENTALS OF MULTICORE SOFTWARE DEVELOPMENT
Edited by Victor Pankratius, Ali-Reza Adl-Tabatabai, and Walter Tichy
FUNDAMENTALS OF PARALLEL MULTICORE ARCHITECTURE
Yan Solihin
THE GREEN COMPUTING BOOK: TACKLING ENERGY EFFICIENCY AT LARGE SCALE
Edited by Wu-chun Feng
GRID COMPUTING: TECHNIQUES AND APPLICATIONS
Barry Wilkinson
HIGH PERFORMANCE COMPUTING: PROGRAMMING AND APPLICATIONS
John Levesque with Gene Wagenbreth
HIGH PERFORMANCE PARALLEL I/O
Prabhat and Quincey Koziol
PUBLISHED TITLES CONTINUED
HIGH PERFORMANCE VISUALIZATION:
ENABLING EXTREME-SCALE SCIENTIFIC INSIGHT
Edited by E. Wes Bethel, Hank Childs, and Charles Hansen
INDUSTRIAL APPLICATIONS OF HIGH-PERFORMANCE COMPUTING:
BEST GLOBAL PRACTICES
Edited by Anwar Osseyran and Merle Giles
INTRODUCTION TO COMPUTATIONAL MODELING USING C AND
OPEN-SOURCE TOOLS
José M Garrido
INTRODUCTION TO CONCURRENCY IN PROGRAMMING LANGUAGES
Matthew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen

INTRODUCTION TO ELEMENTARY COMPUTATIONAL MODELING: ESSENTIAL
CONCEPTS, PRINCIPLES, AND PROBLEM SOLVING
José M. Garrido
INTRODUCTION TO HIGH PERFORMANCE COMPUTING FOR SCIENTISTS
AND ENGINEERS
Georg Hager and Gerhard Wellein
INTRODUCTION TO REVERSIBLE COMPUTING
Kalyan S. Perumalla
INTRODUCTION TO SCHEDULING
Yves Robert and Frédéric Vivien
INTRODUCTION TO THE SIMULATION OF DYNAMICS USING SIMULINK®
Michael A. Gray
PEER-TO-PEER COMPUTING: APPLICATIONS, ARCHITECTURE, PROTOCOLS,
AND CHALLENGES
Yu-Kwong Ricky Kwok
PERFORMANCE TUNING OF SCIENTIFIC APPLICATIONS
Edited by David Bailey, Robert Lucas, and Samuel Williams
PETASCALE COMPUTING: ALGORITHMS AND APPLICATIONS
Edited by David A. Bader
PROCESS ALGEBRA FOR PARALLEL AND DISTRIBUTED PROCESSING
Edited by Michael Alexander and William Gardner
SCIENTIFIC DATA MANAGEMENT: CHALLENGES, TECHNOLOGY, AND DEPLOYMENT
Edited by Arie Shoshani and Doron Rotem
SOFTWARE ENGINEERING FOR SCIENCE
Edited by Jeffrey C. Carver, Neil P. Chue Hong, and George K. Thiruvathukal

SOFTWARE
ENGINEERING
FOR SCIENCE

Edited by
Jeffrey C. Carver
University of Alabama, USA
Neil P. Chue Hong
University of Edinburgh, UK
George K. Thiruvathukal
Loyola University Chicago, Chicago, Illinois
MATLAB® is a trademark of The MathWorks, Inc. and is used with permission. The MathWorks does not
warrant the accuracy of the text or exercises in this book. This book’s use or discussion of MATLAB® soft-
ware or related products does not constitute endorsement or sponsorship by The MathWorks of a particular
pedagogical approach or particular use of the MATLAB® software.
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2017 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Printed on acid-free paper

Version Date: 20160817
International Standard Book Number-13: 978-1-4987-4385-3 (Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibility for the validity of all materials or the consequences of their use. The authors and publishers
have attempted to trace the copyright holders of all material reproduced in this publication and apologize to
copyright holders if permission to publish in this form has not been obtained. If any copyright material has
not been acknowledged please write and let us know so we may rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmit-
ted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter invented,
including photocopying, microfilming, and recording, or in any information storage or retrieval system,
without written permission from the publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.
com (http://www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood
Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and
registration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used
only for identification and explanation without intent to infringe.
Library of Congress Cataloging‑in‑Publication Data
Names: Carver, Jeffrey, editor. | Hong, Neil P. Chue, editor. |
Thiruvathukal, George K. (George Kuriakose), editor.
Title: Software engineering for science / edited by Jeffrey Carver, Neil P.
Chue Hong, and George K. Thiruvathukal.
Description: Boca Raton : Taylor & Francis, CRC Press, 2017. | Series:
Computational science series ; 30 | Includes bibliographical references
and index.
Identifiers: LCCN 2016022277 | ISBN 9781498743853 (alk. paper)
Subjects: LCSH: Science--Data processing. | Software engineering.
Classification: LCC Q183.9 .S74 2017 | DDC 005.1--dc23
LC record available at https://lccn.loc.gov/2016022277
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
Contents
List of Figures xv
List of Tables xvii

About the Editors xix
List of Contributors xxi
Acknowledgments xxv
Introduction xxvii
1 Software Process for Multiphysics Multicomponent Codes 1
Anshu Dubey, Katie Antypas, Ethan Coon, and Katherine Riley
1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.2 Lifecycle . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2.1 Development Cycle . . . . . . . . . . . . . . . . . . . . 4
1.2.2 Verification and Validation . . . . . . . . . . . . . . . 4
1.2.3 Maintenance and Extensions . . . . . . . . . . . . . . 6
1.2.4 Performance Portability . . . . . . . . . . . . . . . . . 7
1.2.5 Using Scientific Software . . . . . . . . . . . . . . . . . 7
1.3 Domain Challenges . . . . . . . . . . . . . . . . . . . . . . . 8
1.4 Institutional and Cultural Challenges . . . . . . . . . . . . . 9
1.5 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.5.1 FLASH . . . . . . . . . . . . . . . . . . . . . . . . . . 12
1.5.1.1 Code Design . . . . . . . . . . . . . . . . . . 12
1.5.1.2 Verification and Validation . . . . . . . . . . 14
1.5.1.3 Software Process . . . . . . . . . . . . . . . . 16
1.5.1.4 Policies . . . . . . . . . . . . . . . . . . . . . 18
1.5.2 Amanzi/ATS . . . . . . . . . . . . . . . . . . . . . . . 19
1.5.2.1 Multiphysics Management through Arcos . . 20
1.5.2.2 Code Reuse and Extensibility. . . . . . . . . 21
1.5.2.3 Testing . . . . . . . . . . . . . . . . . . . . . 21
1.5.2.4 Performance Portability . . . . . . . . . . . . 22
vii
viii Contents
1.6 Generalization . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.7 Additional Future Considerations . . . . . . . . . . . . . . . 25
2 A Rational Document Driven Design Process for Scientific
Software 27
W. Spencer Smith
2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
2.2 A Document Driven Method . . . . . . . . . . . . . . . . . . 31
2.2.1 Problem Statement . . . . . . . . . . . . . . . . . . . . 32
2.2.2 Development Plan . . . . . . . . . . . . . . . . . . . . 33
2.2.3 Software Requirements Specification (SRS) . . . . . . 34
2.2.4 Verification and Validation (V&V) Plan and Report . 35
2.2.5 Design Specification . . . . . . . . . . . . . . . . . . . 37
2.2.6 Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.2.7 User Manual . . . . . . . . . . . . . . . . . . . . . . . 40
2.2.8 Tool Support . . . . . . . . . . . . . . . . . . . . . . . 41
2.3 Example: Solar Water Heating Tank . . . . . . . . . . . . . . 41
2.3.1 Software Requirements Specification (SRS) . . . . . . 42
2.3.2 Design Specification . . . . . . . . . . . . . . . . . . . 45
2.4 Justification . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.4.1 Comparison between CRAN and Other Communities. 48
2.4.2 Nuclear Safety Analysis Software Case Study . . . . . 49
2.5 Concluding Remarks . . . . . . . . . . . . . . . . . . . . . . 50
3 Making Scientific Software Easier to Understand, Test, and
Communicate through Software Engineering 53
Matthew Patrick
3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
3.2 Case Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . 56
3.3 Challenges Faced by the Case Studies . . . . . . . . . . . . . 56
3.3.1 Intuitive Testing . . . . . . . . . . . . . . . . . . . . . 60
3.3.2 Automating Tests . . . . . . . . . . . . . . . . . . . . 62
3.3.3 Legacy Code . . . . . . . . . . . . . . . . . . . . . . . 64
3.3.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 66
3.4 Iterative Hypothesis Testing . . . . . . . . . . . . . . . . . . 66
3.4.1 The Basic SEIR Model. . . . . . . . . . . . . . . . . . 67
3.4.2 Experimental Methodology . . . . . . . . . . . . . . . 68
3.4.3 Initial Hypotheses . . . . . . . . . . . . . . . . . . . . 69
3.4.3.1 Sanity Checks . . . . . . . . . . . . . . . . . 69
3.4.3.2 Metamorphic Relations . . . . . . . . . . . . 70
3.4.3.3 Mathematical Derivations . . . . . . . . . . . 71
3.4.4 Exploring and Refining the Hypotheses . . . . . . . . 71
3.4.4.1 Complexities of the Model . . . . . . . . . . 72
3.4.4.2 Complexities of the Implementation . . . . . 73
3.4.4.3 Issues Related to Numerical Precision . . . . 74
Contents ix
3.4.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.5 Testing Stochastic Software Using Pseudo-Oracles . . . . . . 77
3.5.1 The Huánglóngbìng SECI Model . . . . . . . . . . . . 78
3.5.2 Searching for Differences . . . . . . . . . . . . . . . . . 80
3.5.3 Experimental Methodology . . . . . . . . . . . . . . . 82
3.5.4 Differences Discovered . . . . . . . . . . . . . . . . . . 82
3.5.5 Comparison with Random Testing . . . . . . . . . . . 86
3.5.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.7 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 88
4 Testing of Scientific Software: Impacts on Research
Credibility, Development Productivity, Maturation,
and Sustainability 89
Roscoe A. Bartlett, Anshu Dubey, Xiaoye Sherry Li, J. David Moulton,
James M. Willenbring, and Ulrike Meier Yang
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 90
4.2 Testing Terminology . . . . . . . . . . . . . . . . . . . . . . . 92
4.2.1 Granularity of Tests . . . . . . . . . . . . . . . . . . . 92
4.2.2 Types of Tests . . . . . . . . . . . . . . . . . . . . . . 93
4.2.3 Organization of Tests . . . . . . . . . . . . . . . . . . 94
4.2.4 Test Analysis Tools. . . . . . . . . . . . . . . . . . . . 95
4.3 Stakeholders and Team Roles for CSE Software Testing . . . 95
4.3.1 Stakeholders . . . . . . . . . . . . . . . . . . . . . . . 95
4.3.2 Key Roles in Effective Testing . . . . . . . . . . . . . 96
4.3.3 Caveats and Pitfalls . . . . . . . . . . . . . . . . . . . 97
4.4 Roles of Automated Software Testing in CSE Software . . . 98
4.4.1 Role of Testing in Research . . . . . . . . . . . . . . . 98
4.4.2 Role of Testing in Development Productivity . . . . . 100
4.4.3 Role of Testing in Software Maturity and Sustainability 102
4.5 Challenges in Testing Specific to CSE . . . . . . . . . . . . . 103
4.5.1 Floating-Point Issues and Their Impact on Testing . . 103
4.5.2 Scalability Testing . . . . . . . . . . . . . . . . . . . . 105
4.5.3 Model Testing . . . . . . . . . . . . . . . . . . . . . . 107
4.6 Testing Practices . . . . . . . . . . . . . . . . . . . . . . . . . 109
4.6.1 Building a Test Suite for CSE Codes . . . . . . . . . . 110
4.6.2 Evaluation and Maintenance of a Test Suite . . . . . . 112
4.6.3 An Example of a Test Suite . . . . . . . . . . . . . . . 113
4.6.4 Use of Test Harnesses . . . . . . . . . . . . . . . . . . 114
4.6.5 Policies . . . . . . . . . . . . . . . . . . . . . . . . . . 116
4.7 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
4.8 Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . 118
x Contents
5 Preserving Reproducibility through Regression Testing 119
Daniel Hook
5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
5.1.1 Other Testing Techniques . . . . . . . . . . . . . . . . 120
5.1.2 Reproducibility . . . . . . . . . . . . . . . . . . . . . . 121
5.1.3 Regression Testing . . . . . . . . . . . . . . . . . . . . 122
5.2 Testing Scientific Software . . . . . . . . . . . . . . . . . . . 123
5.2.1 The Oracle and Tolerance Problems . . . . . . . . . . 123
5.2.1.1 Sensitivity Testing . . . . . . . . . . . . . . . 125
5.2.2 Limitations of Regression Testing . . . . . . . . . . . . 125
5.3 Regression Testing at ESG . . . . . . . . . . . . . . . . . . . 126
5.3.1 Building the Tools . . . . . . . . . . . . . . . . . . . . 127
5.3.1.1 Key Lesson . . . . . . . . . . . . . . . . . . . 129
5.3.2 Selecting the Tests . . . . . . . . . . . . . . . . . . . . 129
5.3.2.1 Key Lessons . . . . . . . . . . . . . . . . . . 130
5.3.3 Evaluating the Tests . . . . . . . . . . . . . . . . . . . 130
5.3.3.1 Key Lessons . . . . . . . . . . . . . . . . . . 130
5.3.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . 131
5.4 Conclusions and Future Work . . . . . . . . . . . . . . . . . 132
6 Building a Function Testing Platform for Complex
Scientific Code 135
Dali Wang, Zhuo Yao, and Frank Winkler
6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
6.2 Software Engineering Challenges for Complex Scientific Code 136
6.3 The Purposes of Function Unit Testing for Scientific Code . 136
6.4 Generic Procedure of Establishing Function Unit Testing for
Large-Scale Scientific Code . . . . . . . . . . . . . . . . . . . 137
6.4.1 Software Analysis and Testing Environment
Establishment. . . . . . . . . . . . . . . . . . . . . . . 138
6.4.2 Function Unit Test Module Generation . . . . . . . . 139
6.4.3 Benchmark Test Case Data Stream Generation Using
Variable Tracking and Instrumentation. . . . . . . . . 139
6.4.4 Function Unit Module Validation . . . . . . . . . . . . 139
6.5 Case Study: Function Unit Testing for the ACME Model . . 140
6.5.1 ACME Component Analysis and Function Call-Tree
Generation . . . . . . . . . . . . . . . . . . . . . . . . 140
6.5.2 Computational Characteristics of ACME Code . . . . 141
6.5.3 A Function Unit Testing Platform for ACME Land
Model . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.5.3.1 System Architecture of ALM Function Test
Framework . . . . . . . . . . . . . . . . . . . 144
6.5.3.2 WorkingProcedureoftheALMFunctionTest
Framework . . . . . . . . . . . . . . . . . . . 146
6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 148