Introduction to numerical programming: a practical guide for scientists and engineers using Python

663 Pages·2015·8.35 MB·English

by  Titus A. Beu

Computer Science/Scientifc Programming
SERIES IN COMPUTATIONAL PHYSICS
Steven A. Gottlieb and Rubin H. Landau, Series Editors
“Your journey will be a real pleasure since the book focuses on thorough explanations, hands-on
code examples, and graphical representations.”
—Professor Dr. Alexander K. Hartmann, Institute for Physics, University of Oldenburg
“… an invaluable resource for aspects that are often not suffciently emphasised, despite their
importance for reliable calculations. I strongly recommend it for everyone’s bookshelf.”
—Professor Joan Adler, Technion, Israel Institute of Technology
“… a comprehensive introduction to classical numerical methods… via clear and simple codes
in Python and C/++. … I will recommend it to my students.”
—Professor Mike Wheatland, The University of Sydney
Makes Numerical Programming More Accessible to a Wider Audience
Bearing in mind the evolution of modern programming, most specifcally emergent program-
ming languages that refect modern practice, Numerical Programming: A Practical Guide for
Scientists and Engineers Using Python and C/C++ utilizes the author’s many years of practi-
cal research and teaching experience to offer a systematic approach to relevant programming
concepts. Adopting a practical, broad appeal, this user-friendly book offers guidance to anyone
interested in using numerical programming to solve science and engineering problems. Empha-
sizing methods generally used in physics and engineering—from elementary methods to com-
plex algorithms—it gradually incorporates algorithmic elements with increasing complexity.
Develop a Combination of Theoretical Knowledge, Effcient Analysis Skills,
and Code Design Know-How
The book encourages algorithmic thinking, which is essential to numerical analysis. Establish-
ing the fundamental numerical methods, application numerical behavior and graphical output
needed to foster algorithmic reasoning, coding dexterity, and a scientifc programming style, it
enables readers to successfully navigate relevant algorithms, understand coding design, and de-
velop effcient programming skills. The book incorporates real code and includes examples and
problem sets to assist in hands-on learning.
This text introduces platform-independent numerical programming using Python and C/C++ and
appeals to advanced undergraduate and graduate students in natural sciences and engineering,
researchers involved in scientifc computing, and engineers carrying out applicative calculations.
K16451
K16451_cover.indd 1 7/16/14 11:03 AM
Beu INTRODUCTION TO NUMERICAL PROGRAMMING
INTRODUCTION TO
NUMERICAL
PROGRAMMING
SERIES IN COMPUTATIONAL PHYSICS
Steven A. Gottlieb and Rubin H. Landau
Series Editors
Parallel Science and Engineering Applications: The Charm++ Approach
Sanjay Kale and Abhinav Bhatele, Eds.
Introduction to Numerical Programming: A Practical Guide for Scientists
and Engineers Using Python and C/C++
Titus Adrian Beu
Forthcoming
Visualization in Computational Physics and Materials Science
Joan Adler
Introduction to Classical Dynamics: A Computational View
Kelly Roos and Joseph Driscoll
SERIES IN COMPUTATIONAL PHYSICS
Steven A. Gottlieb and Rubin H. Landau, Series Editors
INTRODUCTION TO
NUMERICAL
PROGRAMMING
A Practical Guide for Scientists and Engineers
Using Python and C/C++
Titus Adrien Beu
Babeș-Bolyai University
Faculty of Physics
Cluj-Napoca, Romania
CRC Press
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300
Boca Raton, FL 33487-2742
© 2015 by Taylor & Francis Group, LLC
CRC Press is an imprint of Taylor & Francis Group, an Informa business
No claim to original U.S. Government works
Version Date: 20140716
International Standard Book Number-13: 978-1-4665-6968-3 (eBook - PDF)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts have been
made to publish reliable data and information, but the author and publisher cannot assume responsibility for the valid-
ity of all materials or the consequences of their use. The authors and publishers have attempted to trace the copyright
holders of all material reproduced in this publication and apologize to copyright holders if permission to publish in this
form has not been obtained. If any copyright material has not been acknowledged please write and let us know so we may
rectify in any future reprint.
Except as permitted under U.S. Copyright Law, no part of this book may be reprinted, reproduced, transmitted, or uti-
lized in any form by any electronic, mechanical, or other means, now known or hereafter invented, including photocopy-
ing, microfilming, and recording, or in any information storage or retrieval system, without written permission from the
publishers.
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http://
www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923,
978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For
organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for
identification and explanation without intent to infringe.
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
To Mihaela and Victor
Contents
Series Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Author . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xvii
Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xix
1 Approximate Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.1 Sources of Errors in Numerical Calculations . . . . . . . . . . . . . . . . 1
1.2 Absolute and Relative Errors . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.3 Representation of Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . 4
1.4 Significant Digits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.5 Errors of Elementary Operations . . . . . . . . . . . . . . . . . . . . . . . 7
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 10
2 Basic Programming Techniques . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1 Programming Concepts . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.2 Functions and Parameters . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.3 Passing Arguments to Python Functions . . . . . . . . . . . . . . . . . . 15
2.4 Passing Arguments to C/C++ Functions . . . . . . . . . . . . . . . . . . 17
2.5 Arrays in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.6 Dynamic Array Allocation in C/C++ . . . . . . . . . . . . . . . . . . . . 19
2.7 Basic Matrix Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 30
3 Elements of Scientific Graphics . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.1 The Tkinter Package . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
3.2 The Canvas Widget . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
3.3 Simple Tkinter Applications . . . . . . . . . . . . . . . . . . . . . . . . . 35
3.4 Plotting Functions of One Variable . . . . . . . . . . . . . . . . . . . . . 39
3.5 Graphics Library graphlib.py . . . . . . . . . . . . . . . . . . . . . . 44
3.6 Creating Plots in C++ Using the Library graphlib.py . . . . . . . . 57
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 61
4 Sorting and Indexing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
4.2 Bubble Sort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
4.3 Insertion Sort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
vii
viii Contents
4.4 Quicksort . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
4.5 Indexing and Ranking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
4.6 Implementations in C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 75
4.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 84
5 Evaluation of Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
5.1 Evaluation of Polynomials by Horner’s Scheme . . . . . . . . . . . . . . 85
5.2 Evaluation of Analytic Functions . . . . . . . . . . . . . . . . . . . . . . . 88
5.3 Continued Fractions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
5.4 Orthogonal Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
5.5 Spherical Harmonics—Associated Legendre Functions . . . . . . . . . . 98
5.6 Spherical Bessel Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 102
5.7 Implementations in C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 105
5.8 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 125
6 Algebraic and Transcendental Equations . . . . . . . . . . . . . . . . . . . . . . 127
6.1 Root Separation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
6.2 Bisection Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
6.3 Method of False Position . . . . . . . . . . . . . . . . . . . . . . . . . . . 132
6.4 Method of Successive Approximations . . . . . . . . . . . . . . . . . . . 134
6.5 Newton’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
6.6 Secant Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
6.7 Birge–Vieta Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
6.8 Newton’s Method for Systems of Nonlinear Equations . . . . . . . . . . 147
6.9 Implementations in C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 151
6.10 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 157
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 168
7 Systems of Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 169
7.2 Gaussian Elimination with Backward Substitution . . . . . . . . . . . . 169
7.3 Gauss–Jordan Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . 179
7.4 LU Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 187
7.5 Inversion of Triangular Matrices . . . . . . . . . . . . . . . . . . . . . . . 195
7.6 Cholesky Factorization . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
7.7 Tridiagonal Systems of Linear Equations . . . . . . . . . . . . . . . . . . 203
7.8 Block Tridiagonal Systems of Linear Equations . . . . . . . . . . . . . . 207
7.9 Complex Matrix Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 208
7.10 Jacobi and Gauss–Seidel Iterative Methods . . . . . . . . . . . . . . . . . 209
7.11 Implementations in C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 213
7.12 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 231
8 Eigenvalue Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233
8.2 Diagonalization of Matrices by Similarity Transformations . . . . . . . 234
8.3 Jacobi Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
8.4 Generalized Eigenvalue Problems for Symmetric Matrices . . . . . . . . 243
8.5 Implementations in C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 246
Contents ix
8.6 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 264
9 Modeling of Tabulated Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 265
9.1 Interpolation and Regression . . . . . . . . . . . . . . . . . . . . . . . . . 265
9.2 Lagrange Interpolation Polynomial . . . . . . . . . . . . . . . . . . . . . 268
9.3 Neville’s Interpolation Method . . . . . . . . . . . . . . . . . . . . . . . . 273
9.4 Cubic Spline Interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 276
9.5 Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
9.6 Multilinear Regression Models . . . . . . . . . . . . . . . . . . . . . . . . 287
9.7 Nonlinear Regression: The Levenberg–Marquardt Method . . . . . . . 293
9.8 Implementations in C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 301
9.9 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 331
10 Integration of Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
10.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 333
10.2 Trapezoidal Rule; A Heuristic Approach . . . . . . . . . . . . . . . . . . 333
10.3 The Newton–Cotes Quadrature Formulas . . . . . . . . . . . . . . . . . 335
10.4 Trapezoidal Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
10.5 Simpson’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 339
10.6 Adaptive Quadrature Methods . . . . . . . . . . . . . . . . . . . . . . . . 341
10.7 Romberg’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344
10.8 Improper Integrals: Open Formulas . . . . . . . . . . . . . . . . . . . . . 348
10.9 Midpoint Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 352
10.10 Gaussian Quadratures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
10.11 Multidimensional Integration . . . . . . . . . . . . . . . . . . . . . . . . 361
10.12 Adaptive Multidimensional Integration . . . . . . . . . . . . . . . . . . . 369
10.13 Implementations in C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 372
10.14 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 393
11 Monte Carlo Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
11.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
11.2 Integration of Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
11.3 Importance Sampling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399
11.4 Multidimensional Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . 402
11.5 Generation of Random Numbers . . . . . . . . . . . . . . . . . . . . . . 408
11.6 Implementations in C/C++ . . . . . . . . . . . . . . . . . . . . . . . . . . 415
11.7 Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 417
References and Suggested Further Reading . . . . . . . . . . . . . . . . . . . . 426
12 Ordinary Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
12.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 427
12.2 Taylor Series Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
12.3 Euler’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 431
12.4 Runge–Kutta Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
12.5 Adaptive Step Size Control . . . . . . . . . . . . . . . . . . . . . . . . . . 440
12.6 Methods for Second-Order ODEs . . . . . . . . . . . . . . . . . . . . . . 447
12.7 Numerov’s Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 454
12.8 Shooting Methods for Two-Point Problems . . . . . . . . . . . . . . . . 457