Python Recipes for Earth Sciences

463 Pages·2022·11.071 MB·English

by  Martin H. Trauth

Springer Textbooks in Earth Sciences,
Geography and Environment
The Springer Textbooks series publishes a broad portfolio of textbooks on Earth
Sciences, Geography and Environmental Science. Springer textbooks provide com-
prehensive introductions as well as in-depth knowledge for advanced studies. A
clear, reader-friendly layout and features such as end-of-chapter summaries, work
examples, exercises, and glossaries help the reader to access the subject. Springer
textbooks are essential for students, researchers and applied scientists.
Martin H. Trauth
Python Recipes for Earth
Sciences
Martin H. Trauth
Institute of Geosciences
University of Potsdam
Potsdam, Brandenburg, Germany
ISSN 2510-1307 ISSN 2510-1315 (electronic)
Springer Textbooks in Earth Sciences, Geography and Environment
ISBN 978-3-031-07718-0 ISBN 978-3-031-07719-7 (eBook)
https://doi.org/10.1007/978-3-031-07719-7
© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland
AG 2022
This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether
the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse
of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and
transmission or information storage and retrieval, electronic adaptation, computer software, or by similar
or dissimilar methodology now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, expressed or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.
Responsible Editor: Dr. Annett Büttner
This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Preface
The book Python Recipes for Earth Sciences is designed to help undergraduate
and postgraduate students, doctoral students, post-doctoral researchers, and
professionals alike find quick solutions to common data analysis problems in
the earth sciences. It provides a minimal amount of theoretical background and
demonstrates the application of all described methods via examples. The book
is based on the popular MATLAB Recipes for Earth Sciences book that I wrote
many years ago, which is now available in its 5th edition (Springer 2021). The
idea for this new book was to translate the text of the MATLAB book to another,
increasingly popular programming language in the spirit of the classic Numerical
Recipes book by W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery
(Cambridge University Press 2007).
The present book contains Python scripts that can be used to solve typical
problems in the earth sciences via simple statistics, time series analysis,
geostatistics, and image processing. It also demonstrates the application of
selected advanced techniques of data analysis, such as nonlinear time series
analysis, adaptive filtering, bootstrapping, and terrain analysis. The book’s
supplementary electronic material (available online through Springer Link)
includes recipes that contain all the example data and all the Python commands
featured in the book. The Python codes can be easily modified for use with the
reader’s own data and projects. The Python software, which has existed for more
than 30 years, is used since it not only provides numerous ready-to-use algorithms
for most methods of data analysis, but also allows the existing routines to be
modified and expanded and additionally enables new software to be developed.
However, most methods are introduced with very simple codes before any Python-
specific functions are used. As a result, users of alternatives to Python will also
find value in this book.
In order to derive the maximum benefit from this book, the reader will need to
have access to the Python software and be able to execute the recipes while read-
ing the book. The Python recipes yield various graphs on the screen that are not
shown in the printed book. The tutorial-style book does, however, contain numer-
ous figures, thereby making it possible to go through the text without actually run-
V
VI Preface
ning Python on a computer. I developed the recipes using Python 3.8.8, though
most recipes will also work with earlier software releases, but not with Python 2.
While undergraduates participating in a course on data analysis might wish to go
through the entire book, more experienced readers may choose to refer to only one
particular method in order to solve a specific problem. The concept of the book
and the contents of its chapters are therefore outlined below in order to make it
easier to use for readers with a variety of different requirements.
• Chapter 1—This chapter introduces some fundamental concepts of samples and
populations. It also links the various types of data and questions that can be
answered using the data to the methods described in the subsequent chapters.
• Chapter 2—This chapter offers a tutorial-style introduction to Python designed
for earth scientists. Readers who are already familiar with the software are
advised to proceed directly to the subsequent chapters.
• Chapters 3 and 4—These two chapters cover fundamentals in univariate and
bivariate statistics. They contain basic concepts in statistics and also introduce
advanced topics, such as resampling schemes and cross-validation. Readers
who are already familiar with basic statistics might want to skip these chapters.
• Chapter 5 and 6—Readers who wish to work with time series are advised to
read both of these chapters. Time series analysis and signal processing are
closely linked. A good knowledge of statistics is required to work successfully
with these methods.
• Chapters 7 and 8—I recommend reading through both of these chapters since
the processing methods used for spatial data and for images have much in
common. Moreover, spatial data and images are often combined in the earth
sciences, for instance, when projecting satellite images onto digital elevation
models.
• Chapter 9—Data sets in the earth sciences often have many variables and many
data points. Multivariate methods are applied to a great variety of large data
sets, including satellite imagery data. Any reader who is particularly interested
in multivariate methods is advised to read Chaps. 3 and 4 before proceeding to
this chapter.
• Chapter 10—Methods for analyzing circular and spherical data are widely used
in the earth sciences. Structural geologists, for example, measure and analyze
the orientation of slickensides (or striae) on a fault plane, and the statistical
analysis of circular data is used in paleomagnetic applications. Microstructural
investigations include the analysis of grain shapes and quartz c-axis orienta-
tions in thin sections.
When using this book, it is important to keep in mind that it is based on a book
that uses MATLAB for data analysis in the earth sciences. An attempt has been
made to remain as close as possible to the original didactic concept and text of the
MATLAB book, even if it may seem unusual or awkward for some Python users.
On the other hand, this didactic concept will be familiar to the many users who,
Preface VII
for whatever reason, have made switch from MATLAB to Python. These users,
like me, are familiar with MATLAB and first look for similar solutions in Python
instead of programming Python code from scratch. This happens very often and
is reflected in many Python packages, such as NumPy and Matplotlib, which are
highly similar to MATLAB toolboxes, right down to their identical function names
and input arguments.
Readers who place both books side by side will notice that I have directly trans-
lated the code in order to make the numerical and graphical output as similar as
possible. However, since different random generators are used in MATLAB and
Python, the results do not always match with one another. In these cases, after
generating synthetic data, the sample files generated with MATLAB are used in
the Python examples in order to reproduce the results. There may also be some
discrepancies in cases for which MATLAB produces a result of zero but for which
Python produces a result slightly different from zero, such as –3.0552e–17. Such
discrepancies are likely due to the less stringent floating-point error mitigation in
many Python packages.
The attentive reader will also notice that in some cases, there are minor
discrepancies between the graphs generated with Python and the illustrations in
the first half of the book. This is also due to the fact that the random generators
used here provide different results and that not all of the used random generators
can be reset. Therefore, some results (e.g., the bootstrapping results) are different
between individual runs. However, the figures at least qualitatively reflect the
results of statistics, time series analysis, and signal processing. For the second
half of the book, the graphics were newly created to reflect the peculiarities in the
output of graphics packages such as Matplotlib and PyGMT, which certainly led to
some very nice results.
I wrote the Python scripts, which are based on the MATLAB scripts, with the
best of my knowledge and with the support of the colleagues mentioned below.
However, I cannot offer any warranty, especially when using software such as
Python and the Python packages, which themselves offer no warranty. For exam-
ple, I have found a number of discrepancies between results from Python and
MATLAB, which I attribute to minor floating-point issues with some Python pack-
ages, such as the use of very small values instead of (correctly) zero in the calcula-
tion. I have flagged these discrepancies in the text. I recommend that the reader
always test the code (be it with Python or MATLAB) with synthetic examples
before applying it to real examples because it is also possible that users have made
their own mistakes when using floating-point numbers.
This book has benefited from the comments of many people, in particular from
my contributing authors, Robin Gebbers and Norbert Marwan, as well as from
many colleagues and friends worldwide who have read the work, pointed out
mistakes, and made suggestions for improvement. I wish thank Ed Manning and
Ryan DeLaney for their professional proofreading, which has greatly improved
the readability of my book. I also very much appreciate the expertise and patience
of Elisabeth Sillmann at blaetterwaldDesign, who has supported and advised me
with the graphic design. I am very grateful to Annett Büttner, Marion Schneider,
VIII Preface
Helen Rachner, Nirmal Iyer, Stefan Kreickenbaum, and their team at Springer for
their continuing interest and support in publishing my books. I am very grateful
to Nadine Berner, Bodo Bookhagen, Matt Hall, Hauke Krämer, Pawel Lacho-
vicz, Norbert Marwan, and Maurizio Petrelli for answering my Python questions.
I would also like to thank Brunhilde Schulz, Andreas Bohlen, Ingo Orgzall, and
their team at UP Transfer GmbH for organizing short courses on the book through-
out the past two decades at the universities of Kiel, Bremen, Bratislava, Ghent,
Munich, Nairobi, Cologne, Stockholm, Amsterdam, Aberystwyth, and Potsdam
as well as at UA Barcelona, UC London, Brown University Providence, the BGR
Hannover, the BGI Bayreuth, the NHM Vienna, and GNS Science Wellington.
I additionally wish to thank the NASA/GSFC/METI/ERSDAC/JAROS and the
U.S./Japan ASTER Science Team as well as director Mike Abrams for allowing me
to include the ASTER images in this book. I am grateful to Stuart W. Frye and
his team at the NASA Goddard Space Flight Center (GSFC) for allowing me to
include EO-1 data in the book as well as for our fruitful discussions while working
on the section about Hyperion images.
Potsdam Martin H. Trauth
April 2022
Contents
1 Data Analysis in the Earth Sciences .......................... 1
1.1 Introduction ......................................... 1
1.2 Data Collection ...................................... 1
1.3 Types of Data ........................................ 3
1.4 Methods of Data Analysis .............................. 6
Recommended Reading ..................................... 8
2 Introduction to Python ..................................... 9
2.1 Introduction ......................................... 9
2.2 Getting Started ....................................... 11
2.3 Python Syntax ....................................... 12
2.4 Array Manipulation ................................... 19
2.5 Data Types in Python .................................. 25
2.6 Data Storage and Handling ............................. 37
2.7 Control Flow ........................................ 43
2.8 Scripts and Functions .................................. 47
2.9 Basic Visualization Tools ............................... 50
2.10 Generating Code to Recreate Graphics .................... 54
2.11 Publishing and Sharing MATLAB Code ................... 54
2.12 Creating Graphical User Interfaces ....................... 54
Recommended Reading ..................................... 55
3 Univariate Statistics ....................................... 57
3.1 Introduction ......................................... 57
3.2 Empirical Distributions ................................ 58
3.3 Examples of Empirical Distributions ...................... 63
3.4 Theoretical Distributions ............................... 71
3.5 Examples of Theoretical Distributions. . . . . . . . . . . . . . . . . . . . . 79
3.6 Hypothesis Testing .................................... 81
3.7 The t-Test ........................................... 84
3.8 The F-Test .......................................... 88
3.9 The χ2-Test ......................................... 92
IX