Jamison Conley A Geographer’s Guide to Computing Fundamentals Python in ArcGIS Pro Springer Textbooks in Earth Sciences, Geography and Environment The Springer Textbooks series publishes a broad portfolio of textbooks on Earth Sciences, Geography and Environmental Science. Springer textbooks provide comprehensive introductions as well as in-depth knowledge for advanced studies. A clear, reader-friendly layout and features such as end-of-chapter summaries, work examples, exercises, and glossaries help the reader to access the subject. Springer textbooks are essential for students, researchers and applied scientists. Jamison Conley A Geographer's Guide to Computing Fundamentals Python in ArcGIS Pro Jamison Conley Department of Geography and Geology West Virginia University Morgantown, WV, USA ISSN 2510-1307 ISSN 2510-1315 (electronic) Springer Textbooks in Earth Sciences, Geography and Environment ISBN 978-3-031-08497-3 ISBN 978-3-031-08498-0 (eBook) https://doi.org/10.1007/978-3-031-08498-0 © The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2022 This work is subject to copyright. All rights are solely and exclusively licensed by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use. The publisher, the authors, and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, expressed or implied, with respect to the material contained herein or for any errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Cover Image credit: Art by Jocelyn J. Nolan This Springer imprint is published by the registered company Springer Nature Switzerland AG The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland This book is dedicated to my family, and especially my wife, Lee Ann Nolan. Without her guidance, this book would not have begun. Without her patience and support, this book would not have been completed. I am grateful to her for all her help, and a dedication here is insufficient thanks, but it’s a nice start. Preface Like many textbooks, this book is an outgrowth of classroom activities that I have designed over several years and iterations of teaching a course on GIS programming. It grew out of programming lab exercises that I had developed, and the recogni- tion that having the exercises integrated with the rest of the material, instead of separated into their own sections, or online supplemental material, greatly assists students’ understanding of how the computer science principles work and helps them apply those principles, especially when shifting from one programming language to another. Like students in my classes, I am not assuming that you, the reader, have any background in computer science or program- ming experience. I am, though, assuming you are very familiar with ArcGIS Pro. If you are unfamiliar with ArcGIS Pro, there are many resources available from ESRI and other sources. ESRI offers a range of Web trainings through http://www. esri.com/training/, which may provide a quick way to get up to speed on ArcGIS Pro. Here is how I envision you might use this book. I developed it for use in a course I teach on GIS programming, although you might be coming to it from the perspective of a researcher or professional who wants to employ Python in your GIS work, instead of the student perspective. There are 21 chapters organized into 4 parts. Part I, titled Getting Acquainted with Python, which has Chaps. 1–6, serves as an introduction to computer science principles in general and Python in particular. This part is important for those with little or no programming experience, or in other words, those for whom terms like variable, statement, and function are unknown. Part II moves from general programming and Python to the application of programming within ArcGIS Pro. It does so through the lens of data structures, such as rasters and vectors. Part III builds upon data structures and then addresses the creation of algorithms and the development of new tools within ArcGIS Pro to implement and use those algorithms. Finally, Part IV expands upon this and then moves to more complex programs that involve multiple Python files, and the development of Python Packages to use these within ArcGIS Pro. In a classroom setting, I use Parts I–III in a fast-paced single-semester course. With more time, I would suggest a two- course sequence starting with Parts I and II in the first semester and Parts III and IV in the second semester. In a non-class- room setting, I would encourage those who have little or no programming background to start with Part I, while those who have programming experience and only want to apply existing Python knowledge to ArcGIS Pro can jump into Part II and beyond. Most chapters are structured to encourage readers to understand both the computer science principles that underly pro- gramming within a GIS context as well as the application of those principles. This comes from having the following struc- ture, which embeds practical exercises within the text of the chapter. They begin with opening remarks, which usually introduce one or more relevant computer science concepts for the chapter in question. Then there is a series of guided tasks which take the student through the concepts introduced in the opening remarks. These tasks also elaborate on the remarks, presenting more complex aspects of the topic alongside the software within the task. The guided tasks have instructions for readers to complete within Python and ArcGIS Pro, as well as questions which gauge and reinforce their understanding of the material as it is presented. In an instructional context, these can provide the questions for computer lab activities that parallel the book. Lastly, the chapters conclude with an unguided task. This challenges the readers to reinforce and apply the concepts and skills from the introduction and the guided tasks to a new situation. These unguided tasks also have a dedicated debugging task included within most chapters following the chapter on debugging (Chap. 6) in order to develop and reinforce debugging skills. I encourage the reader to follow along with the guided tasks as you proceed through the chapter and take the time to debug the debugging exercises. These are based upon the most common bugs students have presented to me over the course of 13 years of teaching this material, so learning to recognize and fix these bugs can be very helpful to your suc- cess as a programmer. Lastly, taking the time to complete the unguided tasks will help reinforce the application of the prin- ciples within the chapter. vii viii Preface Together, the overall structure of the book, as well as the chapter structure mentioned above, seeks to prepare the readers for a long-term ability to do GIS programming, whether in industry or academic research. In particular, the intent of integrat- ing computer science principles and the exercises into the same chapter is that a firmer understanding of the principles will help the reader transition from one language, such as VBA in ArcMap, to a new platform, such as Python in ArcGIS Pro. This understanding prepares readers for working in a dynamic, rapidly changing technology field. Credits are due to Esri, both in general for the development of the software and in specific for use of the screen captures. Screenshot images are the intellectual property of Esri and is used herein with permission. Copyright © 2021 Esri and its licensors. All rights reserved. As the dedication says, no project that even approaches the magnitude of a book happens alone. I would be remiss if I did not acknowledge those who have assisted me along the way to this guide, whether intentionally or unwittingly. Those whose influence is most direct are the 100-plus students who took the GIS programming courses I have taught from 2009 until the publication date of this book, especially those from 2019 to 2021, whose course text consisted of early drafts of Chaps. 1–17 of this book. They provided valuable feedback in the revision process. Those whose influence is less direct are the colleagues and especially the instructors I have had as an undergraduate and graduate student at Gustavus Adolphus College and Pennsylvania State University. Their modes of teaching both geography and computer science have undoubtedly filtered into how I teach, and thereby the content of this book. Especial gratitude for effective teaching in this regard goes to Max Hailperin and Mark Gahegan. Lastly, the most important influence comes from my family. They provided an environment and setting that allowed me to complete this book, whether through stepping up to help out (thank you, Jason Nolan!), gen- eral encouragement (thank you, Jocelyn and Evan, my stepchildren!), or the many ways my wife, herself a librarian, has guided me through the writing and publication process. This truly could not have come to completion without her. Morgantown, WV, USA Jamison Conley Contents Part I Getting Acquainted with Python 1 Introduction to Scripting and Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Introductory Comments: Scripting Versus Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 Task 1: Why Use Scripting? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Task 2: Algorithms and Computational Thinking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 Task 3: Non-guided Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2 Basics of Programming: Variables, Operators, and Statements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Introductory Comments: Parsing a Line of Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Task 1: Basics of Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Variables, Literals, and Operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 Sequences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 Delimiters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 Expressions and Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Task 2: Finding and Fixing Bugs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 Task 3: Unguided Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3 Basics of Programming: Control Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Introductory Comments: Enabling the Computer to Make Simple Decisions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Task 1: Explore the Python Notebook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Task 2: Conditional Statements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Task 3: Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Task 4: Parsing Complex Statements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Task 5: Debugging Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Task 6: Unguided Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 4 Basics of Programming: Functions and Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Introductory Comments: Managing More Complex Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Task 1: Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 Task 2: Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Task 3: Parsing Functions and Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Task 4: Debugging Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Task 5: Non-guided Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 5 Reading a Python Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Introductory Comments: The Value of Documentation and Comments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Task 1: Importing a Python File into IDLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Task 2: Use Online Python Help Documentation to Understand Part of the Script . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Task 3: Running Code Statement by Statement to Understand It . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 Task 4: Using Functions to Examine a Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 print Task 5: Using the Function to Examine the Contents of a File . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55 next() Task 6: Adding Comments to the Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Task 7: Non-guided Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 ix x Contents 6 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Introductory Comments: The Types of Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Task 1: Debugging in the Python Notebook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Task 2: Using the IDLE Debugger . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Task 3: Non-guided Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Part II Data Structures in ArcPy and Programming 7 Introduction to ArcPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Introductory Comments: Objects and Classes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Task 1: Reading ArcPy Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Task 2: Accessing and Examining Basic ArcPy Classes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 Task 3: Using a Tool Through ArcPy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 Task 4: Debugging Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85 Task 5: Non-guided Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 8 Basics of Accessing Data in ArcGIS. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Introductory Comments: Superclasses, Subclasses, and Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Task 1: Using arcpy.Describe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Task 2: Inheritance of Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Task 3: Running an ArcPy Script in IDLE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Task 4: Debugging Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Task 5: Non-guided Work: Writing Your Own Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 9 Rasters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Introductory Comments: GIS-Specific Data Structures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Task 1: Slope Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Task 2: Reclassification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Task 3: Aspect Considerations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101 Task 4: Map Algebra . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Task 5: Debugging Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Task 6: Unguided Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 10 Tables and Cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Introductory Comments: How a Computer Represents Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Task 1: Introduction to Search Cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Task 2: Introduction to Update Cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Task 3: Deleting and Inserting Rows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Task 4: More Advanced Use of Cursors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108 Task 5: Debugging Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 Task 6: Writing Your Own Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 11 Vectors and Geometries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Introductory Comments: Data Structures for Points, Lines, and Polygons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 Task 1: Access Shape Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 Task 2: Use Geometry-Related Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 Task 3: Creating New Geometries with Buffers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Task 4: Carry Out Geometry Operations with Two Polygons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 Task 5: Creating a New Geometry from Scratch . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121 Task 6: Debugging Exercise . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 Task 7: Unguided Work. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 Challenge Task 8 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123