Table Of Content

Earth Systems Data and Models Daniel McInerney Pieter Kempeneers Open Source Geospatial Tools Applications in Earth Observation Earth Systems Data and Models Series editors Bernd Blasius, Oldenburg, Germany William Lahoz, Reading, UK Dimitri P. Solomatine, Delft, The Netherlands More information about this series at http://www.springer.com/series/10525 Daniel McInerney Pieter Kempeneers (cid:129) Open Source Geospatial Tools Applications in Earth Observation 123 Daniel McInerney PieterKempeneers Laboratoire desSciences duClimat etde Remote Sensing(TAP) l’Environnement VITO NV Commissariat aL’Energie Atomique Mol Gif-Sur-Yvette Belgium France ISBN 978-3-319-01823-2 ISBN 978-3-319-01824-9 (eBook) DOI 10.1007/978-3-319-01824-9 LibraryofCongressControlNumber:2014951735 SpringerChamHeidelbergNewYorkDordrechtLondon ©SpringerInternationalPublishingSwitzerland2015 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionor informationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserofthe work. Duplication of this publication or parts thereof is permitted only under the provisions of theCopyrightLawofthePublisher’slocation,initscurrentversion,andpermissionforusemustalways beobtainedfromSpringer.PermissionsforusemaybeobtainedthroughRightsLinkattheCopyright ClearanceCenter.ViolationsareliabletoprosecutionundertherespectiveCopyrightLaw. The use of general descriptive names, registered names, trademarks, service marks, etc. in this publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse. While the advice and information in this book are believed to be true and accurate at the date of publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with respecttothematerialcontainedherein. Printedonacid-freepaper SpringerispartofSpringerScience+BusinessMedia(www.springer.com) To our families: Sian, Lelia and Hugo Helga, Pauline, Charlotte and Finn Foreword Over the years I have heard people refer to one package or another as the “swiss armyknifeofGIStools”.Buttheideaofaswissarmyknifeistohavetheminimum setofcapabilitieswithyouwhenitishardtohaveeverything.Indesktopandcloud geospatial processing the better choice is to have a whole toolbox! The suite of open source geospatial tools available to academic, research, gov- ernment, and commercial users of geospatial data is rich and production ready. Often, the greatest challenge in utilizing them is understanding where the desired capabilities exist and how they can be connected. Pieter and Daniel bring together information on a great suite or toolbox of geospatialsoftware,whichcanbeeasilymixedandmatchedtodoawidevarietyof tasks. The Orfeo Toolbox has a sophisticated suite of advanced image processing operations. The GDAL/OGR suite offers data translation, reorganization, and reprojectionandpktoolsaddsabunchofdatamanipulationtoolsmissinginGDAL/ OGR, essentially “the missing bits” and more. Desktop GIS in graphical user interface environment is widely used, and open sourcepackagessuchasQGISareagreatasset.Butwhenitcomestoheavylifting forgeospatialdataprocessing,theflexibilityandscriptabilityofasuiteofcommand linetoolscannotbebeaten.Withcommandlinetoolsitispossibletoautomatethe processingoflargeamountsofdata.Itisalsopossibletoensurethatourprocessing stepsarerepeatable—thatwe havecaptured allthestepsinaclearscript.Wehave alsomadeiteasiertocommunicatewhatwehavedoneintheformoftextualscripts of a set of command line operations. Thetoolspresentedherehavebeentheworkhorses,whetherfromthecommand line or from Python wrappers at many organizations. Much of the raster data preparation for Google Earth has been done with these tools and Python scripting, and we are using a similar process in my current job at Planet Labs, where we are working towards building global daily mosaics at better than 5 m resolution. vii viii Foreword I hope this book will help you fill your geospatial toolbox. But in our modern “post scarcity” world, realize your toolbox can continue to grow. Keep exploring the details of the packages you know, and capabilities in other packages from the growing suite of the open source geospatial community! Mountain View, CA, USA, June 2014 Frank Warmerdam Preface This book is about open source geospatial command line software and has been written with a wide audience in mind that includes graduate students, scientific researchers, geospatial, and remote sensing analysts and practitioners. Its broad objectiveistoprovideaclearandaccessibleintroductiontotherangeofcommand line utilities supported by illustrative examples. One of the aims of the book is to demonstrate the power and efficiency of the command line to audiences that have predominantly used software packages driven by graphical user interfaces. It is hoped that this book will provide the novice command line user with sufficient insights to be able to integrate the tools into their day-to-day workflows. Never- theless, we also anticipate that it will also serve as a reference guide to the more seasoned command line user wishing to review the specifics of a tool or refresh their understanding of the implementation of a tool. On embarking on this book, we set ourselves an objective of de-mystifying the commandlineandshowingitsrelevancetogeospatialanalysis.Weappreciatethat forthosethathavelittleornoexperiencewiththecommandline,itcanseemlikea very daunting task to learn. In parallel, we acknowledge that there is a steep learning curve associated in getting to grips with it. We would like to emphasize that this early investment in time will soon be repaid many times when your familiaritywiththecommandlineincreasesandyoucanquicklyre-runcommands or sets of commands chained together in scripts. While there are many packages currently available that can be used for geo- spatialanalysis,thisworkfocusesonaspecificsubsetofthem.Thisisdueeitherto their unique functionality that they provide or the fact that they underpin other software packages. For vector and raster data processing, they are: (cid:129) GDAL/OGR (cid:129) pktools (cid:129) Orfeo Toolbox In less detail, we also cover a number of packages that deal with three-dimen- sional point clouds obtained from LiDAR instruments. Rather than giving an ix x Preface exhaustive description, we provide a brief overview with some usage examples of the following packages: (cid:129) liblas (cid:129) PDAL (cid:129) LAStools (cid:129) PulseTools (cid:129) SPDLib A key motivating factor behind this undertaking is the capacity of the selected toolstoprovidehighlyeffectivecapabilitiesforprocessinggeospatialdata.Another is the fact that while they continue to be developed at a rapid rate, the documen- tationandrelatedtrainingmaterialassociatedwiththemaredistributedondisparate websites such that the consolidation of the explanation of a particular tool and its commandlineoptionsisdeemednecessary.Thetoolsthatwecoverinthebookall have the following in common: (cid:129) free and open source; (cid:129) ability to operate in batch mode (command line processing); (cid:129) available on a Linux operating system and other systems. These three requirements are described in more detail. First, free and open source software has gained in popularity over the last decade. The terms free and open source have been used in different contexts with different meanings. Without entering into the discussion, it is worth recalling the idea of the free software foundation (FSF), in the following terms: “Free as in free speech, not as in free beer.” The philosophical view of freedom is ethical and goes beyond the idea of opensource,whichisratherfocusedonpracticalissues,suchasmakingorhaving powerful, reliable software (Stallman 2009). Nevertheless, the two categories overlap. The unrestricted use offree software isparticularly important for capacity building, in particular in developing countries. With the access to remote sensing data archives (Landsat, MODIS, Proba-V, and the upcoming Sentinel missions) becoming more open, both data and processing software are now available to a large community. This provides new opportunities for research in Earth system sciences and its related policy support. Furthermore, command line processing is a powerful technique that is, in the viewoftheauthors,currentlyunderused.Thisisparticularlytrueforgeospatialdata analysis where datasets typically require processing and manipulation. There are three ways in which users can interact with software. The first is a graphical user interface(GUI),whichiswellknowninWindowsenvironmentsandtypicallyused in most proprietary software. Users interact with the software via menus, dialog boxes, and mouse actions (move, clicks, drag, and drop). In many cases the software combines a development environment with an image viewer in a so-called what-you-see-is-what-you-get(WYSIWYG)application.Thisallowstheusertosee anddirectlyinteractwiththeresultsoftheactionsperformedonthegeospatialdata. Suchapplicationsareintuitivefornoviceusersastheylearntheapplicationthrough explorative use. False user input can be easily avoided by letting the user choose