GPU parallel program development using CUDA

477 Pages·2018·4.096 MB·English

by  Tolga Soyata

GPU Parallel Program
Development Using CUDA
Chapman & Hall/CRC
Computational Science Series
SERIES EDITOR
Horst Simon
Deputy Director
Lawrence Berkeley National Laboratory
Berkeley, California, U.S.A.
PUBLISHED TITLES
COMBINATORIAL SCIENTIFIC COMPUTING
Edited by Uwe Naumann and Olaf Schenk
CONTEMPORARY HIGH PERFORMANCE COMPUTING: FROM PETASCALE
TOWARD EXASCALE
Edited by Jeffrey S. Vetter
CONTEMPORARY HIGH PERFORMANCE COMPUTING: FROM PETASCALE
TOWARD EXASCALE, VOLUME TWO
Edited by Jeffrey S. Vetter
DATA-INTENSIVE SCIENCE
Edited by Terence Critchlow and Kerstin Kleese van Dam
ELEMENTS OF PARALLEL COMPUTING
Eric Aubanel
THE END OF ERROR: UNUM COMPUTING
John L. Gustafson
EXASCALE SCIENTIFIC APPLICATIONS: SCALABILITY AND
PERFORMANCE PORTABILITY
Edited by Tjerk P. Straatsma, Katerina B. Antypas, and Timothy J. Williams
FROM ACTION SYSTEMS TO DISTRIBUTED SYSTEMS: THE REFINEMENT APPROACH
Edited by Luigia Petre and Emil Sekerinski
FUNDAMENTALS OF MULTICORE SOFTWARE DEVELOPMENT
Edited by Victor Pankratius, Ali-Reza Adl-Tabatabai, and Walter Tichy
FUNDAMENTALS OF PARALLEL MULTICORE ARCHITECTURE
Yan Solihin
THE GREEN COMPUTING BOOK: TACKLING ENERGY EFFICIENCY AT LARGE SCALE
Edited by Wu-chun Feng
GRID COMPUTING: TECHNIQUES AND APPLICATIONS
Barry Wilkinson
GPU PARALLEL PROGRAM DEVELOPMENT USING CUDA
Tolga Soyata
PUBLISHED TITLES CONTINUED
HIGH PERFORMANCE COMPUTING: PROGRAMMING AND APPLICATIONS
John Levesque with Gene Wagenbreth
HIGH PERFORMANCE PARALLEL I/O
Prabhat and Quincey Koziol
HIGH PERFORMANCE VISUALIZATION:
ENABLING EXTREME-SCALE SCIENTIFIC INSIGHT
Edited by E. Wes Bethel, Hank Childs, and Charles Hansen
INDUSTRIAL APPLICATIONS OF HIGH-PERFORMANCE COMPUTING:
BEST GLOBAL PRACTICES
Edited by Anwar Osseyran and Merle Giles
INTRODUCTION TO COMPUTATIONAL MODELING USING C AND
OPEN-SOURCE TOOLS
José M Garrido
INTRODUCTION TO CONCURRENCY IN PROGRAMMING LANGUAGES
Matthew J. Sottile, Timothy G. Mattson, and Craig E Rasmussen
INTRODUCTION TO ELEMENTARY COMPUTATIONAL MODELING: ESSENTIAL
CONCEPTS, PRINCIPLES, AND PROBLEM SOLVING
José M. Garrido
INTRODUCTION TO HIGH PERFORMANCE COMPUTING FOR SCIENTISTS
AND ENGINEERS
Georg Hager and Gerhard Wellein
INTRODUCTION TO MODELING AND SIMULATION WITH MATLAB® AND PYTHON
Steven I. Gordon and Brian Guilfoos
INTRODUCTION TO REVERSIBLE COMPUTING
Kalyan S. Perumalla
INTRODUCTION TO SCHEDULING
Yves Robert and Frédéric Vivien
INTRODUCTION TO THE SIMULATION OF DYNAMICS USING SIMULINK®
Michael A. Gray
PEER-TO-PEER COMPUTING: APPLICATIONS, ARCHITECTURE, PROTOCOLS,
AND CHALLENGES
Yu-Kwong Ricky Kwok
PERFORMANCE TUNING OF SCIENTIFIC APPLICATIONS
Edited by David Bailey, Robert Lucas, and Samuel Williams
PETASCALE COMPUTING: ALGORITHMS AND APPLICATIONS
Edited by David A. Bader
PROCESS ALGEBRA FOR PARALLEL AND DISTRIBUTED PROCESSING
Edited by Michael Alexander and William Gardner
PUBLISHED TITLES CONTINUED
PROGRAMMING FOR HYBRID MULTI/MANY-CORE MPP SYSTEMS
John Levesque and Aaron Vose
SCIENTIFIC DATA MANAGEMENT: CHALLENGES, TECHNOLOGY, AND DEPLOYMENT
Edited by Arie Shoshani and Doron Rotem
SOFTWARE ENGINEERING FOR SCIENCE
Edited by Jeffrey C. Carver, Neil P. Chue Hong, and George K. Thiruvathukal
GPU Parallel Program
Development Using CUDA
Tolga Soyata
CRCPress
Taylor&FrancisGroup
6000BrokenSoundParkwayNW,Suite300
BocaRaton,FL33487-2742
©2018byTaylor&FrancisGroup,LLC
CRCPressisanimprintofTaylor&FrancisGroup,anInformabusiness
NoclaimtooriginalU.S.Governmentworks
Printedonacid-freepaper
InternationalStandardBookNumber-13:978-1-4987-5075-2(Hardback)
This book contains information obtained from authentic and highly regarded sources. Reasonable efforts
have been made to publish reliable data and information, but the author and publisher cannot assume
responsibilityforthevalidityofallmaterialsortheconsequencesoftheiruse.Theauthorsandpublishers
haveattemptedtotracethecopyrightholdersofallmaterialreproducedinthispublicationandapologize
tocopyrightholdersifpermissiontopublishinthisformhasnotbeenobtained.Ifanycopyrightmaterial
hasnotbeenacknowledgedpleasewriteandletusknowsowemayrectifyinanyfuturereprint.
ExceptaspermittedunderU.S.CopyrightLaw,nopartofthisbookmaybereprinted,reproduced,trans-
mitted, or utilized in any form by any electronic, mechanical, or other means, now known or hereafter
invented, including photocopying, microfilming, and recording, or in any information storage or retrieval
system,withoutwrittenpermissionfromthepublishers.
Forpermissiontophotocopyorusematerialelectronicallyfromthiswork,pleaseaccesswww.copyright.com
(http://www.copyright.com/)orcontacttheCopyrightClearanceCenter,Inc.(CCC),222RosewoodDrive,
Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and regis-
tration for a variety of users. For organizations that have been granted a photocopy license by the CCC,
aseparatesystemofpaymenthasbeenarranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are
usedonlyforidentificationandexplanationwithoutintenttoinfringe.
——————————————————————————————————————————————–
Library of Congress Cataloging-in-Publication Data
——————————————————————————————————————————————–
Names:Soyata,Tolga,1967-author.
Title:GPUparallelprogramdevelopmentusingCUDA
/byTolgaSoyata.
Description:BocaRaton,Florida:CRCPress,[2018]|Includesbibliographical
referencesandindex.
Identifiers:LCCN2017043292|ISBN9781498750752(hardback)|
ISBN9781315368290(e-book)
Subjects:LCSH:Parallelprogramming(Computerscience)|CUDA(Computerarchitecture)|
Graphicsprocessingunits–Programming.
Classification:LCCQA76.642.S672018|DDC005.2/75–dc23
LCrecordavailableathttps://lccn.loc.gov/2017043292
Visit the Taylor & Francis Web site at
http://www.taylorandfrancis.com
and the CRC Press Web site at
http://www.crcpress.com
To my wife Eileen
and my step-children Katherine, Andrew, and Eric.

Contents
List of Figures xxiii
List of Tables xxix
Preface xxxiii
About the Author xxxv
art
P I Understanding CPU Parallelism
hapter
C 1(cid:4) Introduction to CPU Parallel Programming 3
1.1 EVOLUTIONOFPARALLELPROGRAMMING 3
1.2 MORECORES,MOREPARALLELISM 4
1.3 CORESVERSUSTHREADS 5
1.3.1 More Threads or More Cores to Parallelize? 5
1.3.2 Influence of Core Resource Sharing 7
1.3.3 Influence of Memory Resource Sharing 7
1.4 OURFIRSTSERIALPROGRAM 8
1.4.1 Understanding Data Transfer Speeds 8
1.4.2 The main() Function in imflip.c 10
1.4.3 Flipping Rows Vertically: FlipImageV() 11
1.4.4 Flipping Columns Horizontally: FlipImageH() 12
1.5 WRITING,COMPILING,RUNNINGOURPROGRAMS 13
1.5.1 Choosing an Editor and a Compiler 13
1.5.2 Developing in Windows 7, 8, and Windows 10
Platforms 13
1.5.3 Developing in a Mac Platform 15
1.5.4 Developing in a Unix Platform 15
1.6 CRASHCOURSEONUNIX 15
1.6.1 Unix Directory-Related Commands 15
1.6.2 Unix File-Related Commands 16
1.7 DEBUGGINGYOURPROGRAMS 19
1.7.1 gdb 20
1.7.2 Old School Debugging 21
1.7.3 valgrind 22
ix