Ascend AI Processor Architecture and Programming: Principles and Applications of CANN

301 Pages·2020·17.424 MB·English

by  Xiaoyao Liang

Ascend AI Processor Architecture
and Programming
Ascend AI Processor Architecture
and Programming
Principles and Applications of CANN
Xiaoyao Liang
Supervisor and Academic Leader, Department of Computer Science and Engineering,
Shanghai Jiaotong University, Shanghai, China
Elsevier
Radarweg29,POBox211,1000AEAmsterdam,Netherlands
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UnitedKingdom
50HampshireStreet,5thFloor,Cambridge,MA02139,UnitedStates
©2020TsinghuaUniversityPressLimited.PublishedbyElsevierInc.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicor
mechanical,includingphotocopying,recording,oranyinformationstorageandretrievalsystem,without
permissioninwritingfromthepublisher.Detailsonhowtoseekpermission,furtherinformationaboutthe
Publisher’spermissionspoliciesandourarrangementswithorganizationssuchastheCopyrightClearance
CenterandtheCopyrightLicensingAgency,canbefoundatourwebsite:www.elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(other
thanasmaybenotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroadenour
understanding,changesinresearchmethods,professionalpractices,ormedicaltreatmentmaybecome
necessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluatingandusing
anyinformation,methods,compounds,orexperimentsdescribedherein.Inusingsuchinformationormethods
theyshouldbemindfuloftheirownsafetyandthesafetyofothers,includingpartiesforwhomtheyhavea
professionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeanyliability
foranyinjuryand/ordamagetopersonsorpropertyasamatterofproductsliability,negligenceorotherwise,or
fromanyuseoroperationofanymethods,products,instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-12-823488-4
ForinformationonallElsevierpublications
visitourwebsiteathttps://www.elsevier.com/books-and-journals
Publisher:GlynJones
EditorialProjectManager:NaomiRobertson
ProjectManager:SelvarajRaviraj
Coverdesigner:ChristianJ.Bilbow
TypesetbySPiGlobal,India
About the Author
Xiaoyao Liang is a professor, PhD supervisor, and academic leader in the Department of
ComputerScienceandEngineering,ShanghaiJiaotongUniversity.HegraduatedfromHarvard
University, United States, and has worked in many companies such as Nvidia and Intel. His
research directions include computer architecture, integrated circuit design, general graphics
processor, and artificial intelligence processor architecture. He has published more than 80
papers, including for international top academic conferences ISCA, HPCA, MICRO, ISSCC,
DAC, ICCAD, etc., among which two papers were selected as the annual best paper of
computer architecture (IEEE MICRO TOP PICKS).
ix
Preface
With the recent success of deep learning in the field of artificial intelligence, from CPUs to
GPUs, to various application-specific processors, we are embracing another golden age of
computer architecture evolution. However, the research and development cycle of processor
chips ranges from a few years to a dozen years. To be successful in this new trend of the
computer industry, the most important aspects include the perseverance and spirit of the
craftsman.
WhenHuaweiinvitedmetowriteareferencebookfortheirAscendAIprocessors,Iaccepted
their request without any hesitation, due to the high recognition of Huawei as a hardware
technologycompanyandthesincerityofitsemployeesinensuringtheyareattheforefrontof
revolutionary technologies.
Huawei’s launch of the AI processor for artificial intelligence computing offers much more
computing power and significantly less power consumption. It brings greater possibilities for
a wide range of deep learning applications. A famousChinese proverb says “the journey of a
thousandmilesbeginswithasinglestep.”TheAscendAIprocessorstillhasalongwaytogo.
For a high-end processor, the development of the ecosystem and the acceptance by program
developers play very important roles, which is critical to the success of this product. The
purpose of writing this book is to primarily demystify the Ascend AI processor to the world
throughexploringitsinnerdesignconcept.Fromboththesoftwareandhardwareperspective,
thistextintroducesgooddevelopmentpracticesfortheAscendAIprocessorplatforms.Rome
wasnotbuiltinoneday.Ifcreatinganecosystemisajourneyofathousandmiles,thisbookis
the first step in this journey.
This book can be used as a selective textbook in the field of artificial intelligence processor
design.Sincethetargetreadersaremainlyengineers,wehavelimitedthecomplexityofabstract
formulas, theorems, and theoretical derivations as much as possible. The reader only needs to
possess basic mathematical knowledge and programming skills, and there are no course
prerequisites.TheidealaudienceforthisbookisR&Dpersonnelwhorequirelarge-scaledeep
learning calculations in the field of artificial intelligence, computer science, electronics,
biomedicine, physics, chemistry, or financial statistics. The book, therefore, provides a
comprehensive and effective reference to companies and developers of AI processors.
xi
Preface
This book is divided into six chapters. The content includes basic theory of neural networks,
computer processor design, open-source neural network architectures, Ascend AI processor
hardware and software architecture, programming theory and methods, as well as use-case
presentations,etc.Byintroducingboththetheoryandpracticalexperiences,wehopetoassist
readers in understanding the DaVinci architecture used by the AI processor and master its
specific programming and usage methods. Through this process, readers will be capable of
creating their own artificial intelligence applications.
Many thanks to Xiaolei Wang, Zishan Jiang, and Xing Li for their great contributions to the
writingofthisbook.Theyhaveinjectedgreatenergyintosortingreferences,textediting,and
testing of all the sample code in this book. Without their full commitment, it would not have
been possible to complete this book successfully.
ThankstoZiyuanChenandseveralothersforcarefullyeditingandmodifyingtheillustrations
in this book as well as making the content clearer. After their feedback, the interpretation of
images and concepts is more specific and clear.
Thanks to Huawei for the resources and support provided during the writing of this book.
Thanks to the support of Mr. Dongliang Sheng and Ms. Zhifang Zhong from Tsinghua
UniversityPressfortheircarefulandmeticulousefforts.Theirinvolvementhasguaranteedthe
quality of this book.
Thereareinevitableomissionsanderrorsinthebook,andreadersarerecommendedtoprovide
feedback, as appropriate.
xii
CHAPTER 1
Theoretical basis
1.1 Brief history of artificial intelligence
Whenthedevelopmentofaskillreachesthepeak,itcanreflecthighlyanthropomorphic
intelligence.FromAncientChina’sMasterYan’sabilitytosinganddance,toAncientArabic
Jazari’sautomaticpuppets(asshowninFig.1.1),therelentlesspursuitofintelligenceneverends.
Humanbeingshopetogivewisdomandthoughttomachines,whichcanbeusedtoliberate
productiveforces,facilitatepeople’slives,andpromotesocialdevelopment.Fromancientmyths
tosciencefiction,andnowtomodernscienceandtechnology,theyalldemonstratethehuman
desireforintelligence.Thebirthofartificialintelligence(AI)wasaslowandlongprocess,
whereastheimprovementofAIiskeepingpacewiththedevelopmentofhumanknowledgeand
eventranscendinghumanwisdominsomeaspects.Intheearlydays,thedevelopmentofFormal
Reasoningprovidedaresearchdirectionforthemechanizationofhumanintelligence.
In the mid-17th century, Gottfried Wilhelm Leibniz, Rene Descartes, and Thomas Hobbes
(Fig.1.2)devotedthemselvestothesystematicstudyofrationalthinking.Thesestudiesledto
theemergenceoftheformalsymbolsystemthatbecamethebeaconofAIresearch.Bythe20th
century, the contributions of Bertrand Arthur William Russell, Alfred North Whitehead, and
KurtGodelinthefieldofmathematicallogicprovidedatheoreticalbasisforthemechanization
ofmathematicalreasoning.ThecreationoftheTuringMachineprovidedsupportingevidence
of machine thought from the view of semiotics. In the engineering domain, from Charles
Babbage’s original idea of “Instrumental Analysis” to the large ENIAC decoding machine
servinginWorldWarII,wehavewitnessedtherealizationofthetheoriesofAlanTuringand
John von Neumann (see Fig. 1.3), which has accelerated the development of AI.
1.1.1 Birth of AI
In the mid-20th century, scientists from different fields made a series of preparations for
thebirthofAI,includingShannon’sinformationtheory,Turing’scomputationaltheory,andthe
development of Neurology.
In 1950, Turing published the paper Computing Machinery and Intelligence. In this paper,
the famous Turing test was proposed: If a machine can answer any questions to it, using
the same words that an ordinary person would, then we may call that machine intelligent.
AscendAIProcessorArchitectureandProgramming.https://doi.org/10.1016/B978-0-12-823488-4.00001-1
#2020TsinghuaUniversityPressLimited.PublishedbyElsevierInc. 1
2 Chapter 1
Fig.1.1
Jazari’s automatic elephant clock. Picture from: https://commons.wikimedia.org/wiki/File:Al-jazari_
elephant_clock.png.
The proposal of the Turing test is of great significance to the development of AI in more
recent times. In 1951, the 24-year-old Marvin Minsky, as well as Dean Edmonds, built the
StochasticNeuralAnalogReinforcementCalculator.Minskycontinuedtoworkinthefieldof
AI,playingahugeroleinpromotingthedevelopmentofAI,whichcontributedtohiswinning
of TuringAward. In 1955, a program called Logic Theorist was introduced and subsequently
refined. The ingenious method proved 38 of 52 theorems in the Principles of Mathematics.
With this work, the authors Allen Newell and Herbert Simon opened a new methodology for
intelligent machines.
Theoretical basis 3
Fig. 1.2
Leibniz,Descartes,Hobbes(fromlefttoright).Leibnizphotosource:https://commons.wikimedia.org/wiki/
File:Gottfried_Wilhelm_Leibniz.jpg; Descartes photo source: https://en.wikipedia.org/wiki/Ren%C3%A9_
Descartes; Hobbes photo source: https://commons.wikimedia.org/wiki/File:Thomas_Hobbes_(portrait).jpg.
Fig. 1.3
TuringandVonNeumann(fromlefttoright).Turingphotosource:https://upload.wikimedia.org/wikipedia/
commons/thumb/a/a1/Alan_Turing_Aged_16.jpg/220px-Alan_Turing_Aged_16.jpg; Von Neumann photo
source: https://commons.wikimedia.org/wiki/File:JohnvonNeumann-LosAlamos.jpg.
Ayearlater,10participantsattheDartmouthConference,includingMcCarthy,Shannon,and
Nathan Rochester, argued that “any aspect of learning or intelligence should be accurately
described so that people can build a machine to simulate it.” In that moment, AI entered the
world with its mission clearly defined, opening up a brand new world of science.