Artificial Intelligence Technology

308 Pages·2022·9.949 MB·English

by  Huawei Technologies Co., Ltd.

Official Textbooks for Huawei ICT Academy
ARTIFICIAL
INTELLIGENCE
TECHNOLOGY
Huawei Technologies Co., Ltd.
fi
Arti cial Intelligence Technology
Huawei Technologies Co., Ltd.
fi
Arti cial Intelligence
Technology
HuaweiTechnologiesCo.,Ltd.
Hangzhou,China
ThisworkwassupportedbyHuaweiTechnologiesCo.,Ltd.
ISBN978-981-19-2878-9 ISBN978-981-19-2879-6 (eBook)
https://doi.org/10.1007/978-981-19-2879-6
©Posts&TelecomPress2023.Thisbookisanopenaccesspublication.
JointlypublishedwithPosts&TelecomPress,Beijing,China
TheprinteditionisnotforsaleinChina(Mainland).CustomersfromChina(Mainland)pleaseorderthe
printbookfrom:Posts&TelecomPress.
Open Access This book is licensed under the terms of the Creative Commons Attribution-
NonCommercial-NoDerivatives 4.0 International License (http://creativecommons.org/licenses/by-nc-
nd/4.0/),whichpermitsanynoncommercialuse,sharing,distributionandreproductioninanymediumor
format,aslongasyougiveappropriatecredittotheoriginalauthor(s)andthesource,providealinktothe
CreativeCommonslicenseandindicateifyoumodifiedthelicensedmaterial.Youdonothavepermission
underthislicensetoshareadaptedmaterialderivedfromthisbookorpartsofit.
Theimagesorotherthirdpartymaterialinthisbookareincludedinthebook’sCreativeCommonslicense,
unlessindicatedotherwiseinacreditlinetothematerial.Ifmaterialisnotincludedinthebook’sCreative
Commonslicenseandyourintendeduseisnotpermittedbystatutoryregulationorexceedsthepermitted
use,youwillneedtoobtainpermissiondirectlyfromthecopyrightholder.
Thisworkissubjecttocopyright.Allcommercialrightsarereservedbytheauthor(s),whetherthewhole
orpartofthematerialisconcerned,specificallytherightsofreprinting,reuseofillustrations,recitation,
broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation
storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology
nowknownorhereafterdeveloped.Regardingthesecommercialrightsanon-exclusivelicensehasbeen
grantedtothepublisher.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublishers,theauthors,andtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishersnortheauthorsorthe
editorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublishersremainneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSingaporePteLtd.
The registered company address is: 152 Beach Road, #21-01/04 Gateway East, Singapore 189721,
Singapore
Preface
The rapid development of information technology in China since the twenty-first
century has completely changed the way people study, work, and live. As a key
research topic in information science, artificial intelligence, also known as AI, has
gonethroughtumultuouslygoodandhardtimeseversincetheconceptwascoined
halfacenturyago.ItisinspiringtoseethatAIhasachievedatremendousprogress
andhasbeenappliedinawiderangeoffieldsoverthepastseveralyears,asaresult
of the advancements in computing power and big data technology. Today, AI is
developing at an exponential rate, and its fruits will be continuously consolidated
anddeepened.
Despiteitsdrawbacks,thefutureofAIisbelievedtobepromising.AsChinahas
put AI on its national agenda, the interests of almost all the industries have been
triggered,andtherelevantpractitionersareeagertoknowthestatusquoandresearch
hotspotsofAI,aswellasitsbasicprinciplesandresearchmethodologies.
AlthoughabulkofAIthemedbookshavebeenpublishedbothinChinaandthe
world, providing insightful and precise analysis covering almost all the topics of
almosttheentireAIsub-disciplines,wehavetoadmitthattheycouldbeesotericfor
beginners.ArtificialIntelligenceisatypicalinterdisciplinaryscienceconcernedwith
wide-ranging aspects and a subject unlikely to be exhausted by only one or two
monographs. Therefore, this book aims at presenting our readers fundamental AI
knowledge through a streamlined structure and thought-provoking cases. As a
Chinesesayinggoes,“theteachercanteachyoutheskills,butitdependsonyourself
tomasterthem.” Weexpect our readers could find thetopicsthey are interested in
afterreadingthisbookandpressaheadwithmorein-depthstudies.
Sinceintelligenceisaverycomplicatedsubject,differentpeoplecouldgenerate
distinctiveunderstandingandsimulationofintelligencefromdifferentperspectives
andviewpoints.Therefore,thisbookexploresAIfromdiversifiedperspectivesinthe
eight chapters. Chapter 1 is the general introduction, which introduces the origin,
technology, fields of application, and trend of the development of AI, as well as
Huawei’s AI development strategy. Chapter 2 focuses on the machine learning,
illustrating the types, overall process, and the popular algorithms of machine
v
vi Preface
learning, such as the decision trees, support vector machine, and clustering algo-
rithmscommonlyinusetoday.Chapter3givesanoverviewtothedeeplearningby
reviewingitsevolvement,andelaboratingonitstrainingrules,activationfunctions,
regularization, and optimizers on the basis of neural networks, a widely adopted
approach in deep learning. Chapter 4 is about the deep learning frameworks,
exemplified by TensorFlow 2.0 as one of the three mainstream frameworks.
Chapter 5 brings up the topic of MindSpore, centering on the AI development
frameworkofHuaweiandMindSpore’sdevelopmentandapplication.Chapter6is
aboutHuaweiAtlasAIcomputingsolution,withthediscussionrevolvingaroundthe
software and hardware architecture of Ascend AI processor, Atlas AI computing
platform, and its industrial applications. Chapter 7 introduces Huawei’s smart
terminal AI open platform known as the HUAWEI HiAI platform and the apps
developed on it. Chapter 8 looks at the enterprise intelligence application platform
HuaweiCLOUDbytakingHuaweiCLOUDEIandModelArtsasmajorexamples.
Intermsofthenarrativestyle,eachchapterhasthetheoriesandmethodsthoroughly
expounded, characterized by relatively independent and integral contents. Mean-
while,thechaptersarepresentedinaprogressiveordertoofferasystematicreading
experience.Ourreaderscaneitherreadthebookchapterbychapterorjustjumpto
certain chapters to have a detailed reading. The chapters are written in concise
languages while dealing with profound theories, with part of the derivations of
formulas and theorems provided and palpable examples quoted to help beginners
to master the basic knowledge, grasp the essence, and put it into practice flexibly.
Forthoseabstrusetheories,thisbookonlygivesabriefoverviewwithoutanyfurther
discussions.
The book is edited by Huawei Technologies Co. Ltd, with LvYunxiang, Wang
Luting, Gong Xiaogang, and Chen Miaoran working as the specific editors. Zeng
Honglialsocontributedtothecompilationofseveralchaptersandpreparedmaterials
andrelevantsupportingresourcesforwriting.
Although we worked as careful and prudent as possible during the editing, the
inadequacy is inevitable due to our limitations. Therefore, any comments and
corrections from our readers are most welcome. In the meantime, we also invite
you to share your feedbacks regarding reading this book with us
([email protected]).
Hangzhou,China HuaweiTechnologiesCo.,Ltd.
December2021
Contents
1 AGeneralIntroductiontoArtificialIntelligence. . . . . . . . . . . . . . . . 1
1.1 TheConceptofArtificialIntelligence. . . . . . . . . . . . . . . . . . . . . . 1
1.1.1 WhatIsArtificialIntelligence?. . . . . . . . . . . . . . . . . . . . 1
1.1.2 TheRelationshipBetweenAI,MachineLearning,
andDeepLearning. . . . .. . . . . . . . . . .. . . . . . . . . . .. . 4
1.1.3 TypesofAI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.4 TheHistoryofAI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
1.1.5 TheThreeMainSchoolsofAI. . . . . . . . . . . . . . . . . . . . 12
1.2 AI-RelatedTechnologies. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1.2.1 DeepLearningFramework. . . . . . . . . . . . . . . . . . . . . . . 15
1.2.2 AnOverviewofAIProcessor. . . . . . . . . . . . . . . . . . . . . 15
1.2.3 EcosystemofAIIndustry. . . . . . . . . . . . . . . . . . . . . . . . 20
1.2.4 HuaweiCLOUDEnterpriseIntelligenceApplication
Platform. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
1.3 TheTechnologiesandApplicationsofAI. . . . . . . . . . . . . . . . . . . 25
1.3.1 TheTechnologiesofAI. . . . . . . . . . . . . . . . . . . . . . . . . 25
1.3.2 TheApplicationsofAI. . . . . . . . . . . . . . . . . . . . .. . . . . 29
1.3.3 TheCurrentStatusofAI. . . . . . . . . . . . . . . . . . . . . . . . 32
1.4 Huawei’sAIDevelopmentStrategy. . . . . . . . . . . . . . . . . . . . . . . 32
1.4.1 Full-StackAll-ScenarioAISolutions. . . . . . . . . . . . . . . . 32
1.4.2 DirectionsofHuaweiFull-StackAI. . . . . . . . . . . . . . . . 33
1.5 TheControversyofAI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
1.5.1 AlgorithmicBias. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
1.5.2 PrivacyIssues. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
1.5.3 TheContradictionBetweenTechnologyandEthics. . . . . 37
1.5.4 WillEveryoneBeUnemployed?. . . . . . . . . . . . . . . . . . . 38
1.6 TheDevelopmentTrendsforAI. . . . . . . . . . . . . . . . . . . . . . . . . 38
1.7 ChapterSummary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
1.8 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
vii
viii Contents
2 MachineLearning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.1 IntroductiontoMachineLearning. . . . . . . . . . . . . . . . . . . . . . . . 43
2.1.1 RationalUnderstandingofMachineLearning
Algorithms. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
2.1.2 MajorProblemsSolvedbyMachineLearning. . . . . . . . . 46
2.2 TypesofMachineLearning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.2.1 SupervisedLearning. . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.2.2 UnsupervisedLearning. . . . . . . .. . . . . . . . . . . . . . .. . . 48
2.2.3 Semi-supervisedLearning. . . . . . . . . . . . . . . . . . . . . . . 50
2.2.4 ReinforcementLearning. . . . . . . . . . . . . . . . . . . . . . . . . 51
2.3 TheOverallProcessofMachineLearning. . . . . . . . . . . . . . . . . . 52
2.3.1 DataCollection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.3.2 DataCleaning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
2.3.3 FeatureSelection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
2.3.4 TheConstructionofMachineLearningModels. . . . . . . . 56
2.3.5 ModelEvaluation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
2.4 ModelParametersandHyperparameters. . . . . . . . . . . . . . . . . . . . 63
2.4.1 GradientDescent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
2.4.2 ValidationSetandHyperparameterSearch. . . . . . . . . . . 65
2.4.3 Cross-validation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.5 CommonAlgorithmsofMachineLearning. . . . . . . . . . . . . . . . . . 67
2.5.1 LinearRegression. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
2.5.2 LogisticRegression. . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.5.3 DecisionTree. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.5.4 SupportVectorMachine. . . . . . .. . . . . . . . . . . . . . . .. . 75
2.5.5 K-NearestNeighborAlgorithm. . . . . . . . . . . . . . . . . . . . 76
2.5.6 NaiveBayes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78
2.5.7 EnsembleLearning. . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
2.5.8 ClusteringAlgorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . 81
2.6 CaseStudy. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82
2.7 ChapterSummary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
2.8 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86
3 OverviewofDeepLearning. . . .. . .. . . .. . . .. . . .. . . .. . . .. . . .. 87
3.1 IntroductiontoDeepLearning. . . . . . . . . . . . . . . . . . . . . . . . . . . 87
3.1.1 DeepNeuralNetwork. . . . . . . . . . . . . . . . . . . . . . . . . . 88
3.1.2 TheDevelopmentofDeepLearning. . . . . . . . . . . . . . . . 89
3.1.3 PerceptronAlgorithm. . . . . . . . . . . . . . . . . . . . . . . . . . . 90
3.2 TrainingRules. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.2.1 LossFunction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
3.2.2 GradientDescent. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94
3.2.3 BackpropagationAlgorithm. . . . . . . . . . . . . . . . . . . . . . 97
3.3 ActivationFunction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
3.4 Regularization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
3.4.1 ParameterNormPenalty. . . . . . . . . . . . . . . . . . . . . . . . . 102
Contents ix
3.4.2 DatasetExpansion. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103
3.4.3 Dropout. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.4.4 EarlyStopping. .. . . . . . . . . . . . . .. . . . . . . . . . . . . .. . 105
3.5 Optimizer. . . . . . . . . . . . . . . .. . . . . . . . . . . . . . . . . . . . . .. . . . 106
3.5.1 Momentum. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.5.2 AdagradOptimizer. . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.5.3 RMSpropOptimizer. .. . . . . . . . . . . . . . . . . . . . . .. . . . 108
3.5.4 AdamOptimizer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108
3.6 TypesofNeuralNetwork. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.6.1 ConvolutionalNeuralNetwork. . . . . . . . . . . . . . . . . . . . 109
3.6.2 RecurrentNeuralNetwork. . . . . . . . . . . . . . . . . . . . . . . 114
3.6.3 GenerativeAdversarialNetwork. . . . . . . . . . . . . . . . . . . 117
3.7 CommonProblems. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.7.1 ImbalancedData. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
3.7.2 VanishingGradientandExplodingGradient. . . . . . . . . . 120
3.7.3 Overfitting. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
3.8 ChapterSummary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
3.9 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4 DeepLearningFrameworks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
4.1 IntroductiontoDeepLearningFrameworks. . . . . . . . . . . . . . . . . 123
4.1.1 IntroductiontoPyTorch. . . . . . . . . . . . . . . . . . . . . . . . . 123
4.1.2 IntroductiontoMindSpore. . . . . . . . . . . . . . . . . . . . . . . 124
4.1.3 IntroductiontoTensorFlow. . . . . . . . . . . . . . . . . . . . . . 126
4.2 TensorFlow2.0Basics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.2.1 IntroductiontoTensorFlow2.0. . . . . . . . . . . . . . . . . . . . 127
4.2.2 IntroductiontoTensors. . . . . . . . . . . . . . . . . . . . . . . . . 128
4.2.3 TensorFlow2.0EagerExecution. . . . . . . . . . . . . . . . . . 128
4.2.4 TensorFlow2.0AutoGraph. . . . . . . . . . . . . . . . . . . . . . 128
4.3 IntroductiontoTensorFlow2.0Module. . . . . . . . . . . . . . . . . . . . 129
4.3.1 IntroductiontoCommonModules. . . . . . . . . . . . . . . . . . 129
4.3.2 KerasInterface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
4.4 GetStartedwithTensorFlow2.0. . . . . . . . . . . . . . . . . . . . . . . . . 130
4.4.1 EnvironmentSetup. . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
4.4.2 DevelopmentProcess. . . . . . . . . . . . . . . . . . . . . . . . . . . 131
4.5 ChapterSummary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
4.6 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
5 HuaweiMindSporeAIDevelopmentFramework. . . . . . . . . . . . . . . 137
5.1 IntroductiontoMindSporeDevelopmentFramework. . . . . . . . . . 137
5.1.1 MindSporeArchitecture. . . . . . . . . . . . . . . . . . . . . . . . . 137
5.1.2 HowIsMindSporeDesigned. . . . . . . . . . . . . . . . . . . . . 138
5.1.3 AdvantagesofMindSpore. . . . . . . . . . . . . . . . . . . . . . . 146
5.2 MindSporeDevelopmentandApplication. . . . . . . . . . . . . . . . . . 147
5.2.1 EnvironmentSetup. . . . . . . . . . . . . . . . . . . . . . . . . . . . 147
x Contents
5.2.2 ComponentsandConceptsRelatedtoMindSpore. . . . . . 150
5.2.3 RealizationofHandwrittenDigitRecognitionwith
Mindspore. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152
5.3 ChapterSummary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
5.4 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
6 HuaweiAtlasAIComputingSolution. . . . . . . . . . . . . . . . . . . . . . . . 163
6.1 TheHardwareArchitectureofAscendAIProcessor. . . . . . . . . . . 163
6.1.1 TheLogicArchitectureofAscendAIProcessor
Hardware. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.1.2 DaVinciArchitecture. . . . . . . . . . . . . . . . . . . . . . . . . . 163
6.2 TheSoftwareArchitectureofAscendAIProcessor. . . . . . . . . . . . 168
6.2.1 TheLogicArchitectureofAscendAIProcessor
Software. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
6.2.2 TheNeuralNetworkSoftwareFlowofAscendAI
Processor. . . .. . . .. . . .. . . .. . .. . . .. . . .. . . .. . . .. 171
6.2.3 IntroductiontotheFunctionalModulesofAscendAI
ProcessorSoftwareFlow. . . . . . . . . . . . . . . . . . . . . . . . 173
6.2.4 TheDataFlowofAscendAIProcessor. . . . . . . . . . . . . . 195
6.3 AtlasAIComputingSolution. . . . . . . . . . . . . . . . . . . . . . . . . . . 196
6.3.1 AtlasforAITrainingAcceleration. . . . . . . . . . . . . . . . . 210
6.3.2 AtlasDevice-Edge-CloudCollaboration. . . . . . . . . . . . . 213
6.4 IndustrialImplementationofAtlas. . . . . . . . . . . . . . . . . . . . . . . . 214
6.4.1 Electricity:One-StopICTSmartGridSolution. . . . . . . . 214
6.4.2 IntelligentFinance:HolisticDigitalTransformation. . . . . 215
6.4.3 IntelligentManufacturing:DigitalIntegrationof
MachinesandThoughts. . . . . . . . . . . . . . . . . . . . . . . . . 215
6.4.4 IntelligentTransportation:EasierTravelandImproved
Logistics. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
6.4.5 SuperComputer:BuildingState-LevelAIPlatform. . . . . 217
6.5 ChapterSummary. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
6.6 Exercises. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
7 HUAWEIAIOpenPlatform. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
7.1 IntroductiontoHUAWEIHiAIPlatform. . . . . . . . . . . . . . . . . . . 221
7.1.1 ArchitectureofHUAWEIHiAIPlatform. . . . . . . . . . . . . 222
7.1.2 HUAWEIHiAIFoundation. . . . . . . . . . . . . . . . . . . . . . 224
7.1.3 HUAWEIHiAIEngine. . . . . . . . . . . . . . . . . . . . . . . . . 226
7.1.4 HUAWEIHiAIService. . . . . . . . . . . . . . . . . . . . . . . . . 227
7.2 ApplicationDevelopmentBasedonHUAWEIHiAIPlatform. . . . 227
7.3 PartoftheHUAWEIHiAISolutions. . . . . . . . . . . . . . . . . . . . . . 231
7.3.1 HUAWEIHiAIHelpingtheDeafandDumb. . . . . . . . . . 231
7.3.2 HUAWEIHiAIEnhancingtheVisualEffects
ofYuanbeiDrivingTestApplication. . . . . . . . . . . . . . . . 232
7.3.3 HUAWEIHiAIEmpoweringCtripTravel. . . . . . . . . . . . 233