Computing Systems for Autonomous Driving

239 Pages·2021·7.379 MB·English

by  Weisong Shi, Liangkai Liu

Weisong Shi
Liangkai Liu
Computing
Systems
for Autonomous
Driving
Computing Systems for Autonomous Driving
Weisong Shi • Liangkai Liu
Computing Systems for
Autonomous Driving
WeisongShi LiangkaiLiu
DepartmentofComputerScience DepartmentofComputerScience
WayneStateUniversity WayneStateUniversity
Detroit,MI,USA Detroit,MI,USA
ISBN978-3-030-81563-9 ISBN978-3-030-81564-6 (eBook)
https://doi.org/10.1007/978-3-030-81564-6
©TheEditor(s)(ifapplicable)andTheAuthor(s),underexclusivelicensetoSpringerNatureSwitzerland
AG2021
Thisworkissubjecttocopyright.AllrightsaresolelyandexclusivelylicensedbythePublisher,whether
thewholeorpartofthematerialisconcerned,specificallytherightsoftranslation,reprinting,reuse
ofillustrations,recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,and
transmissionorinformationstorageandretrieval,electronicadaptation,computersoftware,orbysimilar
ordissimilarmethodologynowknownorhereafterdeveloped.
Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etc.inthispublication
doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant
protectivelawsandregulationsandthereforefreeforgeneraluse.
Thepublisher,theauthorsandtheeditorsaresafetoassumethattheadviceandinformationinthisbook
arebelievedtobetrueandaccurateatthedateofpublication.Neitherthepublishernortheauthorsor
theeditorsgiveawarranty,expressedorimplied,withrespecttothematerialcontainedhereinorforany
errorsoromissionsthatmayhavebeenmade.Thepublisherremainsneutralwithregardtojurisdictional
claimsinpublishedmapsandinstitutionalaffiliations.
ThisSpringerimprintispublishedbytheregisteredcompanySpringerNatureSwitzerlandAG
Theregisteredcompanyaddressis:Gewerbestrasse11,6330Cham,Switzerland
Preface
In the last 5 years, with the vast improvements in computing technologies, e.g.,
sensors, computer vision, machine learning, and hardware acceleration, and the
widedeploymentofcommunicationmechanisms,e.g.,dedicatedshort-rangecom-
munications(DSRC),cellularvehicle-to-everything(C-V2X),and5G,autonomous
driving techniques have attracted massive attention from both the academic and
automotivecommunities.
Toachievethevisionofautonomousdriving,determininghowtomakethevehi-
cleunderstandtheenvironmentcorrectlyandmakesafecontrolsinreal-timeisthe
essentialtask.Richsensorsincludingcamera,LiDAR(lightdetectionandranging),
radar,inertialmeasurementunit(IMU),globalnavigationsatellitesystem(GNSS),
andsonar,aswellaspowerfulcomputationdevices,areinstalledonthevehicle.This
designmakesautonomousdrivingarealpowerful“computeronwheels.”Inaddition
to hardware, the rapid development of deep learning algorithms in object/lane
detection, simultaneous localization and mapping (SLAM), and vehicle control
also promotes the real deployment and prototyping of autonomous vehicles. The
autonomousvehicle’scomputingsystemsaredefinedtocovereverything(excluding
the vehicle’s mechanical parts), including sensors, computation, communication,
storage, power management, and full-stack software. Plenty of algorithms and
systems are designed to process sensor data and make a reliable decision in real-
time.
However, news of fatalities caused by early developed autonomous vehicles
(AVs) arises from time to time. Until August 2020, five self-driving car fatalities
happened for level 2 autonomous driving: four of them from Tesla and one from
Uber. All four incidents associated with Tesla are due to perception failure, while
Uber’sincidenthappenedbecauseofthefailuretopredicthumanbehavior.Another
fact to pay attention to is that currently, the field-testing of level 2 autonomous
driving vehicles mostly happens in places with good weather and light traffic
conditionslikeArizonaandFlorida.Therealtrafficenvironmentistoocomplicated
for the current autonomous driving systems to understand and handle easily. The
objectivesoflevel4andlevel5autonomousdrivingrequirecolossalimprovement
ofthecomputingsystemsforautonomousvehicles.
v
vi Preface
Thisbookintendstopresentstate-of-the-artcomputingsystemsforautonomous
drivingandtograbtheattentionofresearchersandpractitionersfrombothautomo-
tiveindustryandcomputerscienceandengineeringcommunity.Thebookconsists
of nine chapters, presenting the landscape, computing frameworks, algorithm
deploymentoptimizations,systemsruntimeoptimizations,datasetandbenchmark-
ing,simulators,hardwareplatforms,smartinfrastructures,andopenchallengesfor
achievingL4/L5autonomousdrivingvehicles,respectively.Thisbookcanbeused
byseniorundergraduatestudentsandgraduatestudentsinengineeringandcomputer
sciencemajors.Wehopethisbookwillserveasareferenceandastartingpointfor
thosewhoareinterestedinworkinginthisfield.
Detroit,MI,USA WeisongShi
Detroit,MI,USA LiangkaiLiu
Acknowledgments
ThisbookisacollectivewisdomoftheworkfromtheConnectedandAutonomous
Driving Laboratory (CAR) at Wayne State University. We would like to thank
all the past and current members in the CAR lab, including Sidi Lu, Qingyang
Zhang, Yifan Wang, Xingzhou Zhang, Baofu Wu, Prabhjot Kaur, Samira Taghavi,
Ren Zhong, Yongtao Yao, Ruijun Wang, Zhaofeng Tian, and Raef Abdallah. All
of them contributed part of the content that is included in this book. We also
thank our partners and sponsors who contributed hardware, software, and dataset
andmadethesestudiespossible,includingCalmCar,Continental,DENSO,Hesai,
iSmartWays, Intel, Navya, Nvidia, PerceptIn, Toyota InfoTech, Velodyne LiDAR,
Xilinx,andtheCityofDetroit.
vii
Contents
1 AutonomousDrivingLandscape ........................................... 1
1.1 ReferenceArchitecture................................................ 1
1.2 MetricsforComputingSystem ....................................... 3
1.3 KeyTechnologies...................................................... 4
1.3.1 Sensors......................................................... 5
1.3.2 DataSource.................................................... 6
1.3.3 AutonomousDrivingApplications........................... 8
1.3.4 ComputationHardware ....................................... 12
1.3.5 Storage......................................................... 13
1.3.6 Real-TimeOperatingSystems................................ 14
1.3.7 MiddlewareSystems .......................................... 15
1.3.8 VehicularCommunication .................................... 15
1.3.9 SecurityandPrivacy........................................... 16
1.4 OverviewoftheBook................................................. 17
2 ComputingFrameworkforAutonomousDriving........................ 19
2.1 In-VehicleApplications ............................................... 19
2.1.1 Real-TimeDiagnostics........................................ 19
2.1.2 AdvancedDriver-AssistantSystems.......................... 20
2.1.3 In-VehicleInfotainment....................................... 20
2.1.4 Third-PartyApplications...................................... 21
2.2 OpenVDAP:AnOpenVehicularDataAnalyticsPlatform
forCAVs ............................................................... 22
2.2.1 Introduction.................................................... 22
2.2.2 OverviewofOpenVDAP...................................... 24
2.2.3 VCU:HeterogeneousVehicleComputingUnit.............. 25
2.2.4 EdgeOSv:AnEdgeOperatingSystemforVehicles......... 28
2.2.5 DDI:DrivingDataIntegrator................................. 30
2.2.6 libvdap:LibraryforOpenVehicularDataAnalytics..... 32
ix
x Contents
2.3 HydraSpace:ComputationalDataStorageforAutonomous
Vehicles ................................................................ 34
2.3.1 Introduction.................................................... 34
2.3.2 Real-timeApplicationRequirement.......................... 35
2.3.3 AccessFrequency ............................................. 36
2.3.4 DataAmountandType........................................ 37
2.3.5 SystemDesign................................................. 38
2.3.6 Summary....................................................... 39
2.4 AC4AV: A Flexible and Dynamic Access Control
FrameworkforConnectedandAutonomousVehicles............... 39
2.4.1 Introduction.................................................... 39
2.4.2 ProblemStatement ............................................ 42
2.4.3 SystemArchitecture........................................... 45
2.4.4 Implementation................................................ 48
2.4.5 Summary....................................................... 54
3 AlgorithmDeploymentOptimization...................................... 57
3.1 CLONE:CollaborativeLearningontheEdges ...................... 57
3.1.1 Introduction.................................................... 57
3.1.2 SystemDesign................................................. 59
3.1.3 DifferencesofTwoApplicationScenarios................... 59
3.1.4 UseCase:FailurePredictionofEVBatteryand
RelatedAccessories........................................... 60
3.1.5 Summary....................................................... 70
3.2 CollaborativeCloud-EdgeComputationforPersonalized
DrivingBehaviorModeling........................................... 70
3.2.1 Introduction.................................................... 70
3.2.2 SystemDesign................................................. 74
3.2.3 GARNN-Cloud................................................ 75
3.2.4 GARNN-Pruned............................................... 76
3.2.5 CGARNN-Edge............................................... 77
3.2.6 RESTfulEngine............................................... 78
3.2.7 Summary....................................................... 79
4 SystemsRuntimeOptimization ............................................ 81
4.1 E2M:AnEnergy-EfficientMiddlewareforAutonomous
MobileRobots ......................................................... 81
4.1.1 Introduction.................................................... 81
4.1.2 Motivation ..................................................... 83
4.1.3 EnergyEfficientMiddleware ................................. 89
4.1.4 Implementation................................................ 97
4.1.5 Discussion ..................................................... 100
4.1.6 Summary....................................................... 101
4.2 DeterminismAnalysisofDeepNeuralNetworkInference
forAutonomousDriving .............................................. 102
4.2.1 Introduction.................................................... 102
Contents xi
4.2.2 TimeVariationinDNNInference............................ 103
4.2.3 UncertaintiesinDNNInference.............................. 105
4.2.4 ProfilingTools................................................. 106
5 DatasetandBenchmark..................................................... 109
5.1 OpenDatasetforAutonomousDriving............................... 109
5.1.1 Introduction.................................................... 109
5.1.2 ObjectDetection............................................... 111
5.1.3 ObjectTracking ............................................... 115
5.1.4 SemanticSegmentation ....................................... 117
5.1.5 Stereo .......................................................... 118
5.1.6 Localization.................................................... 119
5.1.7 AdvanceIntelligentServices ................................. 121
5.1.8 Summary....................................................... 122
5.2 CAVBench: A Benchmark Suite for Connected and
AutonomousVehicles ................................................. 122
5.2.1 Introduction.................................................... 122
5.2.2 RelatedWork .................................................. 124
5.2.3 BenchmarkDesign............................................ 126
5.2.4 BenchmarkCharacterization.................................. 134
5.2.5 Summary....................................................... 141
6 AutonomousDrivingSimulators........................................... 143
6.1 Introduction............................................................ 143
6.2 Motivation.............................................................. 144
6.3 Methodology........................................................... 145
6.4 Simulators.............................................................. 148
6.4.1 MATLAB/Simulink........................................... 148
6.4.2 CarSim......................................................... 149
6.4.3 PreScan ........................................................ 149
6.4.4 CARLA........................................................ 150
6.4.5 Gazebo......................................................... 151
6.4.6 LGSVL......................................................... 152
6.5 Comparison ............................................................ 152
6.6 Challenges ............................................................. 154
6.7 Summary............................................................... 156
7 HardwarePlatforms......................................................... 157
7.1 HydraOne .............................................................. 157
7.1.1 Introduction.................................................... 157
7.1.2 RelatedWork .................................................. 159
7.1.3 DesignandImplementation................................... 159
7.1.4 CaseStudies ................................................... 164
7.1.5 Summary....................................................... 166
7.2 Hydra................................................................... 167