Algorithm & SoC Design for Automotive Vision Systems: For Smart Safe Driving System

296 Pages·2014·13.13 MB·English

by  Jaeseok Kim

Jaeseok Kim · Hyunchul Shin E ditors
Algorithm &
SoC Design for
Automotive
Vision Systems
For Smart Safe Driving System
Algorithm & SoC Design for Automotive Vision
Systems
Jaeseok Kim Hyunchul Shin
•
Editors
Algorithm & SoC
Design for Automotive
Vision Systems
For Smart Safe Driving System
123
Editors
Jaeseok Kim Hyunchul Shin
Yonsei University HanyangUniversity
Seoul Ansan
Korea,Republic of (SouthKorea) Korea,Republic of (SouthKorea)
ISBN 978-94-017-9074-1 ISBN 978-94-017-9075-8 (eBook)
DOI 10.1007/978-94-017-9075-8
Springer Dordrecht Heidelberg New YorkLondon
LibraryofCongressControlNumber:2014942316
(cid:2)SpringerScience+BusinessMediaDordrecht2014
Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations,
recitation,broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionor
informationstorageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilar
methodology now known or hereafter developed. Exempted from this legal reservation are brief
excerpts in connection with reviews or scholarly analysis or material supplied specifically for the
purposeofbeingenteredandexecutedonacomputersystem,forexclusiveusebythepurchaserofthe
work. Duplication of this publication or parts thereof is permitted only under the provisions of
theCopyright Law of the Publisher’s location, in its current version, and permission for use must
always be obtained from Springer. Permissions for use may be obtained through RightsLink at the
CopyrightClearanceCenter.ViolationsareliabletoprosecutionundertherespectiveCopyrightLaw.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this
publicationdoesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexempt
fromtherelevantprotectivelawsandregulationsandthereforefreeforgeneraluse.
While the advice and information in this book are believed to be true and accurate at the date of
publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor
anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with
respecttothematerialcontainedherein.
Printedonacid-freepaper
SpringerispartofSpringerScience+BusinessMedia(www.springer.com)
Preface
Anemergingtrendintheautomobileindustryisitsconvergencewithinformation
technology(IT).Indeed,ithasbeenestimatedthatalmost90 %ofnewautomobile
technologies involve IT in some form. Smart driving technologies that improve
safetyaswellasgreenfueltechnologiesarequiterepresentativeoftheconvergence
betweenITandautomobiles.Theformertechnologies,inparticular,includethree
key elements: sensing of driving environments, detection of objects and potential
hazards (pedestrians, lanes on the road, other vehicles, traffic signals, stationary
objects, etc.), and the generation of driving control signals including warning
signals. The first two elements are usually implemented using machine vision,
radar, and sonar together with vehicle-to-vehicle and vehicle-to-infrastructure
communication. These are the most commonly used technologies to realize novel
systems such as a smart parking assistance, lane keeping assistance, smart cruise
control,collisionavoidance,andultimatelyfullyautonomousself-drivingvehicles.
This book is organized into 10 chapters to cover system-on-a-chip (SoC)
design—includingbothalgorithmsandhardware—relatedwithimagesensingand
objectdetectionbyusingthecameraforsmartdrivingsystems,asshowninFig. 1.
First,lenscorrectiontechniquesfordistortedimagescapturedbycamerasinstalled
on vehicles are presented. Second, novel super-resolution algorithm suited to a
low-cost vehicle camera is presented. Next, image enhancement techniques to
improve object detection from the captured images are discussed. Then, two
chapters present algorithms and techniques for accurate detection of several dif-
ferent types of objects (pedestrians, road lanes other vehicles, traffic signals, sta-
tionaryobjects,etc.).ThisisfollowedbyadiscussionoftheSoCarchitectureand
hardwaredesignnecessarytoimplementthesealgorithmsinrealtime.Finally,the
software environment and reliability issues for automotive SoC platforms are
discussed.
Chapter 1introducestheneedsandrequirementsofanimagevisionsystemfor
smart and safe driving in a novel IT-converged automobile. Next, we present and
discusstheplatformarchitectureforaddressingthecomponentsoftheautomotive
vision system. This platform is implemented as a System-on-a-Chip (SoC) plat-
form, and its hardware architecture along with its software environment are
introduced.
v
vi Preface
Fig.1 SoCplatformarchitectureforautomobilevisionsystem
Chapter 2 presents a lens distortion correction algorithm based on a geometric
invariantsuitabletoavehiclecamera.Thismethodadoptscross-ratioinvariability
for a perspective projection and minimized distortion. In addition, we describe a
newgammacorrectionapproachtodealwithrapidvariationinluminousintensity.
To reduce the computation complexity, we introduce an objective numerical
descriptor for luminance and contrast as well as gamma correction based on a
tone-mapping approach.
Chapter 3 presents a novel super-resolution algorithm suited to a low-cost
vehicle camera. We introduce a smart and robust registration algorithm that takes
into account rotation and shift estimation. To reduce the registration error, this
algorithm determines the optimal reference image where even other super-
resolution algorithms discard this registration error. The algorithm follows the
warp–blur observation model because the blur parameter is considerably larger
than the warp parameter for camera rotation and/or vibration.
Chapter 4 introduces image processing algorithms for improving object
detection.Thesealgorithmscanbeusedtoimprovetheobjectrecognitionratefor
poor images that have been captured under inadequate conditions such as low
illumination, rainfall, or snowfall. Furthermore, a high-dynamic-range tone-
mapping technique that improves image quality is described.
Chapter 5 explains coarse-to-fine vehicle and pedestrian detection techniques.
Inthecaseofvehicledetection,wedescribemono-camera-basedvehicledetection
systemsinwhichlow-leveledgesandhigh-levelbag-of-featuresareincorporated.
Specifically, once initial candidates are obtained using edge information, these
candidates are further verified using a bag-of-features. On the other hand, in the
case of pedestrian detection, we explain certain popular Histogram of Oriented
Gradient(HOG) detectors andpart-baseddetectors. Theuseofedge-based coarse
detection in the base detectors greatly reduces the detection time.
Preface vii
Chapter 6discussesvariousdrivermonitoringsystemsaswellassomemethods
forpredictingunsafedrivingbehaviors.Wedescribethedesignconsiderationsthat
led us to develop the driver monitoring system architecture, discuss the software
responsibleforcontrolanddataacquisition,andpresentsomeofthedatascreening
and feature extraction algorithms that predict unsafe driving behaviors.
Chapter 7 covers various aspects of SoC architectures for automobile vision
system.Afterabriefintroduction, itsurveysexistingSoCarchitecturesaswellas
architecture design issues and methodologies, ranging from single- to multi-core
SoC architectures. The chapter further discusses on-chip communications among
cores, hardware blocks, and memories. It also covers the issue of mapping
applications to SoC architectures.
Chapter 8introducesahardwareaccelerator thatperformsinterestpointdetec-
tion and matching for image-based recognition applications in real time. Interest
pointdetectionandmatchingarebasicandoneofthemostcomputation-intensive
operations, in general, vision tasks such as object recognition/tracking, image
matching/stitching, and simultaneous localization and mapping. This chapter
describesapatternmatching-basedimagerecognitionprocessorthatunifiesarchi-
tecturesforfeaturesfromacceleratedsegmenttestsandbinaryrobustindependent
elementaryfeatureswithlowpowerdissipationandahighframerate.
Chapter 9introducesasoftwaredevelopmentenvironmentforautomotiveSoC.
AUTOSAR, a standardized automotive software architecture, is a partnership of
automotive manufacturers and suppliers working in collaboration to establish an
open industry standard for automotive E/E architectures. This chapter describes
this software architecture in detail, covering the consideration of safety require-
ments, scalability of different vehicle and platform variants, and implementation
andstandardizationofthebasicfunctionsbasedonthecooperationofstandards.In
addition, this chapter covers maintainability throughout the entire product life
cycle, software updates, and upgrades over vehicle lifetime.
Chapter 10 covers reliability issues of automobile electronic system. Current
vehicles are built with complex electronic systems embedded with more than a
hundred microprocessors through complicated automotive networks. In the de
factoISO26262standardintheautomotiveindustry,AutomotiveSafetyIntegrity
Level(ASIL)isclassifiedintofourdifferentlevels.Inthischapter,theISO26262
hardware ASIL is described in detail. Finally, we introduce fault diagnosis
architectures that use various designs for testability techniques such as scan
design, built-in self-test, and IEEE boundary scan design for increasing hardware
reliability.
Wewouldliketoappreciateourcolleagueswhocontributedtoeachchapterin
thisbook.ThisworkwassupportedbytheMinistryofTrade,IndustryandEnergy
(MOTIE) in Korea and the IDEC Platform center (IPC) for Smart cars.
We would like to thank the publishing editor Mark de Jongh and Mrs. Cindy
Zitter at Springer for their encouragement and continuous support to prepare this
book.
viii Preface
Finally, we wish to thank our graduate students who provided their efforts to
prepare some materials in this book.
Jaeseok Kim
Hyunchul Shin
Contents
1 Introduction. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Jaeseok Kim and Hyunchul Shin
2 Lens Correction and Gamma Correction. . . . . . . . . . . . . . . . . . . 11
Sang-Bock Cho
3 Super Resolution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41
Hyo-Moon Cho
4 Image Enhancement for Improving Object Recognition. . . . . . . . 73
Jaeseok Kim
5 Detection of Vehicles and Pedestrians . . . . . . . . . . . . . . . . . . . . . 107
Hyunchul Shin and Irfan Riaz
6 Monitoring Driver’s State and Predicting Unsafe
Driving Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
Hang-Bong Kang
7 SoC Architecture for Automobile Vision System . . . . . . . . . . . . . 163
Kyounghoon Kim and Kiyoung Choi
8 Hardware Accelerator for Feature Point Detection
and Matching. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Jun-Seok Park and Lee-Sup Kim
9 Software Development Environment for Automotive SoC . . . . . . 231
Jeonghun Cho
10 Reliability Issues for Automobile SoCs . . . . . . . . . . . . . . . . . . . . 263
Sungju Park
ix
Chapter 1
Introduction
Jaeseok Kim and Hyunchul Shin
Abstract This chapterintroduces conceptsandtrends related tothe development
of advanced driver assistance systems (ADAS), which aid human drivers in col-
lision avoidance. Some industrial developments and major functional blocks of
ADASarealsointroduced.Wehavefocusedonthecameravision-basedsystems,
which are very popular mainly because of their low development costs and very
diverse potential applications. The system-on-chip (SoC) architecture and com-
ponents of automobile vision systems are also presented.
1.1 Introduction to the Advanced Driver Assistance
System
In 2009, 5.5 million automobile crashes, 2.2 million automobile-related injuries,
and33,963motorvehiclefatalitieswerereportedintheUnitedStatesalone,while
in Europe, more than 40,000 casualties and 1.4 million injuries are caused by
vehicle-related accidents annually [1]. Based on these figures, researchers have
developed crash survival techniques and systems, and emphasized the need for
their installation in all vehicles. To some extent, these systems guarantee the
survival of the driver. Top motor vehicle brands have stringently equipped their
vehicles with these systems, i.e., airbags, active safety electronics, etc. Although
these advancesinpassivesafetyhave madepassengercarssaferthaneverbefore,
the potential for further improvements in passive safety features is limited.
J.Kim(&)
YonseiUniversity,Seoul,Korea
e-mail:[email protected]
H.Shin
HanyangUniversity,Ansan,Korea
e-mail:[email protected]
J.KimandH.Shin(eds.),Algorithm&SoCDesignforAutomotive 1
VisionSystems,DOI:10.1007/978-94-017-9075-8_1,
(cid:2)SpringerScience+BusinessMediaDordrecht2014