High Dynamic Range Video. From Acquisition to Display and Applications

587 Pages·2016·207.05 MB·English

by  Frédéric Dufaux

High Dynamic Range Video
From Acquisition to Display
and Applications
High Dynamic Range Video
From Acquisition to Display
and Applications
Edited by
Frédéric Dufaux
CNRS, Télécom ParisTech
Patrick Le Callet
University of Nantes
Rafał K. Mantiuk
University of Cambridge
Marta Mrak
British Broadcasting Corporation
AMSTERDAM (cid:129) BOSTON (cid:129) HEIDELBERG (cid:129) LONDON
NEW YORK (cid:129) OXFORD (cid:129) PARIS (cid:129) SAN DIEGO
SAN FRANCISCO (cid:129) SINGAPORE (cid:129) SYDNEY (cid:129) TOKYO
Academic Press is an imprint of Elsevier
AcademicPressisanimprintofElsevier
125LondonWall,London,EC2Y5AS,UK
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UK
©2016ElsevierLtd.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronicormechanical,including
photocopying,recording,oranyinformationstorageandretrievalsystem,withoutpermissioninwritingfromthepublisher.
Detailsonhowtoseekpermission,furtherinformationaboutthePublisher’spermissionspoliciesandourarrangements
withorganizationssuchastheCopyrightClearanceCenterandtheCopyrightLicensingAgency,canbefoundatour
website:www.elsevier.com/permissions.
ThisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythePublisher(otherthanasmaybe
notedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperiencebroadenourunderstanding,
changesinresearchmethods,professionalpractices,ormedicaltreatmentmaybecomenecessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluatingandusinganyinformation,
methods,compounds,orexperimentsdescribedherein.Inusingsuchinformationormethodstheyshouldbemindfuloftheir
ownsafetyandthesafetyofothers,includingpartiesforwhomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assumeanyliabilityforanyinjury
and/ordamagetopersonsorpropertyasamatterofproductsliability,negligenceorotherwise,orfromanyuseoroperationof
anymethods,products,instructions,orideascontainedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-08-100412-8
ForinformationonallAcademicPresspublications
visitourwebsiteathttps://www.store.elsevier.com/
Contributors
M.A.Abebe
Technicolor,Cesson-Sévigné,France
N.Adami
UniversityofBrescia,Brescia,Italy
D.Agrafiotis
UniversityofBristol,Bristol,UnitedKingdom
L.Albani
BarcoFIMI,Saronno(VA),Italy
M.A.Ali
UniversityofToronto,Toronto,ON,Canada
P.Andrivon
Technicolor,Cesson-Sevigne,France
A.Artusi
UniversityofGirona,Girona,Spain
A.Badano
USFoodandDrugAdministration,SilverSpring,MD,UnitedStates
R.Boitard
IRISA,Rennes;Technicolor,Cesson-Sevigne,France
S.Bonfiglio
BarcoFIMI,Saronno(VA),Italy
P.Bordes
Technicolor,Cesson-Sevigne,France
K.Bouatouch
IRISA,Rennes,France
R.Brémond
ParisEstUniversity,Marne-la-Vallée,France
D.R.Bull
UniversityofBristol,Bristol,UnitedKingdom
A.Chalmers
UniversityofWarwick,Coventry,UnitedKingdom
R.Cozot
IRISA,Rennes,France
M.P.DaSilva
UniversityofNantes,Nantes,France
xix
xx Contributors
S.Daly
DolbyLaboratories,Inc.,SanFrancisco,CA,UnitedStates
N.-T.Dang
ParisEstUniversity,Marne-la-Vallée,France
F.DeSimone
TelecomParisTech,CNRSLTCI,Paris,France
K.Debattista
UniversityofWarwick,Coventry,UnitedKingdom
F.Dufaux
TelecomParisTech,CNRSLTCI,Paris,France
G.Eilertsen
LinköpingUniversity,Norrköping,Sweden
S.Forchhammer
TechnicalUniversityofDenmark,Bristol,Denmark
E.François
Technicolor,Cesson-Sevigne,France
J.Froehlich
DolbyLaboratories,Inc.,SanFrancisco,CA,UnitedStates
O.Gallo
NVIDIAResearch,SantaClara,CA,UnitedStates
G.Guarnieri
UniversityofTrieste,Trieste,Italy
F.Guerrini
UniversityofBrescia,Brescia,Italy
S.Hajisharif
LinköpingUniversity,Norrköping,Sweden
B.Karr
KennedySpaceCenter,FL,UnitedStates;UniversityofWarwick,Coventry,
UnitedKingdom
J.Korhonen
TechnicalUniversityofDenmark,Bristol,Denmark
J.Kronander
LinköpingUniversity,Norrköping,Sweden
T.Kunkel
DolbyLaboratories,Inc.,SanFrancisco,CA,UnitedStates
M.-C.Larabi
UniversityofPoitiers,Poitiers,France
Contributors xxi
S.Lasserre
Technicolor,Cesson-Sevigne,France
P.LeCallet
UniversityofNantes,Nantes,France
F.LeLéannec
Technicolor,Cesson-Sevigne,France
C.Lèbre
Binocle,Paris,France
R.Leonardi
UniversityofBrescia,Brescia,Italy
O.Letz
Binocle,Paris,France
C.Loscos
UniversityofReimsChampagne-Ardenne,Reims,France
S.Mann
UniversityofToronto;RotmanSchoolofManagementCDL,Toronto,ON,Canada;Meta,
RedwoodCity,CA,UnitedStates
C.Mantel
TechnicalUniversityofDenmark,Bristol,Denmark
R.K.Mantiuk
UniversityofCambridge,Cambridge;BangorUniversity,Bangor,UnitedKingdom
I.Martin
UniversityofGirona,Girona,Spain
S.Miller
DolbyLaboratories,Inc.,SanFrancisco,CA,UnitedStates
M.Narwaria
UniversityofNantes,Nantes,France
M.Okuda
UniversityofKitakyushu,Kitakyushu,Japan
R.R.Orozco
UniversityofGirona,Girona,Spain
T.Pouli
Technicolor,Cesson-Sévigné,France
Y.Pupulin
Binocle,Paris,France
G.Ramponi
UniversityofTrieste,Trieste,Italy
xxii Contributors
E.Reinhard
Technicolor,Cesson-Sévigné,France
T.Richter
UniversityofStuttgart,Stuttgart,Germany
T.Ritschel
MPIInformatik;SaarlandUniversity,Saarbrücken,Germany
U.Seger
RobertBoschGmbH,Leonberg,Germany
P.Sen
UniversityofCalifornia,SantaBarbara,CA,UnitedStates
X.Shu
McMasterUniversity,Hamilton,ON,Canada
J.Unger
LinköpingUniversity,Norrköping,Sweden
G.Valenzise
TelecomParisTech,CNRSLTCI,Paris,France
C.Villa
ParisEstUniversity,Marne-la-Vallée,France
X.Wu
McMasterUniversity,Hamilton,ON,Canada
Y.Zhang
UniversityofBristol,Bristol,UnitedKingdom
Editor Biographies
Frédéric Dufaux is a Centre National de la Recherche Scientifique (CNRS) research director at
TelecomParisTech.HeisalsotheEditor-in-ChiefofSignalProcessing:ImageCommunication.
Frédéric received his M.Sc. in physics and his Ph.D. in electrical engineering from the Swiss
FederalInstituteofTechnologyinLausannein1990and1994,respectively.Hehasmorethan20years
ofexperienceinresearch,previouslyholdingpositionsattheSwissFederalInstituteofTechnologyin
Lausanne,EmitallSurveillance,Genimedia,Compaq,DigitalEquipment,MIT,andBellLabs.Hehas
beeninvolvedinthestandardizationofdigitalvideoandimagingtechnologies,beingamemberofthe
MPEGandJPEGcommittees.HeistherecipientoftwoISOawardsforhiscontributions.Frédéricisa
fellowofIEEE.HewasViceGeneralChairofthe2014InternationalConferenceonImageProcessing.
HeisanelectedmemberoftheIEEEImage,Video,andMultidimensionalSignalProcessingTechnical
CommitteeandtheIEEEMultimediaSignalProcessingTechnicalCommittee.HeisalsoChairofthe
European Association for Signal Processing (EURASIP) Special Area Team on Visual Information
Processing.
His research interests include image and video coding, distributed video coding, 3D video, high
dynamicrange imaging,visual quality assessment, videosurveillance,privacyprotection,image and
videoanalysis,multimediacontentsearchandretrieval,andvideotransmissionoverwirelessnetworks.
He is the author or coauthor of more than 120 research publications and holds 17 patents issued or
pending.
PatrickLe Callet is Professorat the UniversityofNantes/PolytechNantesand leadsresearchatthe
InstitutdeRechercheenCommunicationsetCybernétiquedeNantes(IRCCyN)/CentreNationaldela
RechercheScientifique(CNRS)laboratory.
He received both an M.Sc. and a Ph.D. in image processing from École Polytechnique de
l’Université de Nantes. He was also a student at École Normale Superieure de Cachan, where he
sattheaggrégation(competitiveexamination)inelectronicsoftheFrenchnationaleducationsystem.
He worked as an assistant professor from 1997 to 1999 and as a full-time lecturer from 1999 to
2003 in the Department of Electrical Engineering of the Technical Institute of the University of
Nantes. Since 2003 he has taught at École Polytechnique de l’Université de Nantes (Engineering
School) in the Electrical Engineering Department and the Computer Science Department, where is
now a full professor. Since 2006, he has been the head of the Image and Video Communication
Laboratory at CNRS IRCCyN, a group of more than 35 researchers. He is mostly engaged in
research dealing with the application of human vision modeling in image and video processing.
His interests currently center on 3D image and video quality assessment, watermarking techniques,
and visual attention modeling and applications. He is the coauthor of more than 200 publications
and communications and the co-inventor named on 13 international patents on these topics. He
also cochairs within the Video Quality Expert Group, the HDR Group, and 3DTV activities.
He is currently serving as an associate editor of IEEE Transactions on Image Processing, IEEE
Transactions on Circuit System and Video Technology, EURASIP Journal on Image and Video
Processing,andSPIEElectronicImaging.
xxiii
xxiv Editor Biographies
Rafał K. Mantiuk is a senior lecturer in the Computer Laboratory, University of Cambridge (UK).
He received his Ph.D. from the Max Planck Institute for Computer Science (Germany, 2006), was
a postdoctoral researcher at the University of British Columbia (Canada) and a lecturer at Bangor
University(UK).HehaspublishednumerousjournalarticlesandconferencepaperspresentedatACM
SIGGRAPH,Eurographics,IEEEComputerVisionandPatternRecognition,andSPIEHumanVision
and Electronic Imagingconferences, has been awarded severalpatents, and was recognizedwith the
HeinzBillingAward(2006).Hehasmadecontributionstothefieldofhighdynamicrangeimagingin
theareasofvideocompression,tonemapping,andqualityassessment.Inhisworkheinvestigateshow
knowledgeofthehumanvisualsystemandperceptioncanbeincorporatedwithincomputergraphics
and imagingalgorithms.His recentinterests focuson the designof imagingalgorithmsthatadaptto
human visual performance and viewing conditions in order to deliver the best images given limited
resources,suchasbandwidth,computationtime,anddisplaycontrast.
MartaMrak receivedDipl. Ing.andM.Sc. in electronicengineeringfromthe Universityof Zagreb,
Croatia, and a Ph.D. from Queen Mary University of London, UK. Before joining the Research
and Development Department at the BBC in 2010 to work on video compression research and the
H.265/HEVC standardization, she was a postdoctoral researcher at the University of Surrey and
QueenMaryUniversityofLondon.In2002,shewasawardedaGermanAcademicExchangeService
(DAAD) scholarship for video compression research at the Heinrich Hertz Institute, Germany. She
hascoauthoredmorethan100articles,bookchapters,andstandardizationcontributions,andcoedited
the bookHigh-QualityVisualExperience(Springer,2010).Shehasbeeninvolvedin severalprojects
fundedbyEuropeanandUKresearchcouncilsinrolesrangingfromresearchertoscientificcoordinator.
She is a member of the Multimedia Signal Processing Technical Committee of the IEEE, a senior
memberoftheIEEE,andanareaeditorofSignalProcessing:ImageCommunication,andhasbeena
guesteditorofseveralspecialissuesinrelevantjournalsinthefield.
Preface
Producing truly realistic video is widely seen as the holy grail toward further improving the quality
of experiencefor endusersof multimediaservices. Additionally,numerousprofessionalapplications
canbemadepossibleonlybytheuseofvideosignalswithpropertiesbeyondthewidespread,butvery
limited,parameterspace,suchasstandarddynamicrange.Severalnewapproacheshavebeenproposed
and often successfully implemented in the history of the applications that use video data. Currently
investigateddirectionsincludehighspatialresolution,highframerates,widecolorgamutandhigh-bit-
depthrendering.
The human visual system is able to perceive a wide range of colors and luminous intensities, as
present in outdoor scenes in everyday life, ranging from bright sunshine to dark shadows. However,
currenttraditionalimagingtechnologiescannotcapturenorreproducesuchabroadrangeofluminance.
Theobjectiveofhighdynamicrange(HDR)imagingistoovercometheselimitations,henceleadingto
morerealisticvideocontentandagreatlyenhanceduserexperience.
HDRappliedtostillimageshasbeenanactivefieldofresearchanddevelopmentformanyyears,
especiallyforphotography.However,itsextensiontovideocontenthasbeenconsideredonlyrecently.
Thanks to rapid technological advancements, commercial HDR video cameras and HDR monitors
are becoming available. Nevertheless, the effective deploymentof HDR video technologies involves
redefiningcommoninterfacesforend-to-endcontentdelivery,whichinturnentailsmanytechnicaland
scientificchallengesforbothacademiaandindustry.
ThisbookprovidesanoverviewofrecentlyresearchedtechnologicaldirectionsforenablingHDR
videoinkeyapplicationareas.Morespecifically,itcoversthestateoftheart,discussestheeffectiveness
of various techniques, reviews some of the standardization efforts during the time of writing, and
exploresnewresearchdirections.
By providing a broad coverage, including general and advanced topics, the book should be of
interest to a large readership with different backgrounds and expectations. Target readers include
researchers,students,engineers,practitioners,andmanagers.Thisbookhasbeeneditedwiththeaim
forittobethefirstreferenceonmanytopicsthatencompassHDRvideo,andtobearelevanthandbook
forthoseinvolvedorinterestedinthisfield.
The aim is to cover numerous aspects required to establish HDR video systems, ranging from
content acquisition to display technologies. Although several applications that use HDR video
technologies are discussed, the range of topics covered provides a solid base for the establishment
ofnewHDRapplicationareas.
Thisbookis composedofsix parts. Afteranintroductionto thebackgroundandfundamentalsof
HDR imaging, Part I addresses the problem of HDR content acquisition and production. Different
approachesare describedto captureHDR video,includingthe use of multiple synchronizedsensors,
spatial multiplexing of the sensor response, merging of multiple images captured with different
exposuretimes,andcapturingmultiviewHDRvideo.TheimpactofHDRoncinematographicshooting
isalsodiscussed.
xxv