Vision Models for High Dynamic Range and Wide Colour Gamut Imaging: Techniques and Applications

315 Pages·2019·32.486 MB·English

by  Marcelo Bertalmío

Vision Models for High
Dynamic Range and
Wide Colour Gamut
Imaging
Techniques and Applications
Computer Vision and
Pattern Recognition
Series
SeriesEditors
HorstBischof InstituteforComputerGraphicsandVision,
GrazUniversityofTechnology,Austria
KyoungMu DepartmentofElectricalandComputerEngineering,
SeoulNationalUniversity,RepublicofKorea
SudeepSarkar DepartmentofComputerScienceandEngineering,
UniversityofSouthFlorida,Tampa,UnitedStates
AlsointheSeries:
LinandZhang,Low-RankModelsinVisualAnalysis:Theories,Algorithmsand
Applications,2017,ISBN:9780128127315
Zhengetal.,StatisticalShapeandDeformationAnalysis:Methods,
ImplementationandApplications,2017,ISBN:9780128104934
DeMarsicoetal.,HumanRecognitioninUnconstrainedEnvironments:Using
ComputerVision,PatternRecognitionandMachineLearningMethodsfor
Biometrics,2017,ISBN:9780081007051
Sahaetal.,Skeletonization:Theory,MethodsandApplications,2017,
ISBN:9780081012918
Zhengetal.,StatisticalShapeandDeformationAnalysis:Methods,
ImplementationandApplications,2017,ISBN:9780128104934
Murinoetal.,GroupandCrowdBehaviorforComputerVision,2017,
ISBN:9780128092767
LeoandFarinella,ComputerVisionforAssistiveHealthcare,2018,
ISBN:9780128134450
Alameda-Pinedaetal.,MultimodalBehaviorAnalysisintheWild,2019,Advances
andChallenges,ISBN:9780128146019
Wangetal.,DeepLearningThroughSparseandLow-RankModeling,2019,
ISBN:9780128136591
YingYang,MultimodalSceneUnderstanding:Algorithms,ApplicationsandDeep
Learning,2019,ISBN:9780128173589
Huaetal.,SpectralGeometryofShapes:PrinciplesandApplications,2019,
ISBN:9780128138427
Vision Models for High
Dynamic Range and
Wide Colour Gamut
Imaging
Techniques and Applications
Marcelo Bertalmío
AcademicPressisanimprintofElsevier
125LondonWall,LondonEC2Y5AS,UnitedKingdom
525BStreet,Suite1650,SanDiego,CA92101,UnitedStates
50HampshireStreet,5thFloor,Cambridge,MA02139,UnitedStates
TheBoulevard,LangfordLane,Kidlington,OxfordOX51GB,UnitedKingdom
Copyright©2020ElsevierLtd.Allrightsreserved.
Nopartofthispublicationmaybereproducedortransmittedinanyformorbyanymeans,electronic
ormechanical,includingphotocopying,recording,oranyinformationstorageandretrievalsystem,
withoutpermissioninwritingfromthepublisher.Detailsonhowtoseekpermission,further
informationaboutthePublisher’spermissionspoliciesandourarrangementswithorganizationssuch
astheCopyrightClearanceCenterandtheCopyrightLicensingAgency,canbefoundatourwebsite:
www.elsevier.com/permissions.
Thisbookandtheindividualcontributionscontainedinitareprotectedundercopyrightbythe
Publisher(otherthanasmaybenotedherein).
Notices
Knowledgeandbestpracticeinthisfieldareconstantlychanging.Asnewresearchandexperience
broadenourunderstanding,changesinresearchmethods,professionalpractices,ormedicaltreatment
maybecomenecessary.
Practitionersandresearchersmustalwaysrelyontheirownexperienceandknowledgeinevaluating
andusinganyinformation,methods,compounds,orexperimentsdescribedherein.Inusingsuch
informationormethodstheyshouldbemindfuloftheirownsafetyandthesafetyofothers,including
partiesforwhomtheyhaveaprofessionalresponsibility.
Tothefullestextentofthelaw,neitherthePublishernortheauthors,contributors,oreditors,assume
anyliabilityforanyinjuryand/ordamagetopersonsorpropertyasamatterofproductsliability,
negligenceorotherwise,orfromanyuseoroperationofanymethods,products,instructions,orideas
containedinthematerialherein.
LibraryofCongressCataloging-in-PublicationData
AcatalogrecordforthisbookisavailablefromtheLibraryofCongress
BritishLibraryCataloguing-in-PublicationData
AcataloguerecordforthisbookisavailablefromtheBritishLibrary
ISBN:978-0-12-813894-6
ForinformationonallAcademicPresspublications
visitourwebsiteathttps://www.elsevier.com/books-and-journals
Publisher:MaraConner
AcquisitionEditor:TimPitts
EditorialProjectManager:LindsayLawrence
ProductionProjectManager:NirmalaArumugam
Designer:ChristianJ.Bilbow
TypesetbyVTeX
ALuis, Jose, MónicayMilagros.
AVera, Lucas, Serrana yGraciela.
Enrecuerdo demipadre.
Advance Praise for “Vision Models
for High Dynamic Range and Wide
Colour Gamut Imaging: Techniques
and Applications”
Dr.Bertalmíohaswrittenaveryusefultextthatconnectskeyperceptualphenom-
enatoimagereproduction.Histextnicelyexplainsthescientificandengineering
foundationsformanyimagecomputations,includinggamutmapping,tonemap-
ping,andimagequalitymetrics.Thematerialisathoughtfulreferenceforpeople
who wish to understand how analyses of human visual perception have been
transformed into applications that measure image quality. I am particularly en-
thusiastic about his call for new vision models to guide the future generation of
displaysandrendering!
ProfessorBrianA.Wandell,DirectoroftheCenterforCognitiveand
NeurobiologicalImaging,StanfordUniversity,USA
A lot of our colour science relies on experiments done decades ago, in viewing
conditionsthathaveverylittletodowithhowweconsumedramaticcontentto-
day.MarceloBertalmío’sbookmanagestomovethediscussionforwardintothe
21stcentury–bydescribingindetailhowweseeandprocessvisualcontent,and
thehugerolethatadaptationandefficientencodingofthevisualsignalplaywithin
it.Butalsobydirectlyrelatingthisknowledgetomodernhighdynamicrangedis-
play technology. This book is essential reading for anyone who is interested in
colourscienceforvisualmedia,andIwillkeepbuggingmycolleaguesatFilm-
Lighttoreaditandtoapplythewealthofknowledgecontainedinittowardsour
advancementofvisualstorytellingtechnology.
WolfgangLempp,founderanddirector,FilmLightLtd,UK
A comprehensive and rare synthesis of vision science and imaging technology.
This book is the go-to source for those wishing to learn more about the neuro-
scienceandpsychophysicsofvisionanditsimplicationsforthedesignofimaging
and display pipelines to achieve highest quality for the human observer. Such a
bookislongoverdue.
ProfessorBrunoOlshausen,HelenWillsNeuroscienceInstituteandSchoolof
Optometry,UCBerkeley,USA
Thisbookprovidesanin-depthbackgroundonvisionscienceforengineerswork-
ing on imaging applications. It clearly explains how HDR imaging must essen-
tiallyutilizetheadaptationcharacteristicsofthehumanvisualsystem,andcovers
vii
viii Advance Praise
practical solutions for HDR/SDR and WCG production based on a deep knowl-
edgeofvisionmodels.
Dr.KenichiroMasaoka,NHK(JapanBroadcastingCorporation)Science&
TechnologyResearchLaboratories,Japan
Thisisagreatbookforanyoneinterestedintoneandgamutmapping.Butthisis
alsoagreatbookforvisionscientistsingeneral,linkingourbasicunderstanding
of human vision gained through neuroscience, psychophysics and the theory of
efficientcodingtoapplicationsinimagingandcinema—anditnicelyhighlights
thefundamentallimitationsofourcurrentunderstandingofthehumanvisualsys-
tem. I thoroughly enjoyed reading Marcelo Bertalmío’s book and I hope it finds
thewidereadershipitdeserves.
ProfessorFelixWichmann,TübingenUniversityandMaxPlanckInstitute,Germany
Contents
Abouttheauthor ............................................... xv
Acknowledgements ............................................ xvii
CHAPTER 1 Introduction .................................... 1
1.1 HDRandWCG ................................... 1
1.2 ImprovedimageappearancewithHDR/WCG ............ 1
1.3 TheneedforanHDRecosystem ...................... 2
1.4 ProblemstosolvetoenableanHDRecosystem ........... 3
1.4.1 Newcolourmanagementandgradingtools ......... 3
1.4.2 Newprojectionsystemsformovietheatres ......... 4
1.4.3 ToolsforconversionbetweenHDRandSDR........ 4
1.4.4 Toolsforcolourgamutconversions ............... 5
1.4.5 ProductionandeditingguidelinesforHDRmaterial .. 6
1.4.6 Toolsforpersonalisation ....................... 6
1.5 Overviewofthebook .............................. 7
References....................................... 9
CHAPTER 2 The biological basis of vision: the retina ......... 11
2.1 Howtheretinalimageiscreated ...................... 11
2.2 Retinalstructure .................................. 15
2.3 Photoreceptors .................................... 21
2.3.1 Howphotoreceptorsoperate .................... 21
2.3.2 Photoreceptorsensitivityandspatialdistribution ..... 22
2.3.3 Visualpigmentsandcolourperception ............ 24
2.3.4 Photoreceptorevolutionandtheconemosaic ........ 25
2.4 Receptivefieldsandlateralinhibition ................... 28
2.5 Horizontalcells ................................... 29
2.6 Bipolarcells:thebeginningofparallelprocessinginthevisual
system .......................................... 32
2.6.1 ON/OFF ................................... 33
2.6.2 Transient/sustained ........................... 35
2.6.3 Diffuse/midget/S-coneBC ..................... 35
2.7 Amacrinecells.................................... 36
2.8 Retinalganglioncells .............................. 36
2.8.1 Impulsegeneration ........................... 36
2.8.2 Mosaics,stratification,receptivefields............. 38
2.8.3 Midget,parasolandbistratifiedretinalganglioncells .. 40
2.8.4 Intrinsicallyphotosensitiveretinalganglioncells ..... 43
2.8.5 Retinalganglioncellssensitivetoorientationand
motion .................................... 44
ix
x Contents
References....................................... 44
CHAPTER 3 The biological basis of vision: LGN, visual cortex
and L+NL models .............................. 47
3.1 Lateralgeniculatenucleus ........................... 47
3.1.1 GeneralstructureoftheLGN ................... 47
3.1.2 LayersoftheLGN ........................... 49
3.1.3 ColourrepresentationintheLGN ................ 50
3.1.4 VisualprocessingintheLGN ................... 51
3.2 Cortex .......................................... 52
3.2.1 LayersofV1 ................................ 53
3.2.2 Simpleandcomplexcells ...................... 54
3.2.3 Oculardominancecolumnsandorientationcolumns .. 55
3.2.4 MapsofV1 ................................. 56
3.2.5 MotionandareaMT .......................... 56
3.2.6 Colourrepresentationinthecortex ............... 57
3.3 L+NLmodels .................................... 60
References....................................... 62
CHAPTER 4 Adaptation and efficient coding .................. 65
4.1 Thepurposeofadaptation ........................... 65
4.2 Efficientrepresentation ............................. 67
4.3 Theneuralcode ................................... 70
4.4 Meanandcontrastadaptation ......................... 71
4.4.1 Adaptationtothemean(lightadaptation) .......... 72
4.4.2 Contrastadaptation ........................... 77
4.5 Statisticsofnaturalimagesasafunctionofdynamicrange ... 80
4.5.1 Methods ................................... 81
4.5.2 Results .................................... 81
4.5.3 Discussion ................................. 88
References....................................... 91
CHAPTER 5 Brightness perception and encoding curves ...... 95
5.1 TheWeber–Fechnerlaw:brightnessperceptionislogarithmic 95
5.2 Stevens’law:brightnessperceptionisapowerlaw ......... 98
5.3 ConnectingthelawsofWeber–FechnerandStevens........ 99
5.4 Crispening:brightnessperceptionisneitherlogarithmicnora
powerlaw ....................................... 100
5.4.1 TwostudiesbyWhittle ........................ 101
5.4.2 Whittle’smodelforbrightnessperception:advantages
andlimitations .............................. 104
5.4.3 TwoalternativestoWhittle’smodel ............... 105
5.5 Detectingthresholdsdependsonfrequency:theContrast
SensitivityFunction ................................ 106
5.6 Effectofthesurroundonbrightnessperception:thesystem
gamma ......................................... 109