Stochastic Modeling and Resolution-Free Rendering of Film Grain

18 Pages·2017·31.44 MB·English

Stochastic Modeling and Resolution-Free Rendering of
Film Grain
Alasdair Newson, Bruno Galerne, Julie Delon
To cite this version:
Alasdair Newson, Bruno Galerne, Julie Delon. Stochastic Modeling and Resolution-Free Rendering of
Film Grain. 2016. ￿hal-01358392￿
HAL Id: hal-01358392
https://hal.science/hal-01358392
Preprint submitted on 8 Sep 2016
HAL is a multi-disciplinary open access L’archive ouverte pluridisciplinaire HAL, est
archive for the deposit and dissemination of sci- destinée au dépôt et à la diffusion de documents
entific research documents, whether they are pub- scientifiques de niveau recherche, publiés ou non,
lished or not. The documents may come from émanant des établissements d’enseignement et de
teaching and research institutions in France or recherche français ou étrangers, des laboratoires
abroad, or from public or private research centers. publics ou privés.
Stochastic Modeling and Resolution-Free Rendering of Film Grain
AlasdairNewson1,BrunoGalerne2 andJulieDelon3
1LaboratoireMAP5,Universite´ ParisDescartesandCNRS,SorbonneParisCite´,France
August24,2016
Abstract
greateffortsaremadetorecreatethe“soul”ofcertaintypes
offilmemulsion.
Therealisticsynthesisandrenderingoffilmgrainisacru-
Several factors contribute fundamentally to the feel of
cialgoalformanyamateurandprofessionalphotographers
an analog film. Some of the main ones are image con-
andfilm-makerswhoseartisticworksrequiretheauthentic
trast,colorpaletteandfilmgrain. Filmgrainisthetexture
feel of analog photography. The objective of this work is
causedbythefactthatanalogphotographs,mostlycreated
to propose an algorithm that reproduces the visual aspect
by silver-halide type processes, are the result of many mi-
of film grain texture on any digital image. Previous ap-
croscopicphoto-sensitiveparticlesreactingtolight. These
proachestothisproblemeitherproposeunrealisticmodels
particlesarecalledgrains. Thus,onthemicroscopicscale,
orsimplyblendscannedimagesoffilmgrainwiththedig-
an analog image is binary : either a particle is present, in
italimage,inwhichcasetheresultisinevitablylimitedby
whichcaselightisblocked,oritisnot,inwhichcaselight
the quality and resolution of theinitial scan. In this work,
istransmitted. Becausehumanscanseewithalimitedres-
weintroduceastochasticmodeltoapproximatethephysi-
olution, what we perceive as an image is in reality a local
calrealityoffilmgrain,andproposearesolution-freeren-
averagedensityofgrains. Theresultingvisualaspectisof-
deringalgorithmtosimulaterealisticfilmgrainforanydig-
tenreferredtoas“graininess”.
italinputimage. Byvaryingtheparametersofthismodel,
There are two main approaches to film grain synthesis.
we can achieve a wide range of grain types. We demon-
The first, which is commonly used in many commercial
strate this by comparing our results with film grain exam-
solutions such as DxO’s FilmPack [DxO16], is to apply a
plesfromdedicatedsoftware, andshowthatourrendering
stored example of film grain to the digital image. Advan-
results closely resemble these real film emulsions. In ad-
tagesofthisapproachincludespeedandsimplicity. How-
ditiontorealisticgrainrendering,ourresolution-freealgo-
ever,theresultswillnecessarilybedeterministic,thatisto
rithmallowsforanydesiredzoomfactor,evendowntothe
saythatifweapplythesynthesistwicetothesameimage,
scaleofthemicroscopicgrainsthemselves.
we obtain the same output. This is clearly a considerable
disadvantage for the synthesis of a random phenomenon,
1 Introduction and will be particularly visible if the algorithm is applied
to video sequences. Furthermore, this approach is com-
pletely reliant on the resolution and quality of the original
The digital revolution has changed the way we process,
scan. Thesecondapproachtofilmgrainsynthesisistouse
store and view images. An image unit, the pixel, is sim-
a grain model. This is more frequently used in academic
plyaquantizednumberrepresentinginsomemannerlight
works [MRB06,Yan97,OLK09]. However, most of these
orcolorintensity,anditcanbestoredandtransmittedina
modelsrelyagainonanexampleofscannedfilmgrainfor
uniqueandunambiguousfashion. Thisleadstomanyprac-
synthesis, whichentailsthesamedrawbacksasmentioned
tical advantages of digital over “analog” photography (re-
above. In the case where film grain is modelled as inde-
produceability, robustness etc.). However, when it comes
pendent noise [MRB06,Yan97], with a variance which is
tocreatingimagesofhighartisticandvisualquality,agreat
dependentontheinputimageintensity,thegraintextureis
many amateur and professional photographers and film-
completely uncorrelated spatially, which gives a distinctly
makers prefer to use analog photography, in other words
“digital”feeltotheimage. Indeed, thisisoneofthemain
images produced with silver-halide based processes. In a
criticismsofphotographersandfilm-makerstowardsdigital
recentinterview[Lac15]EdwardLachman,thedirectorof
filmgrainsynthesismethods. InEdwardLachman’sinter-
cinematographyofthefilm“Carol”,madethechoicetouse
viewhediscussesthis,statingthatitispossibleto“recreate
16mmfilminordertocapturethefeelofaspecificperiod.
graindigitallynow,butitispixel-fixated. Itdoesnothave
Totakeanevenmoreextremeexample,therecognizeddi-
this anthropomorphic quality in which the grain structure
rectorQuentinTarantinodeclaredinthe2014Cannesfilm
ineachframeischanging”. Thus,thespatialcorrelationof
festival that “digital projection ... is the death of cinema
filmgrainisoneofitsdefiningfeatures.
as I know it” [Smi14]. Given the opinions of such promi-
nentphotographersandfilm-makers,itisnotsurprisingthat In light of the drawbacks of the previous methods, we
1
Fullresolution Zoom4× Zoom8× Zoom20×
Figure 1: Film grain rendering results with several zoom factors. We propose a stochastic film grain model and a film
grainrenderingalgorithmwhichcanrenderfilmgrainonadigitalimageatanychosenresolution.
proposeaphysicallymotivatedfilmgrainmodelandasyn- 2.1 Previouswork
thesis algorithm to produce the grain texture for a given
The silver-halide photographic process has been exten-
input image. To our knowledge, this is the first synthesis
sivelystudiedsincethebeginningofthetwentiethcentury.
algorithm based on a physical model of the photographic
GurneyandMott[Gur38]proposedacomprehensivephys-
process. Thecontributionsofthisworkare:
ical model of the process, which is widely accepted. Nut-
1. a resolution-free model of film grain based on the ting[Nut13]wasthefirsttostudythestatisticalproperties
physicalprocessofsilver-halidephotography; oftheso-called“randomdotmodel”forfilmgrain,andpro-
posedtheNuttingformulawhichlinkstheopticaldensityof
2. a Monte-Carlo based algorithm to render a given im-
afilmemulsiontotheaveragenumberofgrainspresentand
agewithfilmgrainatachosenresolution;
to the size of the inspected region. An important quantity
studiedintheanalogliteratureisgranularityorroot-mean-
3. tunableparametersbasedonthephysicalcharacteris-
square (rms) granularity. This is experimentally measured
tics of the film grain, such as the grain size and size
using a microdensitometer on any given film emulsion af-
distribution;
ter development, and corresponds to the standard devia-
Ourfilmgrainmodeliscontinuous,andweproduceadis- tionoftheopticaldensityoftheemulsion. Anotheruseful
crete image during the last step of the algorithm, once the connected quantity is that of Selwyn granularity [Wer94],
photographic process has been imitated. This is a signifi- whichisbasicallythegranularitydefinedinsuchafashion
cant advantage, as it results in a resolution-free algorithm that it is independent of the aperture size. A good sum-
which can “zoom” indefinitely on the image, until the in- mary of these basic notions may be found in the paper of
dividual grains are visible. This is not possible for either Bayer[Bay64]inthecontextoftherandomdotmodel. A
the methods based on scanned examples of grain, as the particularlyhottopicconcerningfilmgrainisthatofgrain
resolutionisfixed,orthosewhichmodelfilmgrainasinde- “clumping”. This corresponds to the perceived clustering
pendentlydistributednoise. of film grain. Much of the subsequent analog literature is
concernedwithproposingmathematicalmodelswhichim-
itatethiseffect[CKT73,LTW72,TU83].
2 Background and the photographic
Many approaches use actual scanned examples of film
process grain for the purposes of synthesis. This appears to be
the most popular approach in the industrial environment
Theliteratureconcerningfilmgrainandfilmgrainsynthe- as well as in some academic approaches. Film grain syn-
sisisclearlyseparatedaccordingtowhethertheybelongto thesis products such as Dxo’s “FilmPack” [DxO16] and
theanalogworldorthedigitalworld. Thoseintheformer GrubbaSoftware’s“TrueGrain”[Gru15]toolstakethisap-
categoryareconcernedexclusivelywithidentifyingvisual proach. More precisely, a single grain image is saved for
andstatisticalcharacteristicsoffilmgrain. Thedigitalcat- each film type. In the same philosophy, Schallauer and
egory looks at both removal and synthesis of film grain. Mo¨rzinger [SM06] extract the grain pattern from real im-
Fromonepointofview,filmgrainmaybeconsideredasa agesofgrainandsynthesizeanewgrainimagefromthese
kind of noise, and thus should be removed. However, de- examples, which they then apply to the image in an ad-
noisedimagestendtobetoosmoothandnotvisuallypleas- ditive fashion. Unfortunately, they do not go into detail
ing,andthereforeaneffortisalsomadetotryandrecreate as to how this synthesis is carried about. A similar ap-
the grain [OLK09]. Thus, many works of the digital cate- proachisusedintheFilmEmulationfeatureoftheG’MIC
gorytrytoprovidebothpossibilities. Unfortunately,tothe free software [G’M16] using the random phase texture al-
bestofourknowledgenoneofthesemethodstakeaccount gorithm [GGM11] to synthesize large grain textures from
of the analog literature. One of our goals is to produce small stored samples. Bae et al. [BPD06] use the classi-
a digital algorithm based on the modeling of the physical calHeeger-Bergentexturesynthesis[HB95]approachona
photographicprocessstudiedintheanalogliterature. constantregioninanexamplegrainyimagetoproducefilm
2
Real-worldimage Negativeanalogimage Positiveanalogimage
Figure 2: An illustration of the different steps of the photographic process. The signal from the real-world image is
recordedonthefilmnegativewithdarkfilmgrainsonalightbackground. Thisisinvertedinthepositiveimage.
grain. However,theydonotspecifyhowthisisappliedtoa Fromnegativetopositive Thefinalpositiveimagetakes
given input image. Stephenson and Saunders [SS07] filter form on photographic (photosensitive) paper. To do this,
whitenoiseintheFourierdomain. Yanetal.[Yan97]pro- light is shone through the negative film onto the photo-
posed an additive film grain model with signal-dependent graphic paper. During this step, the image is typically en-
noise. Adrawbackisthattheirapproachsupposesthatfilm largedbyafactorofbetweenfiveandtentimes. Afterthe
grainnoiseisspatiallyuncorrelated,whichisclearlyunre- exposure of the photographic paper, a positive representa-
alistic for film grain noise. Oh et al. [OLK09] propose an tionoftheoriginalimagehasbeenrecorded. Asasimpli-
auto-regressivemodelforfilmgrainremovalandsynthesis. fication, we shall consider that the photographic paper is
Theypointoutthatspatialcorrelationiscrucialforproduc- a continuous recording material, even if the photographic
ingrealisticfilmgrain.However,theyconsiderthataninput paper can contain its own “grain”. An illustration of the
grainyimageisavailable,andthatthecharacteristicsofthe negativeandpositiveimagescanbeseeninFigure2.
grainmaybeextracted.
Acommondrawbackoftheapproachesofthedigitallit-
3 Review of the Boolean model
erature is that no model based on the physical reality of
film grain is proposed. Either a digital example of film
grain, with fixed resolution, is considered to be available Wewishtomodelthephotographicprocessinordertopro-
(which may not always be possible), or spatial correlation duce a realistic image with film grain. From the previous
of the film grain texture is not considered. Furthermore, section, we know that a photographic image is made up
even if a good example of film grain is available, it is not of microscopic grains of solid silver. The simplest way
obvious how to blend this grain with an input image. In of modeling this is to consider that the grains are convex
thiswork,weproposearealisticfilmgrainmodelbasedon sets which are uniformly distributed in the film emulsion.
physicalconsiderationswhichrequiresnoexampleforsyn- This model is implicitly used in much of the “analog” lit-
thesis,andwhichdoesnotneedanysuchblendingprocess. erature concerning film grain [Nut13,Sel35]. This model,
with a few further hypotheses, corresponds very nicely to
a well studied model from the stochastic geometry litera-
2.2 Thephotographicprocess ture[CSKM13],knownastheBooleanmodel.
Thephotographicprocessisbasedontwosteps: filmgrain 3.1 TheBooleanmodel
sensitization and development. Sensitization takes place
whensilverhalidecrystalsareexposedtolightforacertain Let us first recall basic definition and properties regarding
amountoftimeandmade“developable”bytheinteraction the homogeneous Boolean model in R2 which is the most
withincomingphotons. Developmentistheprocesswhich naturalandthemoststudiedexampleofhomogeneousran-
turnsthesensitizedcrystalsintosolidgrainsofsilver.These dom sets [CSKM13]. Let Φ = {x , i ∈ N} represent a
i
stepsproduceanegativeimage,whichisthenconvertedto Poisson process on R2 with intensity λ. These x repre-
i
apositiveimagewithasecondphotographicprocess.Since sent the centers of our grains. We also define a sequence
the grain blocks light, the negative photographic image is ofidenticallyandindependentlydistributed(i.i.d.) random
in fact a binary function which is equal to 0 in the areas compact sets in R2, X ,X ,..., which will represent the
0 1
coveredbythegrains,andequalto1otherwise. grainshapes. TheBooleanmodelistherandomsetZ de-
3
Constantradius Randomradius Triangulargrains
Figure 3: Illustration of the Boolean model. Three dif-
ferent Boolean models with balls of constant radius (left),
Inputpixels(close-up) InhomogeneousBoolean
ballsofrandomradiusfollowingalog-normaldistribution
model
(middle),andrandomlyorientedtriangles.Observethatthe
Figure4: IllustrationofinhomogeneousBooleanmodel.
modelsdisplay“groups”ofgrains,asinrealfilmgrain.
The local intensity λ(y) of the inhomogeneous Boolean
modelischosentorespecttheinputpixelgray-levels.
fined as the union of all the shapes X placed at the loca-
i
tionsx ,thatis,
i
above, one can interpret and simulate such an inhomoge-
(cid:91) neous Poisson process by thinning an homogeneous Pois-
Z = (X +x ).
i i
son process having a large intensity [CSKM13] (which
i∈N
corresponds to a rejection method for Poisson processes).
Thisisaparticularlyflexiblemodel,aswecanchooseany However, in what follows, we will only consider intensity
sort of grain shape and size. In practice we shall use 2D functionsλthatarepiecewiseconstantandthusthecorre-
(cid:83)
balls, in which case Z = B(x ,r ), where r is the sponding Poisson point process can be easily simulated in
i∈N i i i
(possibly random) radius of the ith ball. We also present apiecewisemanner.
some experiments with other shapes in the supplementary Now that we have given a brief description of the
material. Finallyletusdefinetheindicatorfunctionofthe Boolean model, we proceed to see how it can be used to
Boolean model Z as the function 1 (y) that equals 1 if providerealisticfilmgrainrenderingforagivenimage.
Z
y ∈Z and0otherwise.
Let A = πr2 stand for the area of the ball indexed by
i i
4 Stochastic film grain model and
i. GiventhePoissonassumption,thevolumefractionofthe
Booleanmodelisgivenby rendering algorithm
P(1 (y)=1)=1−exp(−λE[A ]), (1)
Z 1
As recalled in Section 2.2, at a microscopic level a posi-
where E[A ] = πE[r2] is the common mean area of the tive analog photograph is a binary set of white grains in a
1 1
i.i.d. balls, and y ∈ R2. If thegrains X areballs ofcon- blackbackground. Givenaninputdigitalimageu,wewill
i
stantradiusr,wehaveP(1 (y)=1)=1−exp(−λπr2). define an inhomogeneous Boolean model such that, when
Z
seenatadistance,thecorrespondingrandombinarysetrep-
Tosummarize,theBooleanmodelconsistsof“white”balls
resentsthesameimageasu,butwiththeadditionalgrain-
on a “black” background, and we use this model to repre-
inesswhichcharacterizesanalogphotography. Intechnical
sentthephysicalrealityoffilmgrain.
terms,weneedtodefineavaryingintensityfunctionλfrom
Figure3showsexamplesofthreedifferentBooleanmod-
aninputimageu. Also, wewillnotmakeanyassumption
els. Animportantpointtonoteisthewell-knowntendency
onthegrainradiusdistributionsince,asresultsshow,thisis
of the model to produce the visual effect of clustering, or
a meaningful parameter for tuning the visual aspect of the
“clumping”, which is crucial to producing realistic film
graintexture.
grain. Now,inafilmemulsion,therewillbeavaryingden-
Let u : {0,...,m − 1} × {0,...,n − 1} ⊂ N2 →
sity of developed grains, which means the Boolean model
[0,u ]betheinputimage,ofsizem×n.Westartbynor-
as we have presented it is not sufficient yet. In stochastic max
malizingtheinputimageutotheinterval[0,1)bydefining
geometry, thisvarying density correspondsto the inhomo-
geneousBooleanmodel,whichwepresentnow. u˜(y)= umua(xy)+ε,whereεisasmallparameter,andumax is
themaximumpossiblegray-levelvalue. Werestricttheim-
ageto[0,1),asaBooleanmodelwithP(1 (y) = 1) = 1
3.2 TheinhomogeneousBooleanmodel Z
wouldrequireadegenerateinfiniteintensityλ.
Asinthehomogeneouscase, theinhomogeneousBoolean
model is built upon a sequence of random positions Φ = 4.1 Stochasticfilmgrainmodel
{x ,i∈N} given by a Poisson process. However, the in-
i
tensity λ of the Poisson process Φ is no longer constant. Ideally, we would like to imitate the photographic process
It is given by a function λ(y) which varies spatially with by choosing λ(y) to reflect the physical concentration of
y ∈ R2. When the intensity function λ is bounded from grainsintheemulsion. Unfortunately,foragiveninputim-
4
age,wecannotnecessarilyknowhowtheimagewastaken Algorithm 1 Sampling of the inhomogeneous Boolean
andthereforewedonothaveaccesstotheaveragenumber modelfromaninputimage.
ofphotonsreceived. Data: u : {0,1,...,m − 1} × {0,1,...,n − 1} →
Oursolutiontothisproblemistodefineλ(y)sothatthe [0,u ](inputimage)
max
average area covered by the balls of the Boolean model Parameters:
within a pixel location equals the input image gray-level. D(µ ,σ2): distributionofgrainradii
r r
Consequently,theglobalcontrastoftheinputimageiscor- Result:
rectlymaintained. Asignificantadvantageofthisapproach x: Listofgraincenters
isthatcontrastchangescanbehandledindependentlyfrom r: Listofgrainradii
therenderingoffilmgrain.
SampleBooleanmodelwithinthewholeimagedomain:
Therefore, usingEquation(1), wesetλtobethepiece- x←∅,r ←∅
wise constant function defined for all y ∈ [0,m)×[0,n) foreach(i,j) ∈ {0,...,m−1}×{0,...,m−1}do
by
Convertgray-leveltotheinterval[0,1):
1 (cid:18) 1 (cid:19) u˜(i,j)= u(i,j)
λ(y)= E[A ]log 1−u˜((cid:98)y(cid:99)) , (2) Computeluomcaaxl+vaεlueofintensityλ:
1
λ= 1 log 1
π(µ2+σ2) (1−u˜(i,j))
where (cid:98)y(cid:99) are the coordinates of the pixel containing the Draw trhe rnumber of grains Q in the square
point y. This provides us with the means to simulate the [i,i+1)×[j,j+1):
inhomogeneousBooleanmodelforanygiveninputimage. Q←Poisson(λ)
However, this model represents the positive image viewed Samplex from U([i,i+1)×[j,j+1))
i=1...Q
atinfiniteresolution. Inreality,weperceiveimageswithfi- Samplegrainradiir ∼D(µ ,σ2)
i=1...Q r r
niteresolution. Furthermore,wewishtoproduceanoutput Addthenewpointstothelist:
digitalimagedefinedatachosenresolution. Thisrequires x←x∪x ; r ←r∪r
i=1...Q i=1...Q
afilteringstep,whichweexplainfurthernow.
4.3 Filmgrainrenderingalgorithm
4.2 Filteringthepositiveimage
Equation (3) gives us a theoretical model of the continu-
ousphotographicimagev(y)whichweperceive. However,
Asexplainedabovewedonotviewtheanalogimagewith
wealsowishtorenderthisimageatanydesiredresolution,
infinite resolution; indeed if we could, all such pictures
which is a non-trivial task. Accordingly, we now present
wouldbebinary! Inreality,weobserveafilteredandsam-
our film grain rendering algorithm, which consists of the
pled version that reveals the gray-levels of the image. To
twofollowingsteps:
this end, we add a last step which imitates this effect by
filteringandsampling1 . Thisstepisinfactessentialfor
Z
• samplingoftheinhomogeneousBooleanmodel;
viewingagray-levelimageandalsoincreatingthe“grainy”
effect.
• evaluation of the filtered inhomogeneous Boolean
There are several origins of this filtering effect. One of model;
these is the optical filtering which takes place during the
transitionfromnegativetopositive. Asecondunavoidable
4.3.1 Realization/sampling of the inhomogeneous
filtering is that of the human visual system. To illustrate
Booleanmodel
howthisaffectsourmodel,letusdenoteafirstfilterwithψ
actingonthenegativeimage,andanotherfilterψ(cid:48)actingon Firstly, wewishtoproducearealizationoftheinhomoge-
the positive image (which we perceive). We consider that neous Boolean model, in other words we wish to sample
(cid:82)∞ ψ(y)dy = 1,andsimilarlyforψ(cid:48). Thus,theresulting thecentersandradiiofthegrainsthroughouttheimage.
−∞
gray-levelν(y)perceivedatpositionyis Sincetheintensityfunctionλ(2)ofourinhomogeneous
Boolean model is constant on each pixel (represented by
v(y)=(ψ(cid:48)∗[1−ψ∗(1−1 )])(y)=(ψ(cid:48)∗ψ∗1 )(y). unitsquares[i,i+1)×[j,j+1)),thePoissonprocessofthe
Z Z
(3) centersofourinhomogeneousmodelcanbepartitionedinto
Theupshotofthisisthatwecanapplyanysortoffilterwe thedisjointunionofm×nPoissonprocesseshavingtheir
like,independently,andaposteriori,toourBooleanmodel. pointsintheirrespectivepixelsquare[i,i+1)×[j,j+1).
Thisisparticularlypractical,aswecanseparatethecreation Then,withinapixelsquare[i,i+1)×[j,j+1),theinten-
oftheBooleanmodelfromtheapplicationofthefilter. In sity is constant and given by (2) with y = (i,j), and one
our experiments, we simply apply a single Gaussian low- cansimulatethecentersusingthestandardPoissonprocess
passfiltertorepresentthecombinedblurringstepsfromthe simulation. This consists in drawing the number of points
negativeimagetotheperceivedimage. Tosummarize,our QaccordingtoaPoissondistributionwithparameterλ(i,j)
continuousfilmgrainmodelconsistsofafilteredindicator andthendrawingQgraincentersx fromauniformdistri-
i
functionoftheinhomogeneousBooleanmodel. butionU([i,i+1)×[j,j+1))andQindependentradiir
i
5
Continuous film grain model Output image Monte carlo pixel evaluation
( + (=
Sample grains Evaluation of filtered Boolean model
Input image Output image
(Algo. 1) (Algo. 2)
Figure5: Illustrationofthestepsofourfilmgrainrenderingalgorithm.
fromtheradiusdistribution(inpracticeaconstantdistribu- defineascalarN representingthenumberofsamplesinthe
tionoralog-normaldistribution). Thismethodisdescribed Monte Carlo simulation. For each output position y, we
in Algorithm 1. Note that this pseudo-code describes our drawalistofoffsets{ξ ,i = 1...N}whoserow-column
i
algorithm in the case where the grains are balls of possi- coordinates follow a Gaussian distribution N(y,σ2). We
blyrandomsize,butitcanbemodifiedtoincludearbitrary producetheoutputpixelvalueusing
shapes,asweshalldescribeinSection4.4.
N
1 (cid:88)
Algorithm 2 Evaluation of an inhomogeneous Boolean v(y)= N 1Z(ξk). (4)
modelwithMonteCarlosimulation. k=1
Data: x ,r sampledinhomogeneousBoolean
i=1...Q i=1...Q
AsN increases,accordingtothelawoflargenumbers:
model,withatotalofQgrains
Parameters:
s: outputzoom 1 (cid:88)N 1 (ξ )−−−−−→E(1 (ξ ))=(cid:90) 1 (t)φ(y−t)dt,
σ: standarddeviationoftheGaussianlow-passfilter N Z k N→+∞ Z 1 R2 Z
k=1
N : numberofiterationsintheMonteCarlomethod (5)
Result: v: Renderedfilmgrainimage whereφisthepdfoftheGaussiandistributionN(0,σ2I ),
2
thatis,thetargetedGaussianblurkernel. I representsthe
Initializetheoutputimageto0: 2
identitymatrixofsize2×2.
v =0
fork =1toN do We denote with s the zoom factor of the output image,
DrawarandomoffsetfromacenteredGaussiandistri- such that the dimensions of the latter is sn × sm. Thus
butionofvarianceσ2: 1 representstheoutputimagegriddiscretizationstep,with
s
ξ ←N(0,σ2I ) respect to the unit square of the input. In simple terms, s
k 2
for(cid:96)=1toQdo representsthe“zoom”oftheoutputimageresolutionwith
y =sx +ξ respecttotheinputimage. Thepseudo-codefortheMonte
(cid:96) k
foreach (a,b)∈{0,...,sm−1}×{0,...,sn−1} CarlosimulationisshowninAlgorithm2.Weusethesame
s. t. (cid:107)y−(a,b)(cid:107) ≤sr do set of random offsets for each pixel, which avoids the re-
2 (cid:96)
v(a,b)=v(a,b)+1 peated use of random number generation, which can be
slow.
Averagethecontributions:
v = 1v
N
return(v) 4.4 Algorithmicdetailsandparallelization
InSection4.3,wepresentedthefilmgrainrenderingalgo-
rithm in two separate parts: the sampling of the inhomo-
4.3.2 EvaluationoftheinhomogeneousBooleanmodel
geneous Boolean model (Algorithm 1), and the evaluation
Wenowhavealistofgraincentersinacontinuousspace. ofthefilteredmodel(Algorithm2). Adisadvantageofthis
The final step in our algorithm is to evaluate the filtered methodisthatallthegrainpositionsmustbestoredinmem-
inhomogeneous Boolean model. Note that the output dis- oryandthenprocessed. Forexample,ifwesupposeagrain
cretizationgridneednotnecessarilybethesameasthatof radiusr = 1 ,withahighresolutionimage(2048×2048)
40
theinputimage;ouralgorithmcanfreelyzoominorouton withconstantgray-levelvaluesof128everywhere, 35GB
themodelcreatedinSection4.3.1. Thisgivesconsiderable ofmemoryisneededtostorethegrainpositionsandradii,
flexibilitytoouralgorithm. if the information is stored with single precision floating
Now,wecannotactuallyperformthecontinuousconvo- point.
lution described by (3) due to computational limitations. We propose two algorithms which address this problem
However,itispossibletoapproximatetheintegralrequired of storing the grain information. The first generates each
bytheconvolutionusingMonteCarlosimulation. Wefirst grain once only, determines the effect of this grain on the
6
Algorithm3Theproposed“grain-wise”filmgrainrender- Algorithm4Theproposed“pixel-wise”filmgrainrender-
ingalgorithm. Theloopcoloredinblueisparallelized. ingalgorithm. Theloopcoloredinblueisparallelized.
Data: u:{0,1,...m−1}×{0,1,...,n−1}→[0,u ]: Data: u:{0,1,...m−1}×{0,1,...,n−1}→[0,u ]:
max max
inputimage inputimage
Parameters: Parameters:
D(µ ,σ2): distributionofgrainradii D(µ ,σ2): distributionofgrainradii
r r r r
s: outputzoom r : maximumradiusallowed
m
σ: standarddeviationoftheGaussianlow-passfilter s: outputzoom
N: numberofiterationsintheMonteCarlomethod σ: standarddeviationoftheGaussianlow-passfilter
Result: v: Synthesized,filmgrainimage N: numberofiterationsintheMonteCarlomethod
Result: v: Imagerenderedwithfilmgrain
SetupN binaryimagesofsizems×nsanddrawN ran-
domoffsets: δ = 1
(cid:100) 1 (cid:101)
fork =1toN do rm
foreach(i,j) ∈ {0,...,sm−1}×{0,...,sn−1}do
v =0
k
ξ ←N(0,σ2I ) v(i,j)=0
k 2
fork =1toN do
foreach(i,j) ∈ {0,...,m−1}×{0,...,n−1}do ξk ←N(0,σ2I2)
(i ,j )= 1((i,j)+ξ )
u˜(i,j)= u(i,j) g g s k
umax+ε
λ= 1 log 1 Get the list of cells which might contain the balls
π(µ2+σ2) (1−u˜(i,j))
r r
Q←Poisson(λ) covering(i ,j ):
g g
Samplex from U([i,i+1)×[j,j+1))
i=1···N
Samplegrainradiiri=1...Q∼D(µr,σr2) foreach (iδ,jδ) ∈ {(cid:98)ig−δrm(cid:99),...,(cid:98)ig+δrm(cid:99)} ×
fork =1toN do {(cid:98)jg−rm(cid:99),...,(cid:98)jg+rm(cid:99)}do
δ δ
for(cid:96)=1toQdo u˜= u(δ.iδ,δ.jδ)
y =sx +ξ umax+ε
(cid:96) k λ= 1 log 1
foreach (a,b) ∈ {0,...,sm − 1} × π(µ2r+σr2) (1−u˜)
Q←Poisson(λ)
{0,...,sn−1}s. t. (cid:107)y−(a,b)(cid:107) ≤sr do
2 (cid:96)
v (a,b)=1
k
for (cid:96)=1toQdo
x←U([i ,i +1)×[j ,j +1))
δ δ δ δ
y =(δ.i ,δ.j )+δ.s
foreach (i,j)∈{0,...,sm−1}×{0,...,sn−1}do g g
r =min(D(µ ,σ2),r )
v(i,j)=0 r r m
if||y−ξ|| <rthen
fork =1toN do 2
v(i,j)=v(i,j)+1
v(i,j)=v(i,j)+v (i,j)
k
Break: go to next Monte Carlo itera-
v(i,j)= 1 v(i,j) tion
N
return(v)
v(i,j)= 1v(i,j)
N
return(v)
outputimage,andthenerasesthegraininformation.Were-
fertothisasthe“grain-wise”approach(seeAlgorithm3).
Unfortunately, this algorithm is not well-adapted to paral- resultofoneMonteCarloiteration.AMonteCarloiteration
lelization on the GPU, due to excessive memory accesses. consists in evaluating the Boolean model on the randomly
Therefore, we propose a second approach (Algorithm 4) shiftedgridsξk+{0,...,sm−1}×{0,...,sn−1},with
which we refer to as the “pixel-wise” algorithm, which is ξk ∼ N(0,σ2I2). Finally, the result of our algorithm is
suitable for GPU parallelization. We describe these ap- simplytheaverageofallthetemporaryimagesvk. Anad-
proachesnow. vantageofthisapproachisthatthememoryrequirementis
independentofthegrainsize,andonlydependsontheout-
putimagesizeandthenumberofMonteCarloiterationsN.
4.4.1 Grain-wisealgorithm
ThisalgorithmcanbeseeninAlgorithm3.
Aspreviouslymentioned,thisapproachsamplesthegrains
sequentiallyforeachpixel,andevaluatestheeffectofeach
4.4.2 Pixel-wisealgorithm
grain on each Monte Carlo iteration. Once a grain has
beengeneratedandprocessed,itsinformation(positionand The second algorithm we present here, which we refer to
radius) are erased from memory. Instead of saving the asthe“pixel-wise”approach, alsoavoidsstoringthegrain
graininformation,westoreasequenceofN binaryimages information,butinquiteadifferentmannerfromthegrain-
v , k ∈ {1...N}, with each image corresponding to the wiseapproach. Themainreasonforproposinganotheral-
k
7
gorithm is that memory accesses should be limited when of our code, and show our execution times with and with-
using the GPU. Therefore, we cannot save a large num- out this acceleration. It can be seen that, for a fixed con-
ber of intermediate images required by the Monte Carlo stant gray-level, the complexity of our algorithm is linear
simulation of Algorithm 3. Instead, we employ an on- withrespecttothenumberofpixelsintheinputimage. We
the-fly Poisson process generation often used in the pro- observe that it is possible to achieve interactive execution
cedural noise literature [LLC∗10]. This type of approach timeswiththeGPUimplementationofouralgorithminthe
consists in using a grid partition of the space R2 and gen- fixed-radiuscase.
erating the Poisson process on-the-fly within each par- InFigure6,weanalyzetheexecutiontimesofouralgo-
tition cell using a local pseudo-random number genera- rithms when the grain radii are variable. As in the rest of
tor [Wor96,LLDD09]. Given a coordinate pair of a given the paper, the distribution of the radii is a log-normal dis-
partition cell, the pseudo-random number generator can tribution. The standard deviation of the grain radii are set
generatethenumberofgrainswhosecentersbelongtothis toacertainfractiona ∈ [0,1)oftheaveragegrainradius.
cell,aswellasthepositionsofthesecenters. Thenumbers Naturally, the fastest results are achieved with the pixel-
given by the pseudo-random number generator are com- wise algorithm on the GPU. This is several order of mag-
pletelyreproducible, sothereisnoneedtostoretheinfor- nitudes faster than the grain-based approach without par-
mation pertaining to the grains. Note that the cell size δ allelization. Another observation is that, with equal pro-
mustbeafractionoftheinputimagepixelsize. cessing power, the pixel-based approach is preferable to
The main task is to evaluate 1 (y) at y ∈ R2 for each the grain-based one when the grain radii are fixed. How-
Z
Monte Carlo sample. This is equal to 1 if there is a point ever, it becomes slower as the standard deviation of the
x such that y ∈ B(x ,r). Therefore, one only needs to radiiincreases,sinceeachevaluationrequiresthatmoreand
i i
simulate the Poisson process in cells intersecting the ball morecellsbevisited. Therefore,thesetwoalgorithmicap-
ofradiusrcenteredaty toevaluate1 (y). Unfortunately, proaches both have strengths and weaknesses in different
Z
this approach is not valid when using random radii given situations. Nevertheless, theconclusionisthattheconsid-
byadistributionproducingunboundedvariates,suchasthe erableprocessingpoweroftheGPUleadstoafasterpixel-
log-normaldistribution. Inthiscase,wespecifyamaximal wisealgorithm,andsothisimplementationispreferablein
value of the radii, r . Therefore, we need to check more allsituationsapartfromwhenweuseaverylargestandard
m
cells in order to evaluate 1 (y), and this number of cells deviation,whichisnotoftenrequired.
Z
obviouslyincreasesquadraticallywithalinearlyincreasing An important parameter which has a great impact on
maximum radius. This approach is described in detail in the quality of the output is the number of iterations of
thepseudo-codeofAlgorithm4. the Monte Carlo approach N. The trade-off here is obvi-
ously execution time versus accuracy. From Equation 5,
4.5 Performancecomparisons we know that the approach converges to the correct con-
volution. Therefore, we control the standard deviation of
We have extensively tested both the grain-wise and pixel- 1 (cid:80)N 1 (ξ ). This standard deviation is bounded by
N i=1 Z i
wise algorithms in different situations in order to identify (cid:113)
1 . Inourexperiments,wesetN =800,givingastan-
the speedups which are achieved. We have tested the fol- 4∗N
dard deviation of 1.77% around the average, which is an
lowingimplementations:
acceptableerror. Forfasterresults,theparameterN canbe
• Grain-wise,noparallelization; reducedtoaround100,whichgivesanerrorof5%. Thevi-
sualqualitywiththisparameterisstillquitehigh,asshown
• Grain-wise,withparallelization(OpenMP)onamulti-
inthesupplementarymaterial.
coreCPU;
• Pixel-wise,noparallelization;
5 Results
• Pixel-wise,withparallelization(OpenMP)onamulti-
coreCPU; Inthissection,wepresenttheresultsofourfilmgrainren-
deringmethod. Firstly,weillustratetheadvantagesofhav-
• Pixel-wise,withparallelizationonaGPU.
ingamodelforfilmgrainrendering,inparticulartheabil-
The machine used for these tests has four Intel Xeon 2.00 ity to handle any given resolution. We illustrate the influ-
GHz processors, each with ten cores (for the purposes of enceofeachofourmodel’sparametersonthevisualoutput,
parallelizationontheCPU).Thepixel-wisealgorithmwas and show that using these parameters, it is possible to ap-
implemented on a GPU in CUDA using an Nvidia Tesla proximatereal examplesof filmgrain emulsions. We also
T10 graphics card. This GPU implementation is based on clearly demonstrate that independently distributed random
thepubliclyavailablesourcecodeof[GLM16]. variablesareinsufficienttoproducerealisticfilmgrain.
Table 1 shows the execution times for our algorithms.
Fortheseexperiments,werenderedfilmgrainonimagesof 5.1 Filmgrainrenderingresults
increasing sizes, whose gray-level values are equal to 128
everywhere. Wesetthegrainradiustor = 0.05pixel. We InFigures1and7,weshowresultsofourfilmgrainrender-
haveshowntheexecutiontimesoftheparallelizedversions ing on some vintage images which will benefit artistically
8
Imagesize
256×256 512×512 1024×1024 2048×2048
Grain-wise,non-para. 157.25s 632.48s 2581.46s 10170.6s
Grain-wise,para.(CPU) 11.43s 46.15s 188.85s 891.10s
Pixel-wise,non-para. 26.938s 107.78s 429.434s 1717.05s
Pixel-wise,para.(CPU) 1.495s 5.454s 26.0176s 81.4815s
Pixel-wise,para.(GPU) 0.131s 0.475s 1.803s 7.161s
#grainsprocessed 3.58×106 14.3×106 52.3×106 229×106
Table1: Algorithmexecutiontimes. InthisTable,weshowtheexecutiontimesforouralgorithmsfordifferentimagesizes.
Wealsonotethetotalnumberofgrainswhichneedtobeprocessedforeachimage. Theimagesusedareofincreasingsizes,
withaconstantgray-levelof128. Thegrainradiusissettor = 0.05pixelsforallgrains,andweuseN = 800MonteCarlo
samples.
Grain-wise, CPU Pixel-wise, CPU-parallel Grain-wise, CPU Pixel-wise, CPU-parallel
Grain-wise, CPU-parallel Pixel-wise, GPU (parallel) Grain-wise, CPU-parallel Pixel-wise, GPU (parallel)
Pixel-wise, CPU Pixel-wise, CPU
nds) 4Timing of CPU/GPU implementations (average grain radius: 0.05) nds) 4 Timing of CPU/GPU implementations (average grain radius: 0.1)
o o
ec 3.5 ec 3.5
me (in s 2. 53 me (in s 2. 53
n ti 2 n ti 2
o o
uti 1.5 uti 1.5
xec 1 xec 1
e e
of 0.5 of 0.5
0 0
g1 0 g1 0
Lo 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 Lo 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Standard deviation of grain radius, as a fraction of the mean radius Standard deviation of grain radius, as a fraction of the mean radius
Figure 6: Execution times of the proposed algorithms with increasing radius standard deviation. The “grain-wise”
approach is implemented with and without parallelization on the CPU, and the “pixel-wise” approach is implemented with
andwithoutparallelizationontheCPU,andwithparallelizationontheGPU.Thesizeoftheimageusedis1024x1024,with
aconstantgrey-levelof128.
9