Photo Editing With Generative Adversarial Networks (GANs)

25 Pages·2017·1.31 MB·English

by  Jennifer Hohn

Photo Editing With Generative
Adversarial Networks (GANs)
GTC, May 2017. Greg Heinrich.
PANDARUS: Alas, I think he shall be come approached and the day When little srain would be attain'd into being never fed, And who is but a chain and subjects of his death, I should not sleep.
G : WHAT IS A GENERATIVE MODEL?
AN
In Machine Learning
A generative model learns to generate samples that have the same
characteristics as the samples in the dataset.

Learn from Shakespeare novels: Produce:
http://karpathy.github.io/2015/05/21/rnn- PANDARUS:
effectiveness/
Alas, I think he shall be come approached
and the day
When little srain would be attain'd into
being never fed,
And who is but a chain and subjects of his
death,
I should not sleep. 22
BASIC REMINDER: BACKPROP
𝜕𝐸
Calculating iteratively
𝑙
𝜕𝑤
𝑖𝑗 Chain rule
𝜕𝐸
Calculation of :
Output of each neuron 𝑗 of layer 𝑙 :
𝑙
𝜕𝑧
𝑗
𝑙 𝑙 𝑙 𝑙−1 𝑙
ℎ = 𝜑 𝑧 = 𝜑 𝑤 ℎ + 𝑏
𝑗 𝑗 𝑖𝑗 𝑖 𝑗 𝜕𝐸 𝜕𝐸 𝜕𝑧𝑙+1 𝜕𝐸 𝜕𝑧𝑙+1 𝜕ℎ𝑙
= 𝑘 = 𝑘 𝑗
𝑖
𝑙 𝑘 𝑙+1 𝑙 𝑘 𝑙+1 𝑙 𝑙
𝜕𝑧 𝜕𝑧 𝜕𝑧 𝜕𝑧 𝜕ℎ 𝜕𝑧
𝑗 𝑘 𝑗 𝑘 𝑗 𝑗
Gradient of E with respect to each weight:
𝜕𝐸
𝑙+1 𝑙
= 𝑤 𝜑′ 𝑧
𝜕𝐸 𝜕𝐸 𝜕𝑧𝑙 𝜕𝐸 𝜕𝑧𝑙+1 𝑗𝑘 𝑗
𝑗 𝑙−1 𝑘
= = ℎ 𝑘
𝑙 𝑙 𝑙 𝑙 𝑖
𝜕𝑤 𝜕𝑧 𝜕𝑤 𝜕𝑧
𝜕𝐸
𝑖𝑗 𝑗 𝑖𝑗 𝑗
𝑙 𝑙+1
= 𝜑′ 𝑧 𝑤
Calculated 𝑗 𝑙+1 𝑗𝑘
𝜕𝑧
during 𝑘
𝑘
forward prop
Multivariate
chain rule
𝑙
Chain 𝜕𝑧
𝑗 only depends on ℎ𝑙−1
rule 𝜕𝑤𝑙 𝑖 33
𝑖𝑗
A : PLAYING THE ADVERSARIAL GAME
G N
Learning on a corpus of images
Let’s play a game opposing two agents:
- The Generator, a little imp in the computer who paints images.
- The Discriminator: you are collectively responsible for playing the
Discriminator.
The game master (me) randomly picks images from either the corpus or
the Generator and shows them to the Discriminator. The goal of the
Discriminator is to identify the source of the images: real (from the
corpus) or fake (painted by the little imp). The goal of the Generator is
to fool the Discriminator.
44
PLAYING THE ADVERSARIAL GAME
Is this a veelhoek* from our corpus?
Note: you don’t have to * veelhoek is the articulation
know what a veelhoek is, of a ubiquitous item in the
you will learn through
language of a tiny country in
examples!
Europe that is well known for
the inferior quality of its
cheese.

Yes, this red square is a veelhoek!
55
PLAYING THE ADVERSARIAL GAME
Is this a veelhoek from our corpus?

No, those squiggly lines aren’t right!
66
PLAYING THE ADVERSARIAL GAME
Is this a veelhoek from our corpus?

Yes, even though it’s blue and tiny!
77
PLAYING THE ADVERSARIAL GAME
Is this a veelhoek from our corpus?

No, those rounded corners are a giveaway!
88
PLAYING THE ADVERSARIAL GAME
Is this a veelhoek from our corpus?

No, but it’s a very good fake!
99
PLAYING THE ADVERSARIAL GAME
Is this a veelhoek from our corpus?

No, it’s the same fake as before!
1100