Training Generative Adversarial Networks with Binary Neurons by End-to-end Backpropagation PDF
Preview Training Generative Adversarial Networks with Binary Neurons by End-to-end Backpropagation
T G A N RAINING ENERATIVE DVERSARIAL ETWORKS WITH B N E - - B INARY EURONS BY ND TO END ACKPROPAGATION Hao-Wen Dong and Yi-Hsuan Yang O UTLINES Backgrounds Generative Adversarial Networks Binary Neurons Straight-through Estimators BinaryGAN Experiments & Results Discussions & Conclusion 2 B ACKGROUNDS G A N ENERATIVE DVERSARIAL ETWORKS Goal—learn a mapping from the prior distribution to the data distribution [2] prior data distribution distribution 4 G A N ENERATIVE DVERSARIAL ETWORKS Goal—learn a mapping from the prior distribution to the data distribution [2] can be intractable prior data distribution distribution 5 G A N ENERATIVE DVERSARIAL ETWORKS Use a deep neural network to learn an implicit mapping generator prior model distribution distribution data distribution 6 G A N ENERATIVE DVERSARIAL ETWORKS Use another deep neural network to provide guidance/critics generator discriminator prior model distribution distribution real/fake data distribution 7 B N INARY EURONS Definition: neurons that output binary-valued predictions 8 B N INARY EURONS Definition: neurons that output binary-valued predictions Deterministic binary neurons (DBNs): (hard thresholding) 𝟏𝟏, 𝒊𝒊𝒊𝒊 𝝈𝝈 𝒙𝒙 > 𝟎𝟎. 𝟓𝟓 𝑫𝑫𝑫𝑫𝑫𝑫 𝒙𝒙 ≡ � 𝟎𝟎, 𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒊𝒊𝒐𝒐𝒐𝒐 9 B N INARY EURONS Definition: neurons that output binary-valued predictions Deterministic binary neurons (DBNs): (hard thresholding) 𝟏𝟏, 𝒊𝒊𝒊𝒊 𝝈𝝈 𝒙𝒙 > 𝟎𝟎. 𝟓𝟓 𝑫𝑫𝑫𝑫𝑫𝑫 𝒙𝒙 ≡ � 𝟎𝟎, 𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒊𝒊𝒐𝒐𝒐𝒐 Stochastic binary neurons (SBNs): (Bernoulli sampling) 𝟏𝟏, 𝒊𝒊𝒊𝒊 𝒛𝒛 < 𝝈𝝈 𝒙𝒙 𝑺𝑺𝑫𝑫𝑫𝑫 𝒙𝒙 ≡ � , 𝒛𝒛~𝑼𝑼 𝟎𝟎, 𝟏𝟏 𝟎𝟎, 𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒐𝒊𝒊𝒐𝒐𝒐𝒐 10
Description:The list of books you might like
