ebook img

A Statistical Machine Learning Perspective of Deep Learning PDF

286 Pages·2017·21.51 MB·English
by  
Save to my drive
Quick download
Download
Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.

Preview A Statistical Machine Learning Perspective of Deep Learning

A Statistical Machine Learning Perspective of Deep Learning: Algorithm, Theory, Scalable Computing Maruan Al-Shedivat, Zhiting Hu, Hao Zhang, and Eric Xing Petuum Inc & Carnegie Mellon University Element of AI/Machine Learning Task Model • Graphical Models • Large-Margin • Deep Learning • Sparse Coding • Nonparametric • Regularized • Spectral/Matrix • Sparse Structured Bayesian Models Bayesian Methods Methods I/O Regression Algorithm • Stochastic Gradient • Coordinate • L-BFGS • Gibbs Sampling • Metropolis- Descent / Back Descent Hastings propagation Implementation … • Mahout • Mllib • CNTK • MxNet • Tensorflow (MapReduce) (BSP) (Async) System Hadoop Spark MPI RPC GraphLab … • Network • Network attached • Server machines • RAM • Cloud compute Platform • Virtual switches storage • Desktops/Laptops • Flash (e.g. Amazon EC2) machines and Hardware • Infiniband • Flash storage • NUMA machines • SSD • IoT networks • Mobile devices • Data centers • GPUs, CPUs, FPGA, TPU • ARM-powered devices © Petuum,Inc. 1 ML vs DL © Petuum,Inc. 2 Plan • Statistical And Algorithmic Foundation and Insight of Deep Learning • On Unified Framework of Deep Generative Models • Computational Mechanisms: Distributed Deep Learning Architectures © Petuum,Inc. 3 Part-I Basics Outline • Probabilistic Graphical Models: Basics • An overview of DL components • Historical remarks: early days of neural networks • Modern building blocks: units, layers, activations functions, loss functions, etc. • Reverse-mode automatic differentiation (aka backpropagation) • Similarities and differences between GMs and NNs • Graphical models vs. computational graphs • Sigmoid Belief Networks as graphical models • Deep Belief Networks and Boltzmann Machines • Combining DL methods and GMs • Using outputs of NNs as inputs to GMs • GMs with potential functions represented by NNs • NNs with structured outputs • Bayesian Learning of NNs • Bayesian learning of NN parameters • Deep kernel learning © Petuum,Inc. 5 Outline • Probabilistic Graphical Models: Basics • An overview of DL components • Historical remarks: early days of neural networks • Modern building blocks: units, layers, activations functions, loss functions, etc. • Reverse-mode automatic differentiation (aka backpropagation) • Similarities and differences between GMs and NNs • Graphical models vs. computational graphs • Sigmoid Belief Networks as graphical models • Deep Belief Networks and Boltzmann Machines • Combining DL methods and GMs • Using outputs of NNs as inputs to GMs • GMs with potential functions represented by NNs • NNs with structured outputs • Bayesian Learning of NNs • Bayesian learning of NN parameters • Deep kernel learning © Petuum,Inc. 6 Fundamental questions of probabilistic modeling • Representation: what is the joint probability distr. on multiple variables? !(# , # , # , … , # ) $ & ' ) • How many state configurations are there? • Do they all need to be represented? • Can we incorporate any domain-specific insights into the representation? • Learning: where do we get the probabilities from? • Maximum likelihood estimation? How much data do we need? • Are there any other established principles? • Inference: if not all variables are observable, how to compute the conditional distribution of latent variables given evidence? • Computing !(+|-) would require summing over 2/ configurations of the unobserved variables © Petuum,Inc. 7 What is a graphical model? • A possible world of cellular signal transduction © Petuum,Inc. 8 GM: structure simplifies representation • A possible world of cellular signal transduction © Petuum,Inc. 9

See more

The list of books you might like

Most books are stored in the elastic cloud where traffic is expensive. For this reason, we have a limit on daily download.