Big Data Analytics: Optimization and Randomization

315 Pages·2015·8.17 MB·English

Big Data Analytics: Optimization
and Randomization
Tianbao Yang†, Qihang Lin\, Rong Jin∗‡
Tutorial@SIGKDD 2015
Sydney, Australia
†DepartmentofComputerScience,TheUniversityofIowa,IA,USA
\DepartmentofManagementSciences,TheUniversityofIowa,IA,USA
∗DepartmentofComputerScienceandEngineering,MichiganStateUniversity,MI,USA
‡InstituteofDataScienceandTechnologiesatAlibabaGroup,Seattle,USA
August 10, 2015
Yang,Lin,Jin TutorialforKDD’15 August10,2015 1/234
URL
http://www.cs.uiowa.edu/˜tyng/kdd15-tutorial.pdf
Yang,Lin,Jin TutorialforKDD’15 August10,2015 2/234
Some Claims
No
This tutorial is not an exhaustive literature survey
It is not a survey on different machine learning/data mining
algorithms
Yes
It is about how to efficiently solve machine learning/data mining
(formulated as optimization) problems for big data
Yang,Lin,Jin TutorialforKDD’15 August10,2015 3/234
Outline
Part I: Basics
Part II: Optimization
Part III: Randomization
Yang,Lin,Jin TutorialforKDD’15 August10,2015 4/234
Big Data Analytics: Optimization and Randomization
Part I: Basics
Yang,Lin,Jin TutorialforKDD’15 August10,2015 5/234
Basics Introduction
Outline
1 Basics
Introduction
Notations and Definitions
Yang,Lin,Jin TutorialforKDD’15 August10,2015 6/234
Basics Introduction
Three Steps for Machine Learning
distance to optimal objective000000...012...123555 011.//5TTT2
0 20 40 60 80 100
iterations
Data Model Optimization
Yang,Lin,Jin TutorialforKDD’15 August10,2015 7/234
Basics Introduction
Big Data Challenge
Big Data
Yang,Lin,Jin TutorialforKDD’15 August10,2015 8/234
Basics Introduction
Big Data Challenge
Big Model
60 million parameters
Yang,Lin,Jin TutorialforKDD’15 August10,2015 9/234
Basics Introduction
Learning as Optimization
Ridge Regression Problem:
min 1 Xn (y −w>x )2+ λkwk2
w∈Rd n i i 2 2
i=1
x ∈ Rd: d-dimensional feature vector
i
y ∈ R: target variable
i
w ∈ Rd: model parameters
n: number of data points
Yang,Lin,Jin TutorialforKDD’15 August10,2015 10/234