Gaussian Mixtures Vol.2 - Introduction to - Alexandre Devert

32 Pages·2012·2.1 MB·English

by  Devert Alexandre

UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC
Gaussian Mixtures Vol.2
Introduction to probabilistic models
Devert Alexandre
SchoolofSoftwareEngineeringofUSTC
December13,2012—Slide1/29
UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC
Table of Contents
Multivariate Gaussian distribution
1
Multivariate mixtures
2
Definition
Parameter estimation
EM clustering
3
Clustering
EM clustering vs. kmeans
DevertAlexandre(SchoolofSoftwareEngineeringofUSTC)—GaussianMixturesVol.2—Slide2/29
UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC
Multivariate Gaussian distribution
The multivariate Gaussian distribution is N dimensions
equivalent of a Gaussian distribution
DevertAlexandre(SchoolofSoftwareEngineeringofUSTC)—GaussianMixturesVol.2—Slide3/29
UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC
Multivariate Gaussian distribution
To build a multivariate Gaussian distribution, step 1
0.40
0.35
0.30
0.25
0.20
0.15
0.10
0.05
0.006 4 2 0 2 4 6
Take a single variate distribution with µ = 0 and σ = 1
DevertAlexandre(SchoolofSoftwareEngineeringofUSTC)—GaussianMixturesVol.2—Slide4/29
UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC
Multivariate Gaussian distribution
To build a multivariate Gaussian distribution, step 2
probability density samples
Pull it to N dimension with a central symmetry
DevertAlexandre(SchoolofSoftwareEngineeringofUSTC)—GaussianMixturesVol.2—Slide5/29
UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC
Multivariate Gaussian distribution
To build a multivariate Gaussian distribution, step 3
probability density samples
Scale its along the axes
DevertAlexandre(SchoolofSoftwareEngineeringofUSTC)—GaussianMixturesVol.2—Slide6/29
UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC
Multivariate Gaussian distribution
To build a multivariate Gaussian distribution, step 4
probability density samples
Rotate it
DevertAlexandre(SchoolofSoftwareEngineeringofUSTC)—GaussianMixturesVol.2—Slide7/29
UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC
Multivariate Gaussian distribution
To build a multivariate Gaussian distribution, step 5
probability density samples
Translate it
DevertAlexandre(SchoolofSoftwareEngineeringofUSTC)—GaussianMixturesVol.2—Slide8/29
UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC
Probability density
Probability density for a multivariate Gaussian
distribution in dimension n
1
Φ(x) = e−1(x−µ)TΣ−1(x−µ)
2
(2π)k|Σ|1
2 2
• µ is a vector ⇒ the mean of the distribution
• σ is a positive definite matrix ⇒ the covariance of
the the distribution
DevertAlexandre(SchoolofSoftwareEngineeringofUSTC)—GaussianMixturesVol.2—Slide9/29
UNIVERSITY OF SCIENCE AND TECHNOLOGY OF CHINA SCHOOL OF SOFTWARE ENGINEERING OF USTC
Parameters estimations
The sample mean for a multivariate Gaussian distribution
n
1 (cid:88)
X¯ = X
i
n
i=1
You can compute it as the mean of the samples
coordinates per coordinates, as introduced in the
univariate case.
DevertAlexandre(SchoolofSoftwareEngineeringofUSTC)—GaussianMixturesVol.2—Slide10/29