Smoothing Spline ANOVA Models

175 Pages·2012·2.18 MB·English

Smoothing Spline ANOVA Models
Chong Gu
DepartmentofStatistics
PurdueUniversity
June 21, 2012
ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 1/45
Outline
1 Introduction
Cubic Spline and Penalized Likelihood
Functional ANOVA Decomposition
R Package gss
2 Estimation and Inference
Splines as Bayes Estimates
Efficient Approximation
Cross-Validation
Bayesian Confidence Intervals
Kullback-Leibler Projection
3 Regression Models
Non-Gaussian Regression
Regression with Correlated Data
4 Density and Hazard Estimation
Density and Conditional Density
Hazard and Relative Risk
ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 2/45
Outline
1 Introduction
Cubic Spline and Penalized Likelihood
Functional ANOVA Decomposition
R Package gss
2 Estimation and Inference
Splines as Bayes Estimates
Efficient Approximation
Cross-Validation
Bayesian Confidence Intervals
Kullback-Leibler Projection
3 Regression Models
Non-Gaussian Regression
Regression with Correlated Data
4 Density and Hazard Estimation
Density and Conditional Density
Hazard and Relative Risk
ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 2/45
Outline
1 Introduction
Cubic Spline and Penalized Likelihood
Functional ANOVA Decomposition
R Package gss
2 Estimation and Inference
Splines as Bayes Estimates
Efficient Approximation
Cross-Validation
Bayesian Confidence Intervals
Kullback-Leibler Projection
3 Regression Models
Non-Gaussian Regression
Regression with Correlated Data
4 Density and Hazard Estimation
Density and Conditional Density
Hazard and Relative Risk
ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 2/45
Outline
1 Introduction
Cubic Spline and Penalized Likelihood
Functional ANOVA Decomposition
R Package gss
2 Estimation and Inference
Splines as Bayes Estimates
Efficient Approximation
Cross-Validation
Bayesian Confidence Intervals
Kullback-Leibler Projection
3 Regression Models
Non-Gaussian Regression
Regression with Correlated Data
4 Density and Hazard Estimation
Density and Conditional Density
Hazard and Relative Risk
ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 2/45
Outline
1 Introduction
Cubic Spline and Penalized Likelihood
Functional ANOVA Decomposition
R Package gss
2 Estimation and Inference
Splines as Bayes Estimates
Efficient Approximation
Cross-Validation
Bayesian Confidence Intervals
Kullback-Leibler Projection
3 Regression Models
Non-Gaussian Regression
Regression with Correlated Data
4 Density and Hazard Estimation
Density and Conditional Density
Hazard and Relative Risk
ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 3/45
Cubic Smoothing Spline
◮ Problem: Observing Y = η(x )+ǫ , i = 1,...,n, where x [0,1]
i i i i
∈
and ǫ N(0,σ2), one is to estimate η(x).
i
∼
◮ Method: Minimize n1 Pni=1(cid:0)Yi −η(xi)(cid:1)2+λR01(cid:0)η′′(x)(cid:1)2dx.
◮ Solution: Piecewise cubic polynomial, with η(3)(x) jumping at knots
ξ < ξ < < ξ (ordered distinctive x ); linear on [0,ξ ] [ξ ,1].
1 2 q i 1 q
··· ∪
◮ Smoothing Parameter: As λ , η(x) gets smoother; at λ = 0 , one
+
↑
interpolates data, at λ = , η(x) = β +β x.
0 1
∞
ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 4/45
Cubic Smoothing Spline
◮ Problem: Observing Y = η(x )+ǫ , i = 1,...,n, where x [0,1]
i i i i
∈
and ǫ N(0,σ2), one is to estimate η(x).
i
∼
◮ Method: Minimize n1 Pni=1(cid:0)Yi −η(xi)(cid:1)2+λR01(cid:0)η′′(x)(cid:1)2dx.
◮ Solution: Piecewise cubic polynomial, with η(3)(x) jumping at knots
ξ < ξ < < ξ (ordered distinctive x ); linear on [0,ξ ] [ξ ,1].
1 2 q i 1 q
··· ∪
◮ Smoothing Parameter: As λ , η(x) gets smoother; at λ = 0 , one
+
↑
interpolates data, at λ = , η(x) = β +β x.
0 1
∞
ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 4/45
Cubic Smoothing Spline
◮ Problem: Observing Y = η(x )+ǫ , i = 1,...,n, where x [0,1]
i i i i
∈
and ǫ N(0,σ2), one is to estimate η(x).
i
∼
◮ Method: Minimize n1 Pni=1(cid:0)Yi −η(xi)(cid:1)2+λR01(cid:0)η′′(x)(cid:1)2dx.
◮ Solution: Piecewise cubic polynomial, with η(3)(x) jumping at knots
ξ < ξ < < ξ (ordered distinctive x ); linear on [0,ξ ] [ξ ,1].
1 2 q i 1 q
··· ∪
◮ Smoothing Parameter: As λ , η(x) gets smoother; at λ = 0 , one
+
↑
interpolates data, at λ = , η(x) = β +β x.
0 1
∞
ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 4/45
Cubic Smoothing Spline
◮ Problem: Observing Y = η(x )+ǫ , i = 1,...,n, where x [0,1]
i i i i
∈
and ǫ N(0,σ2), one is to estimate η(x).
i
∼
◮ Method: Minimize n1 Pni=1(cid:0)Yi −η(xi)(cid:1)2+λR01(cid:0)η′′(x)(cid:1)2dx.
◮ Solution: Piecewise cubic polynomial, with η(3)(x) jumping at knots
ξ < ξ < < ξ (ordered distinctive x ); linear on [0,ξ ] [ξ ,1].
1 2 q i 1 q
··· ∪
◮ Smoothing Parameter: As λ , η(x) gets smoother; at λ = 0 , one
+
↑
interpolates data, at λ = , η(x) = β +β x.
0 1
∞
ChongGu (PurdueUniversity) SmoothingSplineANOVAModels June21,2012 4/45