General Linear Model and 1-Way ANOVA. - University of Illinois at

119 Pages·2007·1.03 MB·English

The General Linear Model & ANOVA
Edpsy 580
Carolyn J. Anderson
Department of Educational Psychology
I L L I N O I S
UNIVERSITY OF ILLINOIS AT URBANA-CHAMPAIGN
MultivariateRelationshipsandMultipleLinearRegression Slide1of119
Outline
n Introduction
u What is it the General Linear Model.
TheGeneralLinearModel
ExplanatoryVariables
u Explanatory variables.
1–WayAnalysisofVariance
u Couple (quick) examples.
What&WhysofANOVA
Designs
ANOVATerminologyand
n One-Factor ANOVA (fixed effects model)
Notation
LeastSquaresEstimation
u Introduction.
PartitioningSumsofSquares
HypothesisTesting: F-test u As a linear model.
ANOVA&SAS
u Hypothesis testing.
UnequalSampleSizes
EffectSize u Example.
Power
ViolationsofAssumptions
n More Examples
MultivariateRelationshipsandMultipleLinearRegression Slide2of119
The General Linear Model
n A General & unifying framework.
TheGeneralLinearModel
u Simple linear and multiple regression.
lTheGeneralLinearModel
lTheGeneralLinearModel
lError,ǫi u Analysis of Variance (ANOVA).
l“Linearintheparameters”
lTheGeneralLinearModel
u Analysis of Covariance (ANCOVA).
ExplanatoryVariables
1–WayAnalysisofVariance
u Other experimental designs.
What&WhysofANOVA
Designs
n Can be extended to
ANOVATerminologyand
Notation
u Generalized Linear model.
LeastSquaresEstimation
PartitioningSumsofSquares
u Multivariate general linear model.
HypothesisTesting: F-test
ANOVA&SAS u Random coefficients linear models.
UnequalSampleSizes
u Random coefficients generalized linear models.
EffectSize
Power
ViolationsofAssumptions
MultivariateRelationshipsandMultipleLinearRegression Slide3of119
The General Linear Model
n Basic Linear form:
TheGeneralLinearModel
lTheGeneralLinearModel Y = β x + β x + β x + . . . + ǫ
i o io 1 i1 2 i2 i
lTheGeneralLinearModel
lError,ǫi
l“Linearintheparameters” n Fixed:
lTheGeneralLinearModel
ExplanatoryVariables u x , x , x , . . . are values of the explanatory (predictor,
io i1 i2
1–WayAnalysisofVariance independent) variables for individual i
What&WhysofANOVA
u β , β , β , . . . are population parameters
o 1 2
Designs
ANOVATerminologyand
Notation
n Random:
LeastSquaresEstimation
PartitioningSumsofSquares u Y is quantitative or numerical response (outcome,
i
HypothesisTesting: F-test dependent) variable for individual i.
ANOVA&SAS
u ǫ is “error” for individual i.
UnequalSampleSizes i
EffectSize
Power
ViolationsofAssumptions
MultivariateRelationshipsandMultipleLinearRegression Slide4of119
ǫ
Error,
i
n Y is random because ǫ is random.
i i
n Standard assumption:
TheGeneralLinearModel
lTheGeneralLinearModel
lTheGeneralLinearModel E(ǫ ) = 0 and var(ǫ ) = σ2
lError,ǫi i i ǫ
l“Linearintheparameters”
lTheGeneralLinearModel and for statistical inference ǫ is normal.
i
ExplanatoryVariables
1–WayAnalysisofVariance
n Sources of Variability—
What&WhysofANOVA
ǫ consists of effects due to
i
Designs
ANOVATerminologyand u Sampling.
Notation
LeastSquaresEstimation
u Measurement imperfections.
PartitioningSumsofSquares
u Individual differences.
HypothesisTesting: F-test
ANOVA&SAS
u Uncontrolled variability.
UnequalSampleSizes
EffectSize u Unsystematic error.
Power
ViolationsofAssumptions
MultivariateRelationshipsandMultipleLinearRegression Slide5of119
“Linear in the parameters”
Linear or non-linear?
Y = β + β x + ǫ
TheGeneralLinearModel i o 1 i1 i
lTheGeneralLinearModel
lTheGeneralLinearModel
lError,ǫi
l“Linearintheparameters”
lTheGeneralLinearModel 2
Y = β x + β x + β x + ǫ
i o io 1 i1 2 i2 i
ExplanatoryVariables
1–WayAnalysisofVariance
What&WhysofANOVA
Y = β + β log(x ) + ǫ
i o 1 i1 i
Designs
ANOVATerminologyand
Notation
Y = e(βo+β1xi1+β2xi2+ǫi)
LeastSquaresEstimation i
PartitioningSumsofSquares
HypothesisTesting: F-test
Y = β + xβ1 + ǫ
ANOVA&SAS i o i1 i
UnequalSampleSizes
EffectSize
Power
ViolationsofAssumptions
MultivariateRelationshipsandMultipleLinearRegression Slide6of119
The General Linear Model
n “Smoothes” the data.
TheGeneralLinearModel
lTheGeneralLinearModel
lTheGeneralLinearModel
lError,ǫi
l“Linearintheparameters” n Summary, description.
lTheGeneralLinearModel
ExplanatoryVariables
1–WayAnalysisofVariance n Prediction.
What&WhysofANOVA
Designs
n Better (smaller) standard errors for means.
ANOVATerminologyand
Notation
LeastSquaresEstimation
n Hypothesis testing.
PartitioningSumsofSquares
HypothesisTesting: F-test
ANOVA&SAS
UnequalSampleSizes
EffectSize
Power
ViolationsofAssumptions
MultivariateRelationshipsandMultipleLinearRegression Slide7of119
The Explanatory Variables
n Quantitative, e.g.
TheGeneralLinearModel
u Age.
ExplanatoryVariables
lTheExplanatoryVariables
u Grade.
lQuantitativeExplanatory
Variables
lQualitativeExplanatory
Variable:hotdogs u Pre-test score.
lGLMforHotDogs
lHotDogswithAlternative
Coding
lGLMforHotDogs
n Qualitative, e.g.
lExample2ofQualitative
Variable
lAlternativeCoding
u Season (winter, spring, summer, fall).
1–WayAnalysisofVariance
What&WhysofANOVA u Teaching materials (text, web, or both).
Designs
u Statistics text (standard, low explanation, high
ANOVATerminologyand
Notation explanation).
LeastSquaresEstimation
u Type of writing (narrative, summary, argument).
PartitioningSumsofSquares
HypothesisTesting: F-test
ANOVA&SAS
UnequalSampleSizes
MultivariateRelationshipsandMultipleLinearRegression Slide8of119
EffectSize
Quantitative Explanatory Variables
y = β + β x + ǫ yˆ = 1.2 + 5.4x
i o 1 i i i i
−→
TheGeneralLinearModel
ExplanatoryVariables
lTheExplanatoryVariables
lQuantitativeExplanatory
Variables
lQualitativeExplanatory
Variable:hotdogs
lGLMforHotDogs
lHotDogswithAlternative
Coding
lGLMforHotDogs
lExample2ofQualitative
Variable
lAlternativeCoding
1–WayAnalysisofVariance
What&WhysofANOVA
Designs
ANOVATerminologyand
Notation
LeastSquaresEstimation
PartitioningSumsofSquares
HypothesisTesting: F-test
ANOVA&SAS
UnequalSampleSizes
MultivariateRelationshipsandMultipleLinearRegression Slide9of119
EffectSize
Qualitative Explanatory Variable: hot dogs
Hot dog eaters who are also concerned with their health may
prefer hot dogs that are lower in calories (and salt). The data
used in this example consist of calories contained in each of 54
TheGeneralLinearModel major hot dog brands. The hot dogs are classified by type:
ExplanatoryVariables
lTheExplanatoryVariables n Beef
lQuantitativeExplanatory
Variables
lQualitativeExplanatory n Meat (mostly pork and beef, but up to 15% poultry meat)
Variable:hotdogs
lGLMforHotDogs n Poultry
lHotDogswithAlternative
Coding Data are from Consumers Reports, June 1986, pp. 366-367.
lGLMforHotDogs
lExample2ofQualitative Summary statistics:
Variable
lAlternativeCoding
Type n Sum Mean Variance Std Dev
1–WayAnalysisofVariance
Beef 20 3137.00 156.85 512.66 22.64
What&WhysofANOVA
Meat 17 2698.00 158.71 636.85 25.24
Designs
Poultry 17 2019.00 118.76 508.57 22.55
ANOVATerminologyand
Notation
Total 54 7854.00 145.44 863.38 29.38
LeastSquaresEstimation
PartitioningSumsofSquares
Do different types of hot dogs differ in terms of calories?
HypothesisTesting: F-test
ANOVA&SAS
UnequalSampleSizes
MultivariateRelationshipsandMultipleLinearRegression Slide10of119
EffectSize