ERIC ED563467: Exploring the Variability in the Validity of SAT Scores and High School GPA for Predicting First-Year College Grades at Different Colleges and Universities

2010·0.08 MB·English

by  ERIC

Exploring the Variability
in the Validityy of SAT
Scores and High School
GPA for Predicting First-
Year College Grades at
Different Colleges and
UUnniivveerrssiittiieess
Jennifer L. Kobrin & Brian F. Patterson
The College Board
Background
The SAT is one of the most researched tests, with
•
hhuunnddrreeddss ooff ppuubblliisshheedd vvaalliiddiittyy ssttuuddiieess.
Vast majority of studies including samples at
•
mullttiiplle iinsttiittuttiions examiine tthhe rellattiionshhiip off
SAT scores and HSGPA with college
ppeerrffoorrmmaannccee iinn aaggggrreeggaattee.
Studies show that the relationship of SAT scores
•
aanndd HHSSGGPPAA wwiitthh FFYYGGPPAA vvaarriieess aatt ddiiffffeerreenntt
institutions/different types of institutions.
2
Prior Research Exploring Variations in
SAT Predictive Validityy
Baird (1983) used college characteristics as predictor
•
variables to predict the size of the simple and multiple
correlations of SAT and HSGPA with FYGPA (validity
coefficients).
AA ffew sttuddiies hhave examiinedd tthhe valliiddiitty off tthhe SSAATT usiing
•
a multilevel view:
BBrroowwnn aanndd ZZwwiicckk ((22000066)) –– VVaalliiddiittyy ooff SSAATT aanndd SSAATT SSuubbjjeecctt TTeessttss
••
for predicting FYGPA at Univ. of CA.
Culpepper and Davenport (2009) – Variability in differential
•
preddiicttiion off FFYYGGPPAA usiing SSAATT andd HHSSGGPPAA bby raciiall//etthhniic
group.
Shen et al. (2010) – created factors of institutional characteristics
•
and used these to predict variability in SAT validity after taking
account of statistical artifacts.
3
Purpose of the Current Study
To demonstrate the utility of a
multilevel model to understand the
rreellaattiioonnsshhiipp ooff iinnssttiittuuttiioonnaall
characteristics to the validity of the
SSAATT andd HHSSGGPPAA ffor preddiicttiing ffiirstt-
yyear collegge GPA ((FYGPA)).
4
Method
Sample consisted of about 150,000 students from 109
•
ccoolllleeggeess aanndd uunniivveerrssiittiieess aaccrroossss tthhee UU..SS..
SAT scores obtained from 2006 college-bound seniors
•
cohort; HSGPA self-repported on SAT Questionnaire.
First phase replicated Baird (1983) to identify variables to
•
use in the multilevel modeling.
(Results from this phase will not be presented today)
Second pphase used Hierarchical Linear Modelingg ((HLM))
•
to model the variability in the relationship of SAT scores
and HSGPA with FYGPA across the 109 institutions.
5
HLM Specifics
Analyses followed a step-wise approach,
•
begginningg with null model ((one-wayy random
effects ANOVA)
FFuullll mmaaxxiimmuumm lliikkeelliihhoooodd eessttiimmaattiioonn uusseedd iinn aallll
••
models.
RRaannddoomm eeffffeeccttss iinncclluuddeedd ffoorr aallll ssttuuddeenntt-lleevveell
•
predictors.
SSttuddentt-llevell preddiicttors ((SSAATT, HHSSGGPPAA)) were
•
centered within institution (group-mean centered)
aanndd iinnssttiittuuttiioonn-lleevveell pprreeddiiccttoorrss wweerree ggrraanndd-mmeeaann
centered.
6
Model-Building Summary
Model 1 – Null Model (ICC = 0.109)
•
Model 2 – added student-level predictors
•
(SAT/1000 and HSGPA)
Model 3 – added average SAT/1000 and average
•
HSGPA as Level 2 predictors.
Models 4 and 5 – added Level 2 predictors one at
•
a time accordingg to apppproximate coefficients and
t-values estimated by the HLM program.
7
Final Model
Level 1:
FFYYGGPPAA = ββ + ββ **((HHSSGGPPAA)) + ββ **((SSAATT ttottall//11000000)) + r .
ij 0j 1j ij 2j ij ij
Level 2:
β = γ + γ (average SAT) + γ (average HSGPA) + γ
0j 00 01 j 02 j 03
(private) + γ ( small) + γ (large) + γ (very large) + μ
j 04 j 05 j 06 j 0j
β = γ + γ (average SAT) + γ (average HSGPA) + γ
1j 10 11 j 12 j 13
(average financial aid) + γ (% white) + μ
14 1j
ββ = γ + γ ((average SSAATT)) + γ ((average HHSSGGPPAA)) + γ
2j 20 21 j 22 j 23
(% submitting SAT) + γ (small) + γ (large) + γ (very
j 24 j 25 j 26
largge)) + μ
jj 22jj..
8
Results: Final Estimation of Fixed Effects for
Interceppt ((Averagge FYGPA))
Variable Coefficient Robust S.E.
IInntteerrcceepptt 22.996633 00.003366
Average SAT/1000 0.912 0.151
AAvveerraaggee HHSSGGPPAA 00.002277 00.112299
Private Institution 0.105 0.039
Small Institution -0.073 0.041
Large Institution -0.009 0.030
Very large institution -0.031 0.045
9
Results: Final Estimation of Fixed Effects for
HSGPA Sloppe ((Predictive Validityy of HSGPA))
Variable Coefficient Robust S.E.
HHSSGGPPAA ssllooppee 00.441199 00.000088
Average SAT/1000 -0.298 0.129
AAvveerraaggee HHSSGGPPAA 00.005533 00.009911
Average Financial Aid -0.000 0.000
Percent of White first-yyear 0.002 0.000
students
10