ERIC ED459186: Colorado Student Assessment Program Technical Report, 2000.

141 Pages·2001·2.6 MB·English

by  ERIC

DOCUMENT RESUME
TM 033 462
ED 459 186
Colorado Student Assessment Program Technical Report, 2000.
TITLE
Colorado State Dept. of Education, Denver.; CTB /
INSTITUTION
McGraw-Hill, Monterey, CA.
2001-00-00
PUB DATE
NOTE
140p.
Descriptive (141)
PUB TYPE Reports
MF01/PC06 Plus Postage.
EDRS PRICE
*Academic Achievement; Achievement Tests; Elementary
DESCRIPTORS
Secondary Education; Ethnicity; Racial Differences; School
Districts; Scores; Sex Differences; *Test COnStrUCtiOn; Test
Results; Test Use; *Testing Programs
Colorado; *Colorado Student Assessment Program
IDENTIFIERS
ABSTRACT
This report presents the results of the statewide spring
2000 administration of the Colorado Student Assessment Program (CSAP).
Third-grade students were assessed in Reading; fourth- and seventh-grade
students were assessed in Reading and Writing; and eighth-grade students were
assessed in Mathematics and Science. The assessments were developed by
CTB/McGraw-Hill in association with the Colorado Department of Education and
were scored, scaled, and analyzed by CTB/McGraw-Hill. This technical report
contains information about the assessments themselves, the results, item
analyses, scaling, and calibration. One appendix presents the frequency
distribution of scale scores, and the other contains a differential item
functioning analysis. (Contains 107 tables and 19 references.)
(SLD)
Reproductions supplied by EDRS are the best that can be made
from the original document.
Colorado
Student Assessment Program
Technical Report
2000
Submitted to the
Colorado Department of Education
cI
Co
ce)
CTB/McGraw-Hill
0
A Diviskm of TheMeGrapHill
Companies
PERMISSION TO REPRODUCE AND
DISSEMINATE THIS MATERIAL HAS
BEEN GRANTED BY
AVAILABLE
BEST COPY
vuoL±s071
TO THE EDUCATIONAL RESOURCES
INFORMATION CENTER (ERIC)
2
2000 CSAP Technical Report
Developed and published under contract with Colorado
Department of Education by CTB/McGraw-Hill, a division of
The McGraw-Hill Companies, Inc., 20 Ryan Ranch Road,
Monterey, California 93940-5703. Copyright © 2001 by
Colorado Department of Education. All rights reserved. No
part of this publication may be reproduced or distributed in any
form or by any means, dowrloaded, or stored in a database or
retrieval system, without the prior written permission of
Colorado Department of Education.
II
3
2000 CSAP Technical Report
Table of Contents
Part 1. Overview of the CSAP Assessments
1
Test Development and Content Validity
3
Test Configuration
4
Part 2. Results
9
Summary Statistics
10
The Relationship Among the Standards
19
Test Reliability
21
Part 3. Item Analyses
22
Part 4. Scaling and Calibration
37
Overview of the IRT Models
37
Scaling and Calibration of the Assessment
39
40
Model Fit
Procedures for Detecting and Reducing Bias in CSAP
55
56
Differential Item Functioning Analyses
58
Differential Item Functioning Ratings
Results of the Differential Item Functioning Analyses
59
Standard Errors
60
References
66
Appendix A. Frequency Distribution of Scale Scores
68-107
Appendix B. Differential Item Functioning Analysis
108-131
iii
4
2000 CSAP Technical Report
Tables
Table 1.
Configuration of 2000 CSAP Assessments
5
Table 2. Configuration of 2000 CSAP Assessments by Grade,
Content Standard, and Item Type
5
Table 3. Lowest and Highest Obtainable Scale Scores
for the CSAP Assessments
9
Table 4.
Scale Score Descriptive Statistics
11
Table 5. Scale Score Descriptive Statistics by Content Standard,
All Students
12
Table 6. Raw Score Descriptive Statistics by Content Standard,
All Students
13
Table 7. Correlations by Content Standard
20
23
Table 8. Multiple-Choice Item Analyses, Grade 3, Reading
Table 9. Constructed-Response Item Analyses, Grade 3, Reading
23
Table 10. Multiple-Choice Item Analyses, Grade 4, Reading
25
26
Table 11. Constructed-Response Item Analyses, Grade 4, Reading
Table 12. Multiple-Choice Item Analyses, Grade 4, Writing
27
iv
5
2000 CSAP Technical Report
Table 13. Constructed-Response Item Analyses, Grade 4, Writing
27
Table 14. Multiple-Choice Item Analyses, Grade 7, Reading
29
Table 15. Constructed-Response Item Analyses, Grade 7, Reading
30
Table 16. Multiple-Choice Item Analyses, Grade 7, Writing
31
Table 17. Constructed-Response Item Analyses, Grade 7, Writing
31
Table 18. Multiple-Choice Item Analyses, Grade 8, Mathematics
33
Table 19. Constructed-Response Item Analyses, Grade 8, Mathematics
33
Table 20. Multiple-Choice Item Analyses, Grade 8, Science
35
Table 21. Constructed-Response Item Analyses, Grade 8, Science
36
Table 22. Item Parameter and Fit Summary', Multiple-Choice Items,
Grade 3, Reading (N=46,265)
41
Table 23. Item Parameter and Fit Summary', Constructed-Response
Items, Grade 3, Reading (N=46,265)
41
Table 24. Item Parameter and Fit Summary', Multiple-Choice Items,
43
Grade 4, Reading (N=40,592)
Table 25. Item Parameter and Fit Summary', Constructed-Response
44
Items, Grade 4, Reading (N=40,592)
Table 26. Item Parameter and Fit Summary', Multiple-Choice Items,
Grade 4, Writing (N=40,438)
45
6
2000 CSAP Technical Report
Table 27. Item Parameter and Fit Summary', Constructed-Response
Items, Grade 4, Writing (N=40,438)
45
Table 28. Item Parameter and Fit Summary', Multiple-Choice Items,
47
Grade 7, Reading (N=34,108)
Table 29. Item Parameter and Fit Summary', Constructed-Response
Items, Grade 7, Reading (N=34,108)
48
Table 30. Item Parameter and Fit Summary', Multiple-Choice Items,
49
Grade 7, Writing (N=33,523)
Table 31. Item Parameter and Fit Summary', Constructed-Response
Items, Grade 7, Writing (N=33,523)
50
Table 32. Item Parameter and Fit Summary', Multiple-Choice Items,
Grade 8, Mathematics (N=49,620)
51
Table 33. Item Parameter and Fit Summary', Constructed-Response
Items, Grade 8, Mathematics (N=49,620)
51
Table 34. Item Parameter and Fit Summary', Multiple-Choice Items,
Grade 8, Science (N=49,955)
53
Table 35. Item Parameter and Fit Summary', Constructed-Response
Items, Grade 8, Science (N=49,955)
54
vi
7
2000 CSAP Technical Report
Table 36. Classical Test Theory Estimate of Overall
Standard Error of Measurement
61
62
Table 37. Scale Score and Associated Standard Errors, Grade 3, Reading
63
Table 38. Scale Score and Associated Standard Errors, Grade 4, Reading
Table 39. Scale Score and Associated Standard Errors, Grade 4, Writing 63
64
Table 40. Scale Score and Associated Standard Errors, Grade 7, Reading
Table 41. Scale Score and Associated Standard Errors, Grade 7, Writing 64
Table 42. Scale Score and Associated Standard Errors, Grade 8,
65
Mathematics
65
Table 43. Scale Score and Associated Standard Errors, Grade 8, Science
Figures
72
Frequency Distributions of Scale Scores: Grade 3, Reading
Figures 1-3.
78
Frequency Distributions of.Scale Scores: Grade 4, Reading
Figures 4-6.
Frequency Distributions of Scale Scores: Grade 4, Writing
83
Figures 7-9.
Figures 10-12. Frequency Distributions of Scale Scores: Grade 7, Reading
89
95
Figures 13-15. Frequency Distributions of Scale Scores: Grade 7, Writing
vii
3
2000 CSAP Technical Report
Figures 16-18. Frequency Distributions of Scale Scores: Grade 8,
Mathematics
101
Figures 19-21. Frequency Distributions of Scale Scores: Grade 8, Science
107
9
2000 CSAP Technical Report
The 2000 Colorado Student Assessment Program
Technical Report
This report presents the results of the statewide Spring 2000 administration of
the Colorado Student Assessment Program (CSAP). Third-grade students were
assessed in Reading; fourth- and seventh-grade students were assessed in Reading
and Writing; and eighth-grade students were assessed in Mathematics and Science.
The assessments were developed by CTB/McGraw-Hill in concert with the Colorado
Department of Education and were scored, scaled, ard analyzed by CTB/McGraw-
Hill.
Part 1: Overview of the CSAP Assessments
The assessment is intended to measure the Colorado Content Standards, as
follow;
The Colorado Model Content Standards: Reading and Writing
Students read and understand a variety of materials. (Reading)
1.
Students write and speak for a variety of purposes and audiences. (Writing)
2.
Students write and speak using conventional grammar, usage, sentence
3.
structure, punctuation, capitalization, and spelling. (Writing)
Students apply thinking skills to their reading, writing, speaking, listening, and
4.
viewing. (Reading)
Students read to locate, select, and make use of relevant information from a
5.
variety of media, reference, and technological sources. (Reading)
Students read and recognize literature as a record of human experience.
6.
(Reading)
ix
I 0