Sample CLA+ Institutional Report.

43 Pages·2017·0.79 MB·English

Spring 2017 CLA+ Results | Institutional Report
Sample University

cla+
Spring 2017 CLA+ Results Sample University

EXECUTIVE SUMMARY

CLA+ is a valuable tool that measures critical thinking and written-communication skills of students in higher
education. Institutions use CLA+ to estimate institutional and individual student growth of these essential
skills, measure the efficacy of curricular and other programs, and demonstrate individual, class, and institutional
proficiency. CLA+ results give individual students an opportunity to better understand their strengths and areas for
improvements in order to master the skills necessary for post-collegiate success. CLA+ Digital Badging gives
students who are proficient and beyond an opportunity to communicate these skills directly to employers.

CLA+ results are a tool to measure growth on these skills and determine how your institution compares to other
colleges and universities using CLA+.

Sample University has a freshman Total CLA+ score of 1035; this score is greater than or equal to the average
freshman score at 49% of CLA+ schools. A score of 1035 demonstrates Basic mastery of the critical-thinking and
written-communication skills measured by CLA+.

Sample University’s senior Total CLA+ score is 1144, which is better than or equal to the average senior score at
55% of CLA+ schools. A score of 1144 signifies Proficient mastery of the skills measured by CLA+.

Given the mean CLA+ performance of Sample University’s freshmen and the entering academic ability of its
seniors, Sample University’s value added is Near what would be expected relative to schools testing similar
populations of students.

Expected vs. Observed CLA+ Scores
1500
1400
1300
1200
All 4-Year CLA+ Colleges and
Universities
1100 Your School
Observed performance equal to
1000
expected performance
900
800
700
700 800 900 1000 1100 1200 1300 1400 1500
EXPECTED MEAN SENIOR CLA+ SCORE

Please note that our value-added model has been updated. We no longer require SAT/ACT scores. See Appendix K
(Modeling Details) for more information. In addition to the information provided here, key metrics contained in this
report include Mastery Levels, subscores, growth estimates, and percentile rankings:

Institutional Report i
OBSERVED CLA+ SCORE
Spring 2017 CLA+ Results Sample University

Mastery Levels
CLA+ Mastery Levels allow distinctions in student performance relative to students’ proficiency in critical
thinking and written communication. These levels contextualize CLA+ scores by interpreting test results in
relation to the qualities exhibited by examinees. Each Mastery Level—Below Basic, Basic, Proficient,
Accomplished, and Advanced—corresponds to specific evidence of critical-thinking and written-
communication skills.

CLA+ Subscores
In addition to total scores, there are six subscores reported across CLA+. The Performance Task—an
essay-based section of the exam—is scored in three skill areas: Analysis and Problem Solving, Writing
Effectiveness, and Writing Mechanics. Students receive criterion-referenced subscores for each skill
category based on key characteristics of their written responses. Selected-Response Questions are also
scored in three areas: Scientific and Quantitative Reasoning, Critical Reading and Evaluation, and Critique
an Argument. These subscores are scored based on the number of correct responses that students
provide.

Growth Estimates
The institutional report contains two types of growth estimates: effect sizes and value-added scores.

Effect sizes characterize the amount of growth shown across classes, and are reported in standard
deviation units. (Standard deviation is a measure of the distance between the mean, or average, and all
other values in a score set.) Effect sizes are calculated by subtracting the mean scores of the freshmen
from the mean scores of each subsequent class and dividing these amounts by the standard deviation of
the freshman scores.

Value-added scores provide estimates of growth relative to other CLA+ schools. Specifically, value-added
scores—also reported in standard deviation units—indicate the degree to which observed senior mean
CLA+ scores meet, exceed, or fall below expectations as established by two factors: the level of education
attained by the parents of the seniors and the mean CLA+ performance of freshmen at the school. The
first variable serves as a proxy for all demographic variables and has been shown to be strongly related to
academic outcomes in previous research, while the second variable serves as a baseline measure for the
academic ability of the students at that school.

Percentile Rankings
Percentile rankings allow for normative interpretations of your students’ performance. These rankings are
provided for your students’ CLA+ scores, as well as for your institutional value-added scores, and indicate
how well your institution performed relative to other CLA+ colleges and universities. Percentile rankings
indicate the percentage of CLA+ institutions whose scores are equal to or less than your own.

Please see Sections 1–6 for a full set of institutional results.

In addition to your institutional results, your CLA+ institutional report includes a wide variety of information related
to the measurement of higher-order thinking skills. Each section and appendix builds on the next to provide you
with a full appreciation of how the CLA+ can support the educational mission at your school. The CLA+ institutional
report’s appendices include information to help you learn about CLA+ measurement, understand relevant
statistical concepts, interpret your school’s data, examine your performance in relation to performance at other
CLA+ schools, and use CLA+ data to enhance student learning at your school.

Institutional Report ii
Spring 2015 CLA+ Results Sample University

TABLE OF CONTENTS

Your Results

1 Summary Results, by Class p. 2

2 Distribution of Mastery Levels p. 3

3 Value-Added Estimates p. 4

4 CLA+ Subscores p. 5

5 Student Effort and Engagement p. 6

6 Student Sample Summary p. 7

Appendices

A Introduction to CLA+ p. 8

B Methods p. 10

C Explanation of Your Results p. 12

D Results Across CLA+ Institutions p. 16

E Institutional Sample p. 20

F CLA+ Tasks p. 24

G Scoring CLA+ p. 27

H Mastery Levels p. 28

I Diagnostic Guidance p. 30

J Scaling Procedures p. 31

K Modeling Details p. 33

L Percentile Lookup Tables p. 37

M Student Data File p. 38

N Moving Forward p. 39

O CAE Board of Trustees and Officers p. 40

Institutional Report 1
Spring 2017 CLA+ Results Sample University

SECTION 1: SUMMARY RESULTS, BY CLASS

Number of Students Tested, by Class
Freshmen: 138 Sophomores: 126 Juniors: 132 Seniors: 118

Summary CLA+ Results, by Class
th th
25 75 MEAN SCORE EFFECT
MEAN STANDARD PERCENTILE PERCENTILE PERCENTILE SIZE V.
SCORE DEVIATION SCORE SCORE RANK FRESHMEN
TOTAL CLA+
Freshmen 1035 144 920 1121 49 --
SCORE
Sophomores 1087 142 994 1167 N/A 0.36
Juniors 1116 149 1014 1207 N/A 0.57
Seniors 1144 145 1059 1221 55 0.76
PERFORMANCE
Freshmen 1036 179 919 1125 50 --
TASK
Sophomores 1061 177 952 1131 N/A 0.14
Juniors 1094 181 973 1230 N/A 0.33
Seniors 1139 176 1047 1211 63 0.58
SELECTED-
Freshmen 1033 181 899 1170 49 --
RESPONSE
QUESTIONS
Sophomores 1113 179 1003 1232 N/A 0.44
Juniors 1139 183 1012 1244 N/A 0.59
Seniors 1149 177 1069 1254 52 0.64
ENTERING
Freshmen 1048 120 910 1183 58 --
ACADEMIC
ABILITY
Sophomores 1050 117 947 1117 N/A --

Juniors 1050 116 947 1150 N/A --

Seniors 1051 112 910 1180 45 --
Sample University has a senior Total CLA+ score of 1144 and percentile rank of 55. The
corresponding Mastery Level for this score is Proficient.

Institutional Report 2
Spring 2017 CLA+ Results Sample University

SECTION 2: DISTRIBUTION OF MASTERY LEVELS

Distribution of CLA+ Scores, by Mastery Level

25
20
15
FRESHMEN 10
5
0
400 600 800 1000 1200 1400 1600 1800

25
20
15
SOPHOMORES 10
5
0
400 600 800 1000 1200 1400 1600 1800

25
20
15
JUNIORS 10
5
0
400 600 800 1000 1200 1400 1600 1800

25
20
15
SENIORS 10
5
0
400 600 800 1000 1200 1400 1600 1800

Mastery Levels, by Class

MEAN MEAN PERCENT
TOTAL CLA+ MASTERY BELOW PERCENT PERCENT PERCENT PERCENT
SCORE LEVEL BASIC BASIC PROFICIENT ACCOMPLISHED ADVANCED
Freshmen 1035 Basic 29 37 23 10 0
Sophomores 1087 Basic 18 36 31 14 2
Juniors 1118 Proficient 13 30 35 19 3
Seniors 1144 Proficient 6 30 37 22 5

Institutional Report 3
Spring 2017 CLA+ Results Sample University

SECTION 3: VALUE-ADDED ESTIMATES

EXPECTED ACTUAL
SENIOR MEAN SENIOR MEAN
CLA+ SCORE CLA+ SCORE
Total CLA+ Score 1120 1144
Performance Task 1111 1139
Selected-Response Questions 1124 1149

VALUE-ADDED PERFORMANCE PERCENTILE CONFIDENCE INTERVAL BOUNDS
SCORE LEVEL RANK LOWER UPPER
Total CLA+ Score 0.55 Near 77 0.04 1.06
Performance Task 0.53 Near 77 -0.03 1.09
Selected-Response Questions 0.57 Near 72 -0.03 1.17

Expected vs. Observed CLA+ Scores
1500
1400
1300
1200
All 4-Year CLA+ Colleges and
Universities
1100 Your School
Observed performance equal to
1000
expected performance
900
800
700
700 800 900 1000 1100 1200 1300 1400 1500
EXPECTED MEAN SENIOR CLA+ SCORE

Please note that our value-added model has updated. We no longer require SAT/ACT scores. See Appendix K
(Modeling Details) for more information.

Institutional Report 4
OBSERVED CLA+ SCORE
Spring 2017 CLA+ Results Sample University

SECTION 4: CLA+ SUBSCORES

Performance Task: Distribution of Subscores (in percentages)
ANALYSIS & WRITING WRITING
PROBLEM SOLVING EFFECTIVENESS MECHANICS
100 100 100
75 75 75
49 49
41 43
50 50 50
FRESHMEN 24 20 23 20
25
2 6 0 25 2 7 0 25 0 11 5 0
0 0 0
1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6

100 100 100
75
51 75 55 75 52
38
50 50 50
28 28
SOPHOMORES 18
25
1 2 0 25 0 12 5 0 25 0 8 2 0
0 0 0
1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6

100 100 100
75 75 75
49 51
44
38
50 50 50
28 28
JUNIORS 18
15
25
1 8 1 25 1 6 1 25 0 2 9 0
0 0 0
1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6

100 100 100
75 75 75 59
48
43
50 32 50 34 50
SENIORS 25
25 16 5 25 13 10 25 5 12
0 0 0 0 0 0
0 0 0
1 2 3 4 5 6 1 2 3 4 5 6 1 2 3 4 5 6

NOTE: The Performance Task subscore categories are scored on a scale of 1 through 6.

Selected-Response Questions: Mean Subscores
SCIENTIFIC & CRITICAL
QUANTITATIVE REASONING READING & EVALUATION CRITIQUE AN ARGUMENT
th th th th th th
25 75 25 75 25 75
Mean Percentile Percentile Mean Percentile Percentile Mean Percentile Percentile
Score Score Score Score Score Score Score Score Score
FRESHMEN 496 420 572 502 413 568 490 388 551
SOPHOMORES 542 477 620 530 465 608 510 451 596
JUNIORS 531 477 620 557 508 619 539 458 598
SENIORS 561 477 620 532 465 588 547 458 598
NOTE: The selected-response section subscores are reported on a scale ranging approximately from 200 to 800.

Institutional Report 5
Spring 2017 CLA+ Results Sample University

SECTION 5: STUDENT EFFORT AND ENGAGEMENT

Student Effort and Engagement Survey Responses
How much effort did you put into the written-response task/ selected-response questions?
A MODERATE
NO EFFORT AT A LITTLE AMOUNT OF A LOT OF MY BEST
ALL EFFORT EFFORT EFFORT EFFORT
PERFORMANCE
TASK
Freshmen 0% 4% 36% 35% 25%
Sophomores 1% 3% 36% 41% 19%
Juniors 0% 4% 40% 32% 24%
Seniors 0% 2% 31% 37% 31%
SELECTED-
RESPONSE
Freshmen 3% 12% 43% 31% 11%
QUESTIONS
Sophomores 2% 6% 48% 26% 19%
Juniors 1% 14% 43% 30% 12%
Seniors 2% 7% 49% 26% 17%
How engaging did you find the written-response task/ selected-response questions?
NOT AT ALL SLIGHTLY MODERATELY VERY EXTREMELY
ENGAGING ENGAGING ENGAGING ENGAGING ENGAGING
PERFORMANCE
TASK
Freshmen 5% 17% 46% 29% 3%
Sophomores 4% 16% 39% 36% 5%
Juniors 9% 19% 42% 28% 2%
Seniors 7% 14% 45% 26% 9%
SELECTED-
RESPONSE
Freshmen 20% 25% 41% 13% 2%
QUESTIONS
Sophomores 12% 21% 46% 18% 4%
Juniors 17% 25% 33% 23% 2%
Seniors 12% 26% 42% 17% 4%

Institutional Report 6
Spring 2017 CLA+ Results Sample University

SECTION 6: STUDENT SAMPLE SUMMARY

Student Sample Summary
FRESHMEN SOPHOMORES JUNIORS SENIORS
DEMOGRAPHIC CHARACTERISTIC N % N % N % N %
TRANSFER Transfer Students -- -- 5 5% 7 7% 10 10%
Non-Transfer Students -- -- 102 95% 93 93% 91 90%
GENDER Male 40 39% 40 37% 40 40% 42 42%
Female 62 61% 65 61% 58 58% 58 57%
Decline to State 0 0% 2 2% 2 2% 1 1%
PRIMARY English 81 79% 83 78% 80 80% 82 81%
LANGUAGE
Other 21 21% 24 22% 20 20% 19 19%
FIELD Sciences & Engineering 21 21% 38 36% 25 25% 24 24%
OF
STUDY Social Sciences 11 11% 12 11% 14 14% 13 13%
Humanities & Languages 14 14% 10 9% 11 11% 15 15%
Business 17 17% 18 17% 25 25% 24 24%
Helping / Services 24 24% 20 19% 21 21% 21 21%
Undecided / Other / N/A 15 15% 9 8% 4 4% 4 4%
RACE/ American Indian / Alaska Native 1 1% 0 0% 1 1% 1 1%
ETHNICITY / Indigenous
Asian (including Indian 11 11% 11 10% 11 11% 11 11%
subcontinent and
Philippines)
Native Hawaiian or other Pacific 0 0% 1 1% 2 2% 0 0%
Islander
African-American / Black 13 13% 15 14% 14 14% 14 14%
(including African and
Caribbean), non-Hispanic
Hispanic or Latino 21 21% 20 19% 15 15% 17 17%
White (including Middle 52 51% 51 48% 50 50% 50 50%
Eastern), non-Hispanic
Other 0 0% 4 4% 3 3% 0 0%
Decline to State 4 4% 5 5% 4 4% 8 8%
PARENT Less than High School 4 4% 7 7% 5 5% 4 4%
EDUCATION
High School 26 25% 24 22% 19 19% 17 17%
Some College 27 26% 22 21% 27 27% 28 28%
Bachelor’s Degree 23 23% 31 30% 30 30% 30 30%
Graduate or Post-Graduate 19 19% 21 20% 19 19% 22 22%
Degree
Don’t Know / N/A 3 3% 2 2% 0 0% 0 0%
Institutional Report 7