THE IMPACT OF ERROR ON OFFENDER RISK CLASSIFICATION By Aaron KT Ho A Dissertation ...

210 Pages·2013·1.18 MB·English

by  Aaron Ho

THE IMPACT OF ERROR ON OFFENDER RISK CLASSIFICATION
By
Aaron K.T. Ho
A Dissertation submitted to the
Graduate School-Newark
Rutgers, The State University of New Jersey
In partial fulfillment of the requirements
For the degree of
Doctor of Philosophy
Graduate Program in Criminal Justice
Written under the direction of
Dr. Todd Clear
And approved by

___________________
___________________
___________________
___________________

Newark, New Jersey

May, 2013
©2013
Aaron K.T. Ho
ALL RIGHTS RESERVED
ABSTRACT
The Impact of Error on Offender Risk Classification
By Aaron K.T. Ho

Dissertation Director: Todd Clear

In criminal justice, offender risk classification seeks to divide individuals into
different groups, normally so that varying levels of program treatment, custody, or
supervision can be effectively and optimally allocated. The goal of effectively separating
offenders based on prearranged criteria, however, is often thwarted by error problems,
resulting in the misclassification of individuals. How the initial error problems
eventually translate into final misclassification is not completely understood. Thus, the
dissertation attempts to model the effects of error on the tolerance of offender risk
classification instruments. Specifically, different properties and characteristics of
classification devices are analyzed to understand their impact on the transfer of error
from initial to final classification phases.
Suitable risk data and instruments that would facilitate the testing of all proposed
research questions and hypotheses in the current study are not readily available. This is
because, in order to explore the different facets of the proposed inquiries, specific
situations are requisite- and these particular situations may be easier tailored into a
fabricated data than to be found in the real world. Thus, relying on both conceptual data
and actual risk data, random and systematic error are simulated and injected into each
ii
risk instrument to gain insight onto how unreliability and invalidity statistically impact
classification.
The risk data are engineered using Monte Carlo Simulation: construction methods
making use of random draws from an error distribution and multiple replications over a
set of known parameters. This methodology is particularly relevant in situations where
the only analytical findings involve asymptotic, large-sample results. Monte Carlo
Simulations enables the construction of multiple datasets in a “laboratory setting” that
would simulate data in the real world. This allows evaluations concerning the impact of
different risk properties on the transfer of error to be made.
For the current study, two main questions are asked: 1) what is the impact of error
in risk data on overall classification outcomes; and 2) how does such error impact validity.
The study found that risk tools generally have a low tolerance for error. The injection of
10 percent error into risk assessment information produced 25 to 40 percent error in
classification outcomes. However, the injection of random error only minimally reduces
classification validity by causing the subgroup recidivism/base rates for each category to
mildly shrink towards the mean. Different risk tools and factors play a critical role in
determining an instrument’s sensitivity to error. Specific risk properties such as
dichotomous risk items, having fewer risk categories, risk items with lower weights, and
having more risk items reduce the sensitivity of error in risk tools. A risk tool’s tolerance
for error is, thereby, controlled by a confluence of factors.
This dissertation facilitates a better understanding of the interplay between error
in risk information and error in classification outcomes. The findings improve
iii
knowledge of the sensitivity of error in offender risk classification instruments.
Furthermore, it explains how the sensitivity of error is aggravated or mitigated by the
inclusion of different common risk device properties.
iv
Acknowledgements
I would first like to thank my dissertation chair, Dr. Todd Clear, for his insights and
guidance throughout this entire process. Second, I would like to thank Christopher Baird
and his team for sharing their risk data on which the tests were conducted. Additionally,
I am indebted to Dr. Robert Apel, Dr. Joel Miller, and Eric Leneskie for their helpful
comments on drafts. Finally, the completion of this endeavor would not have been
possible without the support of my family, thank you.

v
Table of Contents
Abstract……………………………………………...……………………………… ii
Acknowledgements…………………………………………………………………. iv
Table of Contents…………………………………………………………………… v
List of Tables……………………………………………………………………….. vii
Chapter I-Introduction………………………………………………………………. 1
Importance…………………………………………………………………. 2
Purpose……………………………………………………………………... 5
Classification for Risk……………………………………………………… 6
Functions and Classification Characteristics………………………………. 6
Classification and Prediction………………………………………………. 11
Accuracy/Prediction Issue………………………………………………….. 15
Base-rates…………………………………………………………………… 20
Selection Ratios…………………………………………………………….. 23
Cutoff Scores……………………………………………………………….. 23
Validity in Context………………………………………………………….. 26
Constructing Classification Devices………………………………………... 27
Chapter II-Theoretical Framework………………...……………………………….. 31
Data Problems………………………………………………………………. 31
Data and Omission………………………………………………………….. 36
Transporting Risk Devices………………………………………………….. 37
Error in Application………………………………………………………… 39
Problems Focused in Study………………………………………………… 43
Chapter III-Standard for Construction and Evaluation…………………………….. 45
Informal Construction Process……………………………………………… 46
Measures of Validity……………………………………………………….. 50
Evolution and Validity……………………………………………………… 57
Reliability…………………………………………………………………… 71
Equity……………………………………………………………………….. 73
Cost and Efficiency Chicago’s Public............................................................ 75
Chapter IV-Methodology………………………...…………………………………. 78
Statement of the Problem…………………………………………………… 78
Datasets……………………………………………………………………... 79
Procedure for Injecting Error……………………………………………….. 97
Hypotheses………………………………………………………………….. 98
Analytical Plan……………………………………………………………… 105
Measurement/Statistical Analysis………………………………………...… 110
Chapter V-Assessing the sensitivity of error in risk devices- Analyses and Results.. 113
Results- Hypothesis #1……………………………………………………… 114
Hypothesis #2……………………………………………………... 119
Hypothesis #3……………………………………………………... 122
Hypothesis #4…………………………………………………….. 126
Hypothesis #5……………………………………………………... 132
Hypothesis #6…………………………………………………….. 138
Hypothesis #7……………………………………………………... 146
Hypothesis #8…………………………………………………….. 155
vi
Hypothesis #9……………………………………………………… 158
Hypothesis #10…………………………………………………….. 161
Hypothesis #11…………………………………………………….. 165
Hypothesis #12…………………………………………………….. 170
Summary……………………………………………………………………. 173
Hypothesis Summary……………………………………………………….. 176
Chapter VI-Discussions and Conclusions…………………………………………... 178
Overview of Research Findings…………………………………………….. 178
Limitations………………………………………………………………….. 184
Policy/ Future Research Implications………………………………………. 188
References…………………………………………………………………………... 191
Curriculum Vitae……………………………………………………………………. 200

vii
List of Tables
Table 1…………………………………………………………………………….. 18
Table 2…………………………………………………………………………….. 19
Table 3…………………………………………………………………………….. 54
Table 4…………………………………………………………………………….. 56
Table 5…………………………………………………………………………….. 83
Table 6.…………………………………………………….………………………. 84
Table 7…………………………………………………….………………………. 87
Table 8…………………………………………………………………………….. 88
Table 9………………………………………………………………….…………. 91
Table 10…………………………………………………….………………………. 92
Table 11……………………………………………………….……………………. 94
Table 12…………………………………………………….………………………. 96
Table 1A………………………………………….…………………………………. 118
Table 1B…………………………………………………………………………… 118
Table 2A…………………………………………………………………………… 120
Table 2B…………………………………………………………………………… 121
Table 3A…………………………………………………………………………… 126
Table 4A…………………………………………………………………………… 129
Table 5A…………………………………….………………………………………. 135
Table 5B……………………………………….……………………………………. 135
Table 5C…………………………………………………………………………….. 136
Table 5D…………………………………………………………………………….. 136
Table 5E…………………………………………………………………………….. 137
Table 5F…………………………………………………………………………….. 137
Table 6A…………………………………………………………………………….. 140
Table 6B…………………………………………………………………………….. 140
Table 6C……………………………………………………………………………. 144
Table 7A……………………………………………………………………………. 149
Table 7B…………………………………………………………………………….. 150
Table 7C…………………………………………………………………………….. 151
Table 7D…………………………………………………………………………….. 151
Table 7E…………………………………………………………………………….. 152
Table 7F…………………………………………………………………………….. 153
Table 8A……………………………………………………………………………. 156
Table 11A…………………………………………………………………………… 167
Table 11B…………………………………………………………………………… 168

iii
1

Chapter 1
Introduction
In biology, classification refers to the arrangement of living organisms into
different groups due to their similarities and differences. Separating organisms into
different classes or families helps scientists understand what different species have in
common with one another. The classification of offenders is similar. In criminal justice,
offender classification refers to the disaggregating of offenders into groups of individuals
with similar attributes. Classification helps criminologists divide offenders into
meaningful groups to serve a specific purpose, which will depend on the goals of
agencies (Champion, 1994). Based on a scientific model, it allows offenders to be treated
as members of groups for which there is an experience base (Clear, 1988). The actions of
other members of the group to which they belong form this experience. Without the
ability to group offenders, science in the use of classification is of no help to practitioners
in structuring decisions (Clear, 1988).
Misclassifications or the incorrect placement of individuals can occur if sufficient
error enters to distort classification. However, initial error does not necessarily translate
into final misclassification- this will greatly depend on the sensitivity of the classification
instrument to error. In other words, the initial quantity of error may or may not be
commensurate of the amount of error that is experienced in the final classification phase.
Many factors such as base rate, distribution of data, numbers of classification categories,
and number of classification items are speculated to have an impact on the sensitivity of