Reducing Criminal Recidivism Through Predictive Analytics

Tackling criminal recidivism in a new and innovative way

Client Challenge

  • To reduce recidivism by understanding which subgroup of offenders the problem was most pervasive and evaluate the effectiveness of programming.

Solution

  • Aggregated data from disparate systems including the courts, criminal justice institute, and the offender management system.
  • Gathered insights by working alongside the State’s subject matter experts to analyze the data.
  • Developed a generalizable, proprietary algorithm suite, deemed the “Criminal Acts Indicator” tool to highlight relevant information and eliminate less actionable factors.
  • Evaluated program effectiveness in reducing recidivism.

Results

  • The algorithm tool allowed data to be effectively seen in a new way by projecting the future risk of recidivism, enabling the creation of individualized and optimized programming for offenders, and better informing policymakers’ decisions on important issues like sentencing reform.
Data Analytics

Contact Us

Case Study Contact Us Form Template

Recidivism is measured by criminal acts resulting in rearrest, reconviction, and/or return to prison with or without a new sentence during a three-year period following the inmate’s release.

June 17, 2014 National Institute of Justice

Setting The Stage

Indiana taxpayers pay over $700 million per year to run the Indiana Department of Correction. Housing, meals, and medical services equate to roughly $53 per adult offender per day. Of approximately 27,000 offenders in the system, roughly 17,000 are released every year.

This rapid turnover heightens the need for effective rehabilitative services for incarcerated offenders. Reducing criminal recidivism by successfully transitioning offenders into productive citizens not only provides a positive social outcome, the main goal, but can also result in favorable fiscal impacts.

Advanced data analysis techniques make it possible to target the right programs, at the right time, to the right offenders.

The Problem

Studies investigating causes and solutions to recidivism have established that although rates in the US vary among states, all states suffer from similar social and economic consequences. While these studies can be helpful in reforming policy by providing insight into the general effectiveness of programs and services, they rely on assumptions. As a result, the studies lack statistical rigor, which inhibits the ability to make a substantive impact on offenders.

Reducing recidivism requires an in-depth look into when specific subgroups of offenders have the greatest likelihood to recidivate, which suite of programs will optimally reduce their risk of returning to prison, and how to equip policymakers with the information they need to make informed decisions and investments.

The State of Indiana (the State) asked KSM Consulting (KSMC) to work alongside it in the effort to reduce recidivism in Indiana by:

  • Understanding for which subgroup of offenders the problem was most pervasive;
  • Evaluating the effectiveness of programming; and,
  • Providing actionable guidance on specific steps to reduce recidivism.

The KSMC team established a plan to apply advanced analytical techniques to cross-agency data, which would in turn provide actionable insights for the State.

The Solution

The KSMC data analytics team leveraged cross-agency data from the State to help tackle the issue of recidivism in a new and innovative way.

Data Discovery and Analysis
KSMC aggregated data from disparate systems including the courts, criminal justice institute, and the offender management system. The team worked alongside the State’s subject matter experts to analyze the data and understand the insights by applying a generalizable, proprietary algorithm suite, deemed the “Criminal Risk Indicator” tool to highlight relevant information and eliminate less actionable factors. With a full understanding of the data, the team was able to evaluate program effectiveness in reducing recidivism.

Effective vs. Ineffective Questions

When attempting to reduce recidivism, it is important to start by asking the right
questions. The team refrained from focusing on questions that were simplistic, drew their own conclusions, or allowed external factors to influence answers. Instead, questions were tailored to specific offenders.

Ineffective Questions

  • Which programs are effective?
  • Which facility rehabilitates offenders best?

Effective Questions

  • How can we best rehabilitate this offender?
  • If this program is applied at this time to this offender, how much of a decrease in his/her probability of recidivism can we expect?
  • How can we best reduce recidivism for the offender population by spending $XX?

Program Participation Optimization
The State was interested in identifying specific programs that were effective in reducing recidivism. Of the programs evaluated, the team was able to identify not only which programs were effective, but for which offender a program would be most effective, given the individual’s unique characteristics, background, and criminal history.

Upon determining the optimal program for each offender, the team analyzed the marginal impact of completing a secondary program.

Through the development of this tool, case managers can supplement existing practices to identify optimal programming for specific offenders, based on their characteristics such as age or offense. The tool will better inform case managers on what combination of programs are most effective and provide the greatest likelihood of rehabilitative success.

Methodologies

  • Exploratory Data Analysis
  • Feature Selection
  • Advanced Feature Engineering
  • Propensity Score Matching
  • Counter-Factual Estimation
  • Logistic Regression

  • Random Forests
  • Clustering
  • Mixture Modeling
  • High-Dimensional Parameterization
  • MCMC

The Impact

gavelThe “Criminal Risk Indicator” algorithm tool developed by the KSMC data analytics team on top of the SAP HANA® platform allows data to be effectively seen in a new way by:

  • Projecting the future risk of recidivism;
  • Enabling the creation of individualized and optimized programming for offenders; and
  • Better informing policymakers’ decisions on important issues like sentencing reform.

With the tool, KSMC and the State are able to specify program participation for specific offenders, which will lead to a data-driven understanding of the most effective programs for each inmate to combat recidivating.

Faced with a limited amount of funding, the State can more effectively align programs and individual offenders to provide the greatest potential for success.

Using KSMC’s recidivism tool, the State is using data to reexamine the eligibility requirements of each program for offenders. In addition, the State is able to make data-driven decisions about its allocation of programming and funding.

Methodology Deep Dive

Analysis Methodology Brief
Causality conceptualizes program impact as the difference in outcomes under treatment and control, only one of which is observed (e.g. offender enrolls in a program and does not recidivate). Estimating program impact is a missing data problem. Specifically, the counterfactual is unobserved, “what would have happened if the offender did not take the program?”

To determine the effectiveness of offender programs, it is imperative that impact be assessed at the individual offender level; not the treatment level. The importance of the individual approach is due to the non-randomized manner of offender participation. When offenders elect to attempt, are court ordered, or advised to take a program, this may lead to differences in characteristics of program participants and non-participants. Failure to control for characteristic differences (prior conviction history, offender classification, age, education, etc.) can lead to biased program estimates due to confounding program impact with characteristic differences.

To determine the effectiveness of offender programs, it is imperative that impact be assessed at the individual offender level; not the treatment level.

graphEstimating the Program Effect
The graph, shown, portrays two subpopulations defined by the age of the offender. This simple comparison is meant to visualize the challenge and nuances associated with estimating program effects. (In reality, programs often have differences among numerous characteristics.) The population that participated in the program is older relative to the population that did not participate. Given the significant age difference, it is necessary to first identify and control the impact of age, then estimate treatment impact.

If offenders elect to attempt, are court ordered, or advised to take a program, then assignment to the treatment is not random. In observational settings, such as this, balance across treatment and control groups is not guaranteed – resulting in potentially biased estimates of a program’s true effect. To recover an accurate estimate of program impact in the presence of non-randomized treatment assignment, KSMC estimates the counterfactual by applying feature selection, propensity score matching, non-linear regression, MCMC, and machine learning techniques.