What You'll Learn
The Concept
Program evaluation is the systematic assessment of whether a program achieves its intended goals. In criminal justice, programs are evaluated on outcomes such as crime reduction, recidivism, victim satisfaction, cost-effectiveness, and community trust. Evaluation is distinct from monitoring — which tracks program activities — and from research — which tests theoretical propositions. A well-designed evaluation answers a specific question: did this program, implemented in this way, with this population, produce these outcomes?
Process evaluation examines whether a program was implemented as intended. It asks: Did the program reach its target population? Were the activities delivered as designed? Did participants receive the intended dose? Process evaluation is essential because programs that are poorly implemented cannot be expected to produce the outcomes they were designed to achieve. A negative outcome evaluation of a poorly implemented program tells us little about whether the program model itself is effective.
Outcome evaluation examines whether a program achieved its intended goals. The central challenge is establishing causation: did the program cause the observed change, or would the change have occurred anyway? The strongest evidence comes from randomized controlled trials, in which eligible participants are randomly assigned to treatment and comparison conditions. Random assignment distributes both measured and unmeasured differences between groups, allowing researchers to attribute differences in outcomes to the program.
Randomized controlled trials are frequently impossible or inappropriate in criminal justice. Researchers cannot randomly assign people to poverty, incarceration, or criminal records. Quasi-experimental designs — including matched comparison groups, interrupted time series, and difference-in-differences analyses — attempt to approximate experimental conditions. These designs can produce credible causal estimates but require careful attention to the assumptions they rest on and the threats to validity they face.
Implementation quality is one of the most important determinants of program effectiveness. Research on evidence-based programs consistently finds that programs implemented with high fidelity to the program model produce better outcomes than those implemented poorly. This finding has important implications for policy: adopting a program that worked elsewhere does not guarantee it will work in a new context if implementation quality differs. The What Works Clearinghouse and the Crime Solutions database provide ratings of program evidence quality to help practitioners identify well-supported programs.
Unintended consequences are common in criminal justice programs. Displacement — the movement of crime to areas not targeted by an intervention — is a concern for place-based policing strategies. Diffusion of benefits — the spread of crime-reduction effects beyond the targeted area — is also documented. Net-widening — the expansion of social control to populations that would not otherwise have been subject to it — is a common concern with diversion programs. Evaluators must look beyond the intended outcomes to detect these effects.
Conflicting findings across multiple evaluations of the same program are common and do not necessarily mean that the evidence is inconclusive. Differences in findings may reflect differences in implementation quality, target population, context, outcome measures, or evaluation design. Systematic reviews and meta-analyses synthesize findings across multiple studies to estimate overall effects and identify the conditions under which programs are most effective. Students should be skeptical of claims based on a single study and look for patterns across multiple evaluations.
Cost-benefit analysis extends outcome evaluation by comparing the costs of a program to its benefits, expressed in monetary terms. Benefits may include reduced victimization costs, reduced incarceration costs, and increased earnings by program participants. Cost-benefit analysis allows policymakers to compare programs with different outcomes and to assess whether a program's benefits justify its costs. The RAND Corporation, the Washington State Institute for Public Policy, and the Vera Institute of Justice have developed cost-benefit models for criminal justice programs.
Applied Example: Scared Straight and the Importance of Rigorous Evaluation
Scared Straight programs, which expose juvenile offenders to adult prisoners who describe the realities of prison life, were widely adopted in the 1970s and 1980s based on anecdotal reports of success. A rigorous evaluation by Petrosino and colleagues, using a randomized controlled trial design, found that Scared Straight participants were more likely to reoffend than a comparison group that received no intervention. The program not only failed to reduce delinquency — it increased it. This finding illustrates the danger of adopting programs based on intuition or anecdote rather than rigorous evaluation, and the importance of randomized controlled trials for detecting harmful as well as beneficial effects.
Why It Matters
For Police Officers
Officers are often asked to implement new programs and strategies. Understanding how programs are evaluated — and what constitutes credible evidence of effectiveness — helps officers distinguish between programs with a genuine evidence base and those adopted for political or intuitive reasons.
For Students
Program evaluation is the bridge between research and practice. Students who understand evaluation design, implementation science, and the interpretation of conflicting findings can contribute to evidence-based policy development and avoid the trap of adopting programs that sound good but do not work.
Think About It
Why is process evaluation as important as outcome evaluation, and what happens when a program with a strong evidence base is implemented poorly?
What does the Scared Straight example suggest about the risks of adopting programs based on intuition or anecdote?
How should policymakers respond to conflicting findings across multiple evaluations of the same program?
What are the ethical considerations in conducting randomized controlled trials of criminal justice programs?
How can cost-benefit analysis help policymakers choose among programs with different outcomes and costs?
Sources & Further Reading
Petrosino, A., Turpin-Petrosino, C., & Buehler, J. (2003). Scared Straight and other juvenile awareness programs for preventing juvenile delinquency. Campbell Systematic Reviews, 2003(2).
Weisburd, D., Lum, C. M., & Yang, S. M. (2003). When can we conclude that treatments or programs 'don't work'? Annals of the American Academy of Political and Social Science, 587(1), 31–48.
Washington State Institute for Public Policy. (2023). Benefit-Cost Results. https://www.wsipp.wa.gov/BenefitCost
Weisburd, D., & Taxman, F. S. (2000). Developing a multicenter randomized trial in criminology: The case of HIDTA. Journal of Quantitative Criminology, 16(3), 315–340.
Lipsey, M. W., & Cullen, F. T. (2007). The effectiveness of correctional rehabilitation: A review of systematic reviews. Annual Review of Law and Social Science, 3, 297–320.