Advancing Fairness and Transparency: National Guidelines for Post-Conviction Risk and Needs Assessment
Practitioners
For practitioners who use risk and needs assessment on a day-to-day basis, these guidelines provide clarity, inform case planning, and promote buy-in from the people they assess.
Risk and needs assessment instruments are widely used in criminal justice settings across the United States to inform decisionmaking at various stages in case processing. These guidelines were developed to promote accuracy, fairness, transparency, and effective communication and use of risk and needs assessment instruments to inform decisionmaking following conviction.
Our aim is to provide guidance that serves as a reference for policymakers and agency administrators in diverse criminal justice settings, as well as for developers of new instruments. Agencies can apply these guidelines whether they are continuing to use an existing instrument or considering adopting a new one. Readers are encouraged to consider the high level guidelines below alongside the additional resources that delve deeper into each individual guideline.
These guidelines are drawn from empirical research and reflect the perspectives of researchers, practitioners, and policymakers who participated in several discussions on key concerns in the development, validation, and implementation of post-conviction risk and needs assessment instruments. These guidelines also consider existing statements and guidance on the use of risk and needs assessment instruments written by diverse groups and for diverse contexts. They presume that the intended use of post-conviction risk and needs assessment instruments is to support accurate, fair, and transparent decisionmaking regarding a person’s risk of recidivism and, ultimately, to promote public safety and positive outcomes for people in the criminal justice system, including habilitation and community reintegration, through the least restrictive means possible.
High Level Guidelines
I. To promote ACCURACY of post-conviction risk and needs assessment instruments
- Conduct a local evaluation of the post-conviction risk and needs assessment instrument to ensure that the instrument is suitable for the agency’s population.
- Meet the minimum performance thresholds of post-conviction risk and needs assessments completed in the field according to statistical standards.
- Use a continuous quality improvement (CQI) process to ensure successful implementation of the post-conviction risk and needs assessment instrument.
- Use a multistep approach to assess risk and needs over time.
II. To promote FAIRNESS of post-conviction risk and needs assessment instruments
- Examine the results of the post-conviction risk and needs assessment instrument for predictive bias and disparate impact across groups.
- Apply the post-conviction risk and needs assessment instrument results to individual cases in keeping with the Risk-Need-Responsivity (RNR) principles.
- Adopt agencywide strategies to minimize the potential that local implementation of the postconviction risk and needs assessment instrument could promote disparities.
III. To promote TRANSPARENCY of post-conviction risk and needs assessment instruments
- Provide system stakeholders with relevant information on the development, intended use, and validation of the post-conviction risk and needs assessment instrument.
- Develop a written policy that guides the local use of the post-conviction risk and needs assessment instrument.
- Communicate the strengths and limitations of the post-conviction risk and needs assessment instrument to the general public.
IV. To promote effective COMMUNICATION AND USE of postconviction risk and needs assessment instrument results
- Anchor communication of assessment results in the RNR principles.
- Contextualize the results of the post-conviction risk and needs assessment instruments.
- Develop a template for communicating individual results of the post-conviction risk and needs assessment instruments to all relevant stakeholders, including the person being assessed.
Expanded Guidelines
I. Accuracy
Accuracy refers to the degree to which assessment results predict the recidivism outcomes they were designed to predict as indicated by the observed rate and severity of criminal behavior, as well as the identification of individuals at greater and lesser risk of recidivism. Determining accuracy also involves considering whether the post-conviction risk and needs assessment instruments are completed and used as intended to inform case decisions and planning within facilities and in the community.
- Conduct a local evaluation of the post-conviction risk and needs assessment instrument to ensure that the instrument is suitable for the agency’s population.
- Establish inter-rater reliability prior to using the instrument.
- Complete a local validation, ideally prior to using the instrument, to confirm that the assessment results are predicting recidivism using local data and in the context of current and local practices.
- Revalidate assessment results at least every 5 years—or sooner if there are major policy or population changes—to verify that the assessment results continue to meet minimum performance thresholds.
- Consult with experts such as university partners or other experienced evaluators, as needed, to ensure that local evaluations adhere as much as possible to best practices in risk and needs assessment research and standards in test validation.
- Meet the minimum performance thresholds of post-conviction risk and needs assessments completed in the field according to statistical standards.
- Demonstrate good agreement or better among assessors for post-conviction risk and needs assessments completed in the field.
- Demonstrate good validity or better in predicting the likelihood of recidivism with post-conviction risk and needs assessments completed in the field.
- Use a continuous quality improvement (CQI) process to ensure successful implementation of the post-conviction risk and needs assessment instrument.
- Document the protocols for applying the post-conviction risk and needs assessment instrument.
- Prior to their use of the instrument in practice, provide all assessors with training on the rating procedures and protocols for applying assessment results to inform case plans.
- Complete case reviews at least twice yearly during implementation to identify problems to correct through individual coaching or booster training. These case reviews should examine:
- Fidelity to the rating and scoring guidelines.
- Adherence to the implementation protocols.
- Concordance among assessment results and case decisions, resource allocation, and service provision.
- Conduct booster training at least annually for all assessors during implementation. These sessions should:
- Review rating procedures and protocols for using assessment results.
- Require completion of at least one—but ideally two—practice cases to good agreement or better with a “gold standard” or expert rating.
- Address any other issues identified in the case reviews.
- Use a multistep approach to assess risk and needs over time.
- Follow post-conviction risk screening instruments, if used, with a comprehensive risk and needs assessment only for people identified as being at potentially heightened risk of recidivism.
- Re-administer post-conviction risk and needs assessment instruments that include dynamic factors and needs at routine intervals to monitor individual progress and inform amendments to case planning, as needed.
II. Fairness
Fairness is the degree to which assessment results have the same meanings and applications across groups defined by race, ethnicity, gender, or other characteristics such as mental illness. Fairness should be considered in the development, validation, and implementation of post-conviction risk and needs assessment instruments.
- Examine the results of the post-conviction risk and needs assessment instrument for predictive bias and disparate impact across groups.
- Establish whether the likelihood of recidivism increases in similar ways across risk levels for members of groups defined by race, ethnicity, and gender.
- Test whether assessment results identify individual risk levels and needs and predict recidivism in the same way from group to group.
- Compare how assessment results relate to case decisions, resource allocation, and service provision across groups.
- Apply post-conviction risk and needs assessment instrument results to individual cases in keeping with the Risk-Need-Responsivity (RNR) principles.
- Use assessment instrument results to inform the appropriate level of intervention needed to manage the assessed risk of recidivism.
- Identify the dynamic factors and needs to be addressed through intervention.
- Maximize reductions in recidivism by tailoring the interventions to individual motivations, strengths, and abilities.
- Adopt agencywide strategies to minimize the potential that local implementation of a post-conviction risk and needs assessment instrument could promote disparities.
- Select and implement post-conviction risk and needs assessment instruments based on their performance, content, and context.
- Develop and implement strategies to support equitable and safe case decisions, resource allocation, and service provision.
III. Transparency
Transparency refers to how information about the content, structure, and application of these instruments is disseminated to stakeholders. Transparency is relevant in both the development and implementation of risk and needs assessment instruments and requires a proactive communication strategy.
- Provide system stakeholders with relevant information on the development, intended use, and validation of post-conviction risk and needs assessment instruments.
- Articulate the purpose for which the post-conviction risk and needs assessment instrument was developed, including the intended settings, populations, and outcomes.
- Explain the set of factors the assessment considers—including their definitions, scoring, and weighting— in a manner that can be understood by different audiences, particularly those who will be using the results and those who will be affected by them.
- Describe how risk levels are assigned.
- Outline the training requirements for people administering the instrument, including CQI elements described previously.
- Publish the findings of validation studies examining the post-conviction risk and needs assessment instrument in a manner that is accessible to a variety of audiences.
- Develop a written policy that guides the local use of the post-conviction risk and needs assessment instrument.
- Describe the source(s) of information that will be used to complete the post-conviction risk and needs assessments locally and identify potential pitfalls, such as data quality or biases, that may exist in these sources.
- Define the contexts in which and how the results of the post-conviction risk and needs assessment instrument will be used to inform case decisions, resource allocation, and service provision.
- Create the opportunity for input on the written policy from stakeholders.
- Establish a process and timeline to review and update the written policy, as necessary.
- Communicate the strengths and limitations of the post-conviction risk and needs assessment instrument to the general public.
- Make the following information available for easy access by the general public:
- What the post-conviction risk and needs assessment instrument is and is not designed to do.
- How item selection and weighting minimize racial, ethnic, and gender disparities in assessment results while promoting accuracy.
- How the post-conviction risk and needs assessment instrument has been evaluated, including studies of predictive bias and disparate impact, as well as any limitations or gaps in research that remain to be addressed.
- Describe the process through which the post-conviction risk and needs assessment instrument was selected for implementation.
- Clearly state how the post-conviction risk and needs assessment instrument will be used locally.
- Explain how the accuracy and impact of the post-conviction risk and needs assessment instrument on case outcomes will be monitored overall and across groups.
- Make the following information available for easy access by the general public:
IV. Effective Communication and Use
The manner in which individual assessment results are communicated and used can greatly affect their impact on decisionmaking and, consequently, their effectiveness. Improper communication of individual assessment results can undermine efforts to promote accuracy, fairness, and transparency in the use of post-conviction risk and needs assessment instruments and should be a key consideration in their implementation.
- Anchor communication of post-conviction risk and needs assessment results in the RNR principles.
- Describe assessment results as placing an individual in a particular risk level that informs the minimum level of intervention needed to mitigate their risk of recidivism rather than assigning a specific probability or likelihood of recidivism to the individual.
- Identify the presence of risk and protective factors that contribute to the assessment results, emphasizing the dynamic factors and needs that should be addressed through intervention.
- Explain case-specific barriers or facilitators to successful habilitation, above and beyond those described previously.
- Contextualize the results of the post-conviction risk and needs assessment instrument.
- State the likelihood and, when possible, the type(s) of criminal behavior anticipated in the absence of interventions over the timeframe(s) specified by the instrument.
- Define the parameters of the assessment results.
- Identify the type and approximate intensity of interventions that are likely to reduce the anticipated risk of recidivism and support successful case outcomes.
- Develop a template for communicating the individual results of the post-conviction risk and needs assessment instrument to all relevant stakeholders, including the person being assessed.
- Provide a structure and format for presenting the assessment results in a manner that is clear, concise, predictable, and consistent across assessors and cases.
- Use communication strategies that promote comprehension and reduce the impact of potentially problematic information.
- Tailor communication to the target audience, with the potential for different templates for different stakeholders, but avoid sharing assessment results beyond relevant stakeholders.
- Share the template with stakeholders for review and feedback prior to finalizing it.
.
.
.