Casey Family Programs
Eckerd Rapid Safety Feedback Evaluation (replicated in Maine)
From 2016 to 2018 HZA is conducting the outcome evaluation of the Eckerd Rapid Safety Feedback (ERSF) Model in Maine and Connecticut in conjunction with Casey Family Programs and Eckerd Kids, a non-profit group in Florida. In this predictive analytics models, ERSF uses historical data from an agency’s case management system to predict which cases are at the highest risk of resulting in a child fatality or serious injury and a quality assurance process that monitors the assessment and treatment of those cases as they unfold. The outcome evaluation, using information from the model’s data portal and SACWIS data to track events such as the recurrence of abuse, serious injury or death, is designed to measure child welfare practice improvements and child safety outcomes, comparing outcomes for periods before and after implementation using an interrupted time series model.
Allegheny County (PA) Department of Human Services
Evaluation of a Predictive Risk Modeling Tool for Improving the Decisions of Child Welfare Workers
From 2016 to 2017, HZA performed a process evaluation of the Predictive Risk Modeling (PRM) Tool, which was developed for Allegheny County to improve the way child welfare workers make decisions about the children and families they serve. The Tool, which is used by intake staff when a call is received alleging abuse and/or neglect, gathers all relevant current and historical client information and assigns a risk score based on that information. The risk score is used by intake staff in making service decisions. The process evaluation assessed the Tool from the user’s perspective, examining its impact on consistency, efficiency and accuracy of decisions.
Arkansas Division of Children and Family Services
Quality Assurance Unit
HZA’s Quality Assurance Unit, located within the Division since 1997 and operating to the present, produces periodic outcome and performance reports, both for the Legislature and for the administration; monitors the degree to which various service providers comply with their contracts; conducts studies of issues of particular concern to the Division’s administrators; and performs ad hoc data analyses to respond to queries from both within and without the agency. Most of the analyses utilize the state’s SACWIS data, although the contract monitoring function and some of the special studies supplement those analyses with interviews and case readings.
Pennsylvania Office of Children Youth and Families
Data Support and Outcome Measurement
Beginning in 2001 and continuing to the present, HZA provides data support, analysis and outcome measurement for the Commonwealth’s child welfare system. With the lack of a statewide case management system, HZA uses data from the Adoption and Foster Care Analysis and Reporting System (AFCARS) and the National Child Abuse and Neglect Data System (NCANDS) files to respond to ad hoc data requests and generate county data packages on a semi-annual basis. HZA also provided support to OCYF for both rounds of Child and Family Services Reviews, using the federal syntax to enable the Commonwealth and its local counterparts to prepare for the federal reviews.
New Jersey Department of Children and Families
Longitudinal Data, Analysis and Reporting
From 2012 to 2017, HZA produces longitudinal data for analysis and reporting on a web-based platform using data drawn from NJ Spirit, the state’s SACWIS. The reports satisfy requirements of a modified settlement decree as well as the federally required Program Improvement Plan. The statewide dashboard contains 22 descriptive measures and five permanency measures. Measures include, among many others, the percentage of children in out-of-home placement who are abused or neglected by resource parents or facility staff, the percentage of children who exit custody and do not re-enter within 12 months, the percentage of children entering care who are placed with all of their siblings, the timeliness of initiating investigations, caseworker visits with children and caseworker visits with parents. HZA has developed an interactive web platform to display the data at both the statewide level and through county workbooks.
Ohio Department of Job and Family Services
Study of Validity and Reliability of the Family Risk Assessment Matrix
Because risk assessments are designed to determine whether child welfare cases should be opened for services, determining the impact of those services in reducing risk is a critical element in validating the assessment system. The agency asked HZA to focus most of its attention on the utilization of the Risk Assessment tool in the later stages of a child welfare case. The most important finding was that the structure of the Family Risk Assessment Matrix placed too much emphasis on factors that could not change; thus the earlier assessments of high risk masked the impact of the services that were delivered; in addition, workers did not use the tool consistently for follow-up periods. The initial risk assessments were reasonably valid and reliable, but the subsequent ones needed to be modified, if the agency was to direct its casework efforts where they were most needed.