Participatory Evaluation of Practitioner Utilization of an AI-Based Decision Support Tool in a Psychiatric Residential Treatment Facility

Child-serving agencies are increasingly adopting decision support tools informed by sophisticated computational methods and Artificial Intelligence (AI). However, research on the practical application and usage of these tools in child and family services remains limited, despite their potential to perpetuate systemic biases. This paper provides a case study of the implementation of an AI-based decision support tool at a Psychiatric Residential Treatment Facility. Through Ripple Effects Mapping, a participatory evaluation method, the authors conducted focus groups with 31 staff members across different roles. The results revealed that practitioners used the tool to track longitudinal progress; enhance stakeholder communication; and provide person-centered care, such as better matching staff to youth. Implementation barriers included concerns about data accuracy, new staff training needs, and challenges integrating the tool into workflow. The study identified strategies for improvement, including incorporating the tool into daily practices across roles, recognizing front-line staff use of the tool, and investing in training. While AI-based decision support tools must be carefully designed and implemented to avoid perpetuating systemic discrimination, the findings suggest that they can help providers offer appropriate and person-centered care.