Building Bridges Initiative (BBI) Theory of Change (TOC) for Residential Intervention: Program Evaluation

ABSTRACT Building Bridges Initiative (BBI) Theory of Change (TOC) for Residential Intervention: Program Evaluation The Center for Innovation in Population Health (IPH Center) proposes a mixed-methods participatory evaluation to assess the impact of using the BBI’s TOC for Residential Intervention to initiate and implement change processes that improve outcomes at the client, program, and community levels. Because the TOC framework engages residential program providers and clients in the collaborative use of standard program evaluation tools (i.e., logic models driving quality improvement), this pilot project is ideally suited to participatory evaluation. In participatory evaluation, project stakeholders set the evaluation criteria, collect, and analyze data, and use those data to adjust and improve the project. For participatory evaluation of the BBI TOC, we will complete the following tasks: • During the Phase 1 project kick-off session, we will convene an evaluation sub-group of the Project Leadership team with at least one representative from each agency’s internal Project Team. After completing the main kick-off session agenda items, the evaluation sub-group will meet to review the draft list of process and outcome measures (see below) and make recommendations for revisions and additions. This sub-group will be asked to identify relevant administrative data already gathered by agencies 1) at the client, program, and community levels and 2) that might indicate practice pillar outcomes (e.g., what data do our agencies already gather to evaluate whether our services are strengths-based and individualized?). • During Phase 2, we will work with Project Leads and BBI consultants to collect all Project Team rosters and documentation detailed in the process measure list as well as Project-specific and cross-site outcome data. Process and outcome data also will be grouped by priority area/practice pillar. • Three months and 12 months after the completion of Phase 2, we will reconvene the evaluation sub-group to 1) review the process and outcome evaluation data across sites, by priority area/practice pillar, and for each Project and 2) discuss possible interpretations of patterns in the data and explanations of findings. The evaluation sub- group also will discuss what the data indicate in terms of the acceptability, feasibility, and usability of the TOC intervention in initiating and implementing change processes to improve residential program outcomes. • If feasible, we will compare as a control group the pre/post-intervention outcome measures from 4-6 residential programs in Oregon that are not participating in the TOC pilot matched by rural/urban community status and census to participating agencies. KEY PERSONNEL Margaret McGladrey, PhD, is a Research Assistant Professor in the Department of Health Management and Policy at the University of Kentucky College of Public Health who supports the work of the TCOM Implementation, Development, and Educational Support (TIDES) program at the IPH Center. Dr. McGladrey will leverage her expertise in participatory action research, practice-based research, and quality improvement to design and implement a program evaluation protocol for this project.