Validation of Safety Culture Organizational Assessment in Kentucky CDR Teams

PROPOSAL COVERSHEET Enclosed in this document please find our proposed project, Development of a Safety Culture Organizational Assessment in Kentucky Child Death Review Teams, submitted for consideration for the 2021 Children’s Justice Act. The nature of Child Death Review (CDR) work is high-stress, and the end result of this team-based work is of high-consequence. Despite the importance of understanding how CDR teams operate, there is limited research describing decision-making, team dynamics and individual stressors on CDR teams. Organizational assessments are used routinely to help teams to understand and identify areas of change within their decision-making, teamwork, and communication. This proposal outlines a draft strategy for developing a safety culture survey specifically for use among CDR teams in Kentucky. Such a tool could eventually be used to help these teams better understand the culture of their teams, identify areas of vulnerability in their culture and decision-making habits, and facilitate targeted quality improvement efforts to improve the outcomes of their work. Our team has extensive experience and expertise related to work with fatality review teams and development of organizational measurement tools. We work directly with child fatality review teams across the country on creating and maintaining a systems-focused perspective on child fatality reviews, and (as of November 1, 2021) we have engaged 14 public child welfare jurisdictions around the country in their use of an organizational assessment (safety culture survey) to better understand their team decision-making, culture, and communication. We thank you for considering our work and for the opportunity to apply for this award. We look forward to hearing from you soon. Respectfully submitted, Elizabeth Riley, PhD Assistant Professor Center for Innovation in Population Health / Department of Health Management and Policy College of Public Health University of Kentucky [email protected] 859.536.7776 PROPOSAL Problem Description. A Child Death Review (CDR) provides a comprehensive and multidisciplinary review of the circumstances surrounding a child death, in order to better understand how and why children die. Findings from these reviews are used to catalyze action aimed at preventing future deaths and, ultimately, improving the health and safety of communities, families, and children. CDR teams collect extensive data on the child, family, and incident that led to the child’s death. In addition, these teams examine data from the death investigation to determine both case-specific risk/protective factors, and systems-level issues and barriers. Moreover, CDR team members work professionally in roles such as law enforcement, child protection and healthcare where effective teamwork skills are critical and their work puts them at increased risk for burnout, stress, and fatigue. The nature of CDR work is high-stress, and the end result of this team-based work is of high-consequence. Despite the importance of understanding how CDR teams operate, there is limited research describing decision-making, team dynamics and individual stressors on CDR teams. Safety culture surveys are used extensively in areas where work is high-stress and high-consequence, such as healthcare, aviation, and high-risk industrial settings. These assessments of team values, behaviors and beliefs are also an often used as a first step toward system-level culture change efforts in high-stress work settings by helping teams to identify highly actionable needs related to decision-making, teamwork, and communication. This proposal outlines a draft strategy for developing a safety culture survey for CDR; currently, no such tool exists for use with CDR teams. The proposed safety culture survey will include measures of psychological safety, emotional exhaustion, safety climate, and team decision-making. The development of a CDR specific safety culture survey may help teams throughout the state of Kentucky identify specific areas of improvement within their team culture to determine targeted intervention efforts. Background. CDRs enable states and communities to identify underlying risk and protective factors related to child deaths and to use that information to create meaningful change and safer communities. These reviews help to generate a deeper understanding of how the child lived and why the child died. Currently, there are more than 1,500 CDR teams across the country, and CDR teams exist in all 50 states, the District of Columbia, and many Native American and Alaskan Native Tribal Nations. Although these teams sometimes go by different names, review different types of deaths, or operate out of different agencies, these programs share their commitment to learning from the tragedies they face and help protect children in the future. CDR teams are comprised of members from multiple disciplines, agencies, and organizations, including law enforcement, child protective services, the judicial system (e.g., prosecutor), the medical examiner/coroner, public health, and medicine (e.g., pediatricians, first responders). Additionally, teams may choose to include other members depending on the type of death reviewed, community interests, and if mandated in legislation. CDR teams can operate at the state, regional, county, or city level, and each state has an identified CDR coordinator that supports CDR in a variety of ways, depending on funding and structure. Because the CDR team brings together expertise from multiple professionals, communication and collaborative decision making is essential. Teams are most successful when members have open, honest, and cooperative relationships and dialogue. Creating a safety culture is one strategy employed by teams in high-stress, high-consequence work settings to help improve decision making, engage in continuous learning, and advocate for change. A team’s culture is made up of the artifacts and rituals that shape how members understand and participate in their work and environment (Schein, 2010). A safety culture is a facet of a team’s broader culture and is generally understood to be the product of the values, beliefs, and behaviors of its members. In a safety culture, those qualities orient the work toward safety as the prevailing priority for members, and there is a focus on team-based solutions that simultaneously improve decision-making and address workforce development (Commission to Eliminate Child Abuse and Neglect Fatalities, 2016; Cull, Rzepnicki, O''Day, & Epstein, 2013). Teams with a strong safety culture create a space for learning and improvement, particularly following errors or mistakes, and they understand the importance of systems-level thinking in problem solving efforts (Morath & Turnbull, 2005). Habits form the basis of a team’s culture. Culture is an implicit pattern of shared basic assumptions among a group of people (Schein, 2010), and it can be defined, measured and changed using assessment tools and targeted intervention. Culture lives in the implicit routines people enact to problem solve, it underlies how team members “get work done around here.” Safety critical settings such as aviation and healthcare use a set of team-based strategies to develop and maintain the kinds of habits that support a safety culture, such as huddles to help plan ahead, debriefings to promote continued learning, and structured tools to facilitate communication. In a safety culture, safe and engaged teams practice six habits: 1. Spend time identifying what could go wrong. 2. Talk about mistakes and ways to learn from them. 3. Test change in everyday work activities. 4. Develop an understanding of “who knows what” and communicate clearly. 5. Appreciate colleagues and their unique skills. 6. Make candor and respect a precondition to teamwork. Using these habits, teams in a safety culture plan forward, reflect back, test change, communicate clearly, appreciate their colleagues, and manage professionalism. Building a culture of safety in high-risk, high-stress environments takes time, intentionality, effort, and quality data. By first understanding the current state of its culture, a team can then work to improve its work habits in ways that will support the specific needs of that team. High Reliability Organizing. The High Reliability Organization (HRO) paradigm and the related practice of mindful organizing are useful to understanding how teams develop a safety culture using team-based strategies. HROs are organizations that consistently achieve safe outcomes over long periods of time without significant failures (Weick, 1987). HROs exist in unforgiving social and political environments where failures can be catastrophic. HROs are seen in places like the deck of an aircraft carrier, nuclear power plants, and wildland firefighting where achieving low error rates has been largely attributed to the practice of mindful organizing (Vogus, Sutcliffe, & Weick, 2010). Mindful organizing is a team-based practice that allows teams to manage complexity and bias in decision-making (Sutcliffe, 2011) and is understood to represent the ways in which teams monitor, plan, innovate, learn, and support the members within it. The effectiveness of mindful organizing at the team level is driven by and linked to the psychological safety of that team (Renecle, Gracia, Tomas, & Peiró, 2020). Psychologically safe teams support candid feedback, identifying opportunities for learning and improvement, and approaches that treat errors as opportunities to get better - not as a time to punish (Edmondson, 1999). Therefore, effective mindful organizing practice depends on team-members feeling safe to speak up even when their opinion is not shared or may be unpopular (Renecle et al., 2020). Psychological safety is foundational to effective mindful organizing on teams. If a team member feels unsafe speaking up and pointing out concerns, important containment opportunities can be missed. Manageable threats, when not contained, can create cascading events that lead to catastrophic failures (Dekker, 2017). A growing body of literature is demonstrating better outcomes in safety critical settings when teams are observed to be psychologically safe and practicing mindful organizing, but the literature is much more limited on specific strategies that might create these conditions on teams where they are less well developed (Vogus et al., 2010). Curriculums such as Crew Resource Management (Kanki, Anca, & Chidester, 2019) and TeamSTEPPS (King et al., 2008) have been used broadly in healthcare and aviation to support team behaviors that promote psychological safety and are consistent with mindful organizing practice. Generally speaking, these curriculums provide teams with basic strategies that support teamwork and communication and create the conditions needed for mindful organizing to develop. Examples of these strategies that are currently used broadly across safety-critical industries are team cross-monitoring, mutual support, task assistance, and structured debriefing. Even less research has been done exploring how to assess and build mindful organizing and psychological safety in CDR teams using these kinds of tools. Given the high-consequence nature of CDR work, more research is sorely needed in this area. Cognitive Bias. It is well established that humans working in complex settings bring an array of cognitive biases to their work that affect decision-making (Dekker, 2017; Epstein, Schlueter, Gracey, Chandrasekhar, & Cull, 2015). Biases can be explicit and implicit, and they can rest both within individuals and within systems. Some cognitive biases, such as confirmation bias, have been shown to impact the way professionals assess risk and decision-making in child protection work (Spratt, Devaney, & Hayes, 2015); it seems likely that CDR teams might be vulnerable to similar kinds of biases. Confirmation bias demonstrates that if an individual holds an assumption about a situation or person, they will seek out and be persuaded by data that confirms their assumptions. Similarly, that individual is likely to reject data that is disconfirming to the bias, finding it less persuasive. Confirmation bias can be identity-based, such that if a child welfare professional consciously (or unconsciously) believes that men are more likely to abuse their children, they will be persuaded by information that supports this bias. This could impact how they assess risk in the case – perhaps underestimating risk for female caregivers and rating behaviors as more dangerous in cases with male caregivers. CDR teams that hold a similar bias may over-emphasize the importance of male-perpetrator violence risk in their recommendations following a review, at the risk of potentially ignoring or minimizing other findings that contribute to systems-level risk. Many cognitive biases are inextricably connected to our understanding of others, and they impact the decisions that systems – and the professionals within them – make in their work (Drake et al., 2011). A number of studies have demonstrated the extent of race-based disproportionality within child welfare and other public serving systems. For example, Black children make up 13.71% of the general population but represent 22.75% of children in foster care (Annie E. Casey Foundation, 2018). Similarly, American Indian/Alaska Native children account for less than 1% of the population, yet they made up 2.4% of children in foster care. Detlaff and colleagues (2020; 2010) discuss the internal and external factors that lead to racial disproportionality within the child welfare system, including cultural bias within and outside of the agency, a fearful agency climate, communication barriers, ineffective service delivery, and workforce issues. Racial bias in public child welfare systems exists at many levels and from many individuals who interact with child protection systems, from initial maltreatment reports, substantiation of maltreatment allegations, service provision, home removal, and out-of-home care admissions and exits. Bias at all of these levels likely contribute to the long-standing overrepresentation of Black and American Indian/Alaska Native children in the child welfare system (Dettlaff et al., 2011; Kim, Chenot, & Ji, 2011; Wulczyn & Lery, 2007). There has been less empirical work done on the presence of bias in CDR teams. More work is needed to better understand the nature and impact of these potential biases, but it seems likely that similar cognitive biases known to be present in child welfare professionals would be shared by professionals on CDR teams. Enhancing the mindful organizing and psychological safety of these teams may help to protect against team-based bias in decision making. Description of research plan/intervention. Team expertise. The Center for Innovation in Population Health at the University of Kentucky (IPH Center) has extensive experience and expertise related to work with fatality review teams and development and validation of organizational measurement tools. The Safe Systems team (Jordan Constantine, Michael Cull, Tiffany Lindsey, and Elizabeth Riley) at the IPH Center is the primary academic partner and source of technical support for the National Partnership for Child Safety (NPCS), the first member-owned quality improvement collaborative in public child welfare in the United States. The NPCS is grounded in the sciences of safety, improvement, and implementation, the NPCS’ mission is to improve child safety and prevent child maltreatment fatalities by strengthening families and promoting innovations in child protection. Specifically, the NPCS aims to improve the safety and quality of care within the public child welfare system by sharing and using data around critical incident and child fatality reviews, as well as sharing and using data from organizational assessments to better understand and promote a safety culture within these jurisdictions. As of November 2021, the NPCS was comprised of 26 members jurisdictions (public child welfare agencies from across the country), 13 of whom have fully implemented systemic child fatality review processes and 13 of whom are actively working to implement this process in their systems. The Safe Systems team works directly with child fatality review teams across the country to train and support jurisdictions in using a psychologically safe and systems-focused review process that helps teams to collect actionable data from these reviews to promote systems change. Members of the Safe Systems team have extensive experience in child fatality review processes, including prior experience leading CDRs in their own systems and now working to support CDR teams in child fatality reviews across the NPCS. Our deep understanding of child fatality reviews, from a variety of perspectives, leaves us well positioned to work directly with CDR teams in Kentucky. In addition, the Safe Systems team has developed a standardized safety culture survey for use in child welfare systems, and this team has supported the creation, implementation, and use of safety culture survey data in NPCS jurisdictions around the country. This team has facilitated 15 implementations of the safety culture survey in 10 child welfare jurisdictions in the NPCS, collecting data on over 15,000 child welfare professionals. The standard safety culture survey used in the NPCS measures constructs like emotional exhaustion, workplace connectedness, mindful organizing, and psychological safety. However, the Safe Systems team partners with each individual jurisdiction using the survey to adapt and add constructs to the assessment based on the needs of that particular jurisdiction. This process of individualized assessment development facilitated by the Safe Systems team ensures that the tools used in the safety culture survey have a strong empirical grounding while also allowing the entire assessment to be flexibly applied and responsive. Members of the team also partner closely with jurisdictions to analyze, understand, and use their data to drive systems change. The experience of the team in developing and using data from the safety culture survey in child welfare jurisdictions will translate well into validating a safety culture survey specifically for use in CDR teams to better understand team dynamics, communication patterns, and decision making to move these teams towards a culture of safety. The Safe Systems team will also receive support and consultation from the National Center for Fatality Review and Prevention (NCFPR) and will receive support from additional IPH center staff and faculty on this proposed project. The NCFPR is the technical support and data center serving CDR and Fetal and Infant Mortality Review (FIMR) programs across the country, and it offers technical assistance, training, and support with strategic planning to help support teams to develop, implement, and sustain prevention focused CDR processes. Elizabeth Riley, Ph.D. (PI) leads work on designing and implementing the safety culture survey throughout the NPCS; she will be largely responsible for design of the safety culture survey within CDR teams for this project. Michael Cull, Ph.D. (Co-I) is the leader of the Safe Systems Team at the University of Kentucky IPH Center; he will share responsibility with Dr. Riley for overall project conceptualization and implementation. Tiffany Lindsey, Ed.D. and Jordan Constantine, M.A. lead the majority of the work training and facilitating systems-focused child fatality review processes; they will interact most directly with pilot CDR teams to better understand their processes around decision making and team dynamics.