The Opportunity – VAWA 2013 Reauthorization Provides a Natural Experiment for Bystander Efficacy Evaluation

This methods paper describes the fractional factorial design within the context of an evaluation of campus bystander programming. Prior evaluations include relative program comparisons, but do not reflect campus implementation of specific aspects of prevention programming. Campuses use combinations of programs, delivered across modalities, audiences, intensity, and degrees of requirement. Bystander program evaluation, in a natural experiment, must consider combinations of these components. This evaluation offers a novel application of the fractional factorial framework, considering combinations of program components, as implemented, within a multi-campus quasi-experimental design. Leveraging qualitative data, cluster analysis provides an initial identification of bystander program component combinations, mapped to experimental conditions. SAS v9.4 PROC FACTEX constructs possible fractional factorial designs to estimate main effects. From key informant interviews, primary program components are determined: delivery method, level of skill-building, degree of requirement, and intended audience. A total of seven combinations are identified. Using presence or absence of program components, a partial factorial structure is identified, where component clusters comprised seven of the 2⁴ = 16 possible combinations. The smallest number of experimental conditions necessary for estimating these effects is determined with PROC FACTEX. Resulting designs indicate the necessary experimental conditions for determining bystander program component effectiveness available within this study. Our fractional factorial approach offers a novel strategy for planning an evaluation of several bystander program component combinations. This paper provides foundational elements needed to implement a bystander evaluation design, with the requisite emphasis on program components and relevant combinations, while optimizing number of participating campuses.