Can Broad Inferences Be Drawn from Lottery Analyses of School Choice Programs? An Exploration of Appropriate Sensitivity Analyses

School choice programs continue to be controversial, spurring a number of researchers into evaluating them. When possible, researchers evaluate the effect of attending a school of choice using randomized designs to eliminate possible selection bias. Randomized designs are often thought of as the gold standard for research, but many circumstances can limit external validity of inferences from these designs in the context of school choice programs. In this article, we examine whether these limitations are applicable to previous evaluations of voucher, charter schools, magnet, and open-enrollment programs. We devise simple sensitivity analyses that researchers could conduct when analyzing lotteried programs to determine whether there are reasons to be cautious about the breadth of appropriate inferences.