Research, Evaluation, and Technical Assistance: Student Outcome Effects of the Appalachian Math and Science Partnership

Intellectual Merit Academics and policymakers increasingly emphasize that a key to improving educational outcomes in the United States is to enhance the quality of teachers. After many years of research that failed to show significant and systematic effects of inputs into the education process—such as school expenditures, teacher salaries, or pupil-teacher ratios on student outcomes—recent work illustrates that the quality of teachers is a significant factor in explaining student achievement. With increasing emphasis on teacher quality has come a penchant among policymakers to improve training for pre-service teachers; this is the element of the teacher production system most easily manipulated by lawmakers. While the attention to teacher training is important, it will be many years before today’s preservice teachers constitute a majority of the teaching workforce. Hence, the focus of teacher quality enhancement efforts in the near term must be in-service training for those already in the teacher workforce, typically called professional development (PD). One such initiative that falls in the latter category is that of the National Science Foundation’s Math and Science Partnerships (MSP). This project proposes to examine the Appalachian Math and Science Partnership (AMSP) effects on student learning. A goal of the partnership is to close the achievement gap in student math and science knowledge through AMSP activities. This study presents a statistical method for evaluating the effects of AMSP participation on student learning. This project will employ a research design that involves a two equation model to estimate the independent effects of AMSP participation on student outcomes. A two equation model is used because estimation is complicated by the fact that teachers and schools voluntarily choose to participate in AMSP. The first equation of the model corrects for the nonrandom nature of selection of schools and teachers into AMSP which, uncorrected, also can bias the estimates of the program effects. The second equation incorporates the first stage selection estimates and includes other independent variables that would be expected to influence test scores. Broader Impact This project is significant for two reasons. First, and most obviously, the project will directly address whether AMSP is improving student outcomes in participating Appalachian schools. But independently of specific results, this project also makes an important contribution in establishing a scientifically sound method for evaluating the effects of MSPs (and other PD programs) around the country. It is clear that policymakers think math and science outcomes should be enhanced in the United States. The National Science Foundation has funded many partnerships for math and science training. Such large investments of human and financial resources call for evaluation that reaches past teacher participation into student learning results. It is not sufficient to assume that federal dollars spent on innovative PD programs automatically increase student learning. Understanding these programs and whether they consistently generate positive returns for inservice professional development efforts is important in the national movement to improve teacher quality and student learning in math and science.