The challenge of separating effects of simultaneous education projects on student achievement

Xin Ma, Lingling Ma

Research output: Contribution to journalArticlepeer-review


When multiple education projects operate in an overlapping or rear-ended manner, it is always a challenge to separate unique project effects on schooling outcomes. Our analysis represents a first attempt to address this challenge. A three-level hierarchical linear model (HLM) was presented as a general analytical framework to separate program effects while taking into account the hierarchy in educational data. The HLM model was then applied to data from the Commonwealth Accountability Testing System that the State of Kentucky has implemented for years, in an attempt to separate the effects of two education projects aimed at improving mathematics and science education in the Appalachian region: the Appalachian Rural Systemic Initiative (ARSI) closely followed by the Appalachian Mathematics and Science Partnership (AMSP). Even though the HLM model successfully separated ARSI and AMSP effects, relevant statistical issues were discussed to improve future efforts in separating effects of simultaneous education projects on schooling outcomes.

Original languageEnglish
Pages (from-to)45-52
Number of pages8
JournalStudies in Educational Evaluation
Issue number1
StatePublished - Mar 2009

Bibliographical note

Funding Information:
Over the years, there have been federal interventions in Appalachia aimed to improve socioeconomic and educational conditions of the region. Major efforts started in 1995 with the Appalachian Rural Systemic Initiative (ARSI), a five-year capacity-building project sponsored by the National Science Foundation (NSF). ARSI concentrated on Appalachian school districts with at least 30% of students living in poverty. The goal of ARSI was to work with school districts to accelerate improved performance in mathematics and science for all students through high-quality, standards-based teaching supported by aligned and coherent local and regional educational efforts. ARSI sought to involve school districts in mathematics and science education reform through setting a “catalyst school” in each district to be the model of reform in mathematics and science for the rest of the district. Teachers from these schools were assigned to help other teachers in the district implement standards-based instruction and provide support for curriculum development. A “Community Engagement Team” was also created to reinforce school and community leaders of the importance to improve programs in mathematics, science, and technology. Because technology plays an important role in overcoming the regional isolation, ARSI strived to help both teachers and students use technology to improve skill levels in mathematics and science. Local facilities were developed to initiate and lead education reform in collaboration with universities and colleges in the region, with the intention to establish these facilities as centers for mathematics and science education reform even beyond the scope of NSF support. From 1996 to 2000, ARSI worked with catalyst schools from 52 school districts in 47 targeted counties, developing a strong network of committed and competent teacher partners who played an important role in education reform and community building in the region.


  • Simultaneous education projects
  • Statistical modeling of education data
  • Student achievement

ASJC Scopus subject areas

  • Education


Dive into the research topics of 'The challenge of separating effects of simultaneous education projects on student achievement'. Together they form a unique fingerprint.

Cite this