The United States railroad industry has experienced large growth over the last ten years. A growing amount of rail traffic across the country has caused the rail industry to refocus funds and efforts on maintenance of track infrastructure. Unfortunately, in areas with poor drainage conditions or high plasticity soils, ballast fouling and the subsequent stiffness degradation result in reductions in track performance and lead to major maintenance costs. This paper presents the results on an experimental study of the relationship of ballast stiffness and clay fouling. The study made use of a box testing system and seismic wave measurements to evaluate shear wave velocities through a ballast structure with varying levels of void replacement by fine-grained clay foul material. From these shear wave velocity measurements, shear modulus values were obtained for the ballast. The shear modulus values show how the increased severity of fouling and loading of the ballast initially results in stiffness enhancement, but eventually lead to stiffness degradation at a critical range of void volume replacement ratio.