Analysis of Covariance in Randomized Experiments with Heterogeneity of Regression and a Random Covariate: The Variance of the Estimated Treatment Effect at Selected Covariate Values

Researchers detecting heterogeneity of regression in a treatment outcome study including a covariate and random assignment to groups often want to investigate the simple treatment effect at the sample grand mean of the covariate and at points one standard deviation above and below that mean. The estimated variances of the simple treatment effect that have traditionally been used in such tests were derived under the assumption that the covariate values were fixed constants. We derive results appropriate for a two-group experiment that instead presume the covariate is a normally distributed random variable. A simulation study is used to confirm the validity of the analytical results and to compare error estimates and confidence intervals based on these results with those based on assuming a fixed covariate. Discrepancies between estimates for fixed and random covariates of the variability of treatment effects can be substantial. However, in situations where the extent of heterogeneity of regression is like that typically reported, presuming the covariate is random rather than fixed will generally result in only a modest increase in estimated standard errors, and in some circumstances can even result in a smaller estimated standard error. We illustrate the new methods with an empirical data set.