Main Effects Analysis in Clinical Research: Statistical Guidelines for Disaggregating Treatment Groups

Treatment outcome research generally relies on main effects analysis of variance to determine whether treatments are differentially effective. Bryk and Raudenbush (1988) developed a decision strategy for disaggregating treatment groups under conditions of heterogeneity of variance. There is, however, reason to consider disaggregating main effects even when this assumption is not violated. The potential statistical significance of disaggregation can be shown to be a function of the reliability of the dependent measure. With this reliability, residual variance can be partitioned into a systematic (individual differences) component and a random error component. It is then possible to calculate an F test of the ratio of these variances. When this F is statistically significant and the proportion of within-cell systematic variance to total variance is large, disaggregation should be undertaken to search for important individual or treatment difference variables (i.e., interactions).