Performance of model-based vs. permutation tests in the HEALing (Helping to End Addiction Long-termSM) Communities Study, a covariate-constrained cluster randomized trial

Xiaoyu Tang, Timothy Heeren, Philip M. Westgate, Daniel J. Feaster, Soledad A. Fernandez, Nathan Vandergrift, Debbie M. Cheng

Research output: Contribution to journalArticlepeer-review

Abstract

Background: The HEALing (Helping to End Addiction Long-termSM) Communities Study (HCS) is a multi-site parallel group cluster randomized wait-list comparison trial designed to evaluate the effect of the Communities That Heal (CTH) intervention compared to usual care on opioid overdose deaths. Covariate-constrained randomization (CCR) was applied to balance the community-level baseline covariates in the HCS. The purpose of this paper is to evaluate the performance of model-based tests and permutation tests in the HCS setting. We conducted a simulation study to evaluate type I error rates and power for model-based and permutation tests for the multi-site HCS as well as for a subgroup analysis of a single state (Massachusetts). We also investigated whether the maximum degree of imbalance in the CCR design has an impact on the performance of the tests. Methods: The primary outcome, the number of opioid overdose deaths, is count data assessed at the community level that will be analyzed using a negative binomial regression model. We conducted a simulation study to evaluate the type I error rates and power for 3 tests: (1) Wald-type t-test with small-sample corrected empirical standard error estimates, (2) Wald-type z-test with model-based standard error estimates, and (3) permutation test with test statistics calculated by the difference in average residuals for the two groups. Results: Our simulation results demonstrated that Wald-type t-tests with small-sample corrected empirical standard error estimates from the negative binomial regression model maintained proper type I error. Wald-type z-tests with model-based standard error estimates were anti-conservative. Permutation tests preserved type I error rates if the constrained space was not too small. For all tests, the power was high to detect the hypothesized 40% reduction in opioid overdose deaths for the intervention vs. comparison group both for the overall HCS and the subgroup analysis of Massachusetts (MA). Conclusions: Based on the results of our simulation study, the Wald-type t-test with small-sample corrected empirical standard error estimates from a negative binomial regression model is a valid and appropriate approach for analyzing cluster-level count data from the HEALing Communities Study. Trial registration: ClinicalTrials.gov http://www.clinicaltrials.gov;

Original languageEnglish
Article number762
JournalTrials
Volume23
Issue number1
DOIs
StatePublished - Dec 2022

Bibliographical note

Funding Information:
This research was supported by the National Institutes of Health and the Substance Abuse and Mental Health Services Administration through the NIH HEAL (Helping to End Addiction Long-term) Initiative under award numbers UM1DA049394, UM1DA049406, UM1DA049412, UM1DA049415, and UM1DA049417 (ClinicalTrials.gov Identifier: NCT04111939). SM

Publisher Copyright:
© 2022, The Author(s).

Keywords

  • Cluster randomized trials
  • Covariate-constrained randomization
  • Model-based tests
  • Negative binomial regression
  • Permutation tests

ASJC Scopus subject areas

  • Medicine (miscellaneous)
  • Pharmacology (medical)

Fingerprint

Dive into the research topics of 'Performance of model-based vs. permutation tests in the HEALing (Helping to End Addiction Long-termSM) Communities Study, a covariate-constrained cluster randomized trial'. Together they form a unique fingerprint.

Cite this