A comparison of recent nonparametric methods for testing effects in two-by-two factorial designs

Solomon W. Harrar, Fabrizio Ronchi, Luigi Salmaso

Research output: Contribution to journalReview articlepeer-review

13 Scopus citations


The two-way two-levels crossed factorial design is a commonly used design by practitioners at the exploratory phase of industrial experiments. The F-test in the usual linear model for analysis of variance (ANOVA) is a key instrument to assess the impact of each factor and of their interactions on the response variable. However, if assumptions such as normal distribution and homoscedasticity of errors are violated, the conventional wisdom is to resort to nonparametric tests. Nonparametric methods, rank-based as well as permutation, have been a subject of recent investigations to make them effective in testing the hypotheses of interest and to improve their performance in small sample situations. In this study, we assess the performances of some nonparametric methods and, more importantly, we compare their powers. Specifically, we examine three permutation methods (Constrained Synchronized Permutations, Unconstrained Synchronized Permutations and Wald-Type Permutation Test), a rank-based method (Aligned Rank Transform) and a parametric method (ANOVA-Type Test). In the simulations, we generate datasets with different configurations of distribution of errors, variance, factor's effect and number of replicates. The objective is to elicit practical advice and guides to practitioners regarding the sensitivity of the tests in the various configurations, the conditions under which some tests cannot be used, the tradeoff between power and type I error, and the bias of the power on one main factor analysis due to the presence of effect of the other factor. A dataset from an industrial engineering experiment for thermoformed packaging production is used to illustrate the application of the various methods of analysis, taking into account the power of the test suggested by the objective of the experiment.

Original languageEnglish
Pages (from-to)1649-1670
Number of pages22
JournalJournal of Applied Statistics
Issue number9
StatePublished - Jul 4 2019

Bibliographical note

Publisher Copyright:
© 2018, © 2018 Informa UK Limited, trading as Taylor & Francis Group.


  • Permutation
  • factorial design
  • power
  • rank methods
  • simulation

ASJC Scopus subject areas

  • Statistics and Probability
  • Statistics, Probability and Uncertainty


Dive into the research topics of 'A comparison of recent nonparametric methods for testing effects in two-by-two factorial designs'. Together they form a unique fingerprint.

Cite this