Investigating adjudicator bias in concert band evaluations: An application of the Many-Facets Rasch Model

D. Gregory Springer, Kelly D. Bradley

Research output: Contribution to journalArticlepeer-review

5 Scopus citations

Abstract

Prior research indicates mixed findings regarding the consistency of adjudicators’ ratings at large ensemble festivals, yet the results of these festivals have strong impacts on the perceived success of instrumental music programs and the perceived effectiveness of their directors. In this study, Rasch modeling was used to investigate the potential influence of adjudicators on performance ratings at a live large ensemble festival. Evaluation forms from a junior high school concert band festival adjudicated by a panel of three expert judges were analyzed using the Many-Facets Rasch Model. Analyses revealed several trends. First, the use of assigning “half points” between adjacent response options on the 5-point rating scale resulted in redundancy and measurement noise. Second, adjudicators provided relatively similar ratings for conceptually distinct criteria, which could be evidence of a halo effect. Third, although all judges demonstrated relatively lenient ratings overall, one judge provided more severe ratings as compared to peers. Finally, an exploratory interaction analysis among the facets of judges and bands indicated the presence of rater-mediated bias. Implications for music researchers and ensemble adjudicators are discussed in the context of ensemble performance evaluations, and a measurement framework that can be applied to other aspects of music performance evaluations is introduced.

Original languageEnglish
Pages (from-to)377-393
Number of pages17
JournalMusicae Scientiae
Volume22
Issue number3
DOIs
StatePublished - Sep 1 2018

Bibliographical note

Publisher Copyright:
© The Author(s) 2017.

Keywords

  • Adjudicator bias
  • Many-Facets Rasch Model
  • Rasch model
  • concert band
  • measurement
  • performance evaluation
  • rater functioning
  • reliability

ASJC Scopus subject areas

  • Experimental and Cognitive Psychology
  • Music

Fingerprint

Dive into the research topics of 'Investigating adjudicator bias in concert band evaluations: An application of the Many-Facets Rasch Model'. Together they form a unique fingerprint.

Cite this