To Reverse Item Orientation or Not to Reverse Item Orientation, That Is the Question

David M. Dueber, Michael D. Toland, John Eric Lingat, Abigail M.A. Love, Chen Qiu, Rongxiu Wu, Alan V. Brown

Research output: Contribution to journalArticlepeer-review

17 Scopus citations

Abstract

To investigate the effect of using negatively oriented items, we wrote semantic reversals of the items in the Rosenberg Self Esteem Scale, UCLA Loneliness Scale, and the General Belongingness Scale and used them to create four experimental conditions. Participants (N = 2,019) were recruited through Amazon’s Mechanical Turk. Data were assessed for dimensionality, item functioning, instrument properties, and associations with other variables. Regarding dimensionality, although a two-factor model (positively vs. negatively oriented factors) exhibits better fit than a unidimensional model across all conditions, bifactor indices were used to argue that a unidimensional interpretation of the data can be employed. With respect to item functioning, factor loadings were found to be nearly invariant across conditions, but thresholds were not. Concerning instrument properties, inclusion of negatively oriented items results in lower mean scores and higher score variances. Instruments with both positively and negatively oriented items demonstrated lower reliability estimates than those with only one orientation. For associations with other variables, path coefficients in a model where loneliness mediates the effects of belongingness on life satisfaction and self-esteem were found to vary across conditions. Findings suggest that negatively oriented items have minor impact on instrument quality, but influence measurement model and path coefficients.

Original languageEnglish
Pages (from-to)1422-1440
Number of pages19
JournalAssessment
Volume29
Issue number7
DOIs
StatePublished - Oct 2022

Bibliographical note

Publisher Copyright:
© The Author(s) 2021.

Keywords

  • bifactor
  • confirmatory factor analysis, CFA
  • dimensionality
  • item orientation
  • item wording
  • measurement invariance
  • psychological measures

ASJC Scopus subject areas

  • Clinical Psychology
  • Applied Psychology

Fingerprint

Dive into the research topics of 'To Reverse Item Orientation or Not to Reverse Item Orientation, That Is the Question'. Together they form a unique fingerprint.

Cite this