Concerns for the reliability and validity of the national stroke project stroke severity scale

Nada El Husseini, Kenneth J. Shea, Larry B. Goldstein

Research output: Contribution to journalArticlepeer-review

4 Scopus citations


Background: The National Stroke Project (NSP) was a retrospective cohort study of US Medicare beneficiaries hospitalized with stroke or transient ischemic attack (TIA). The NSP included a simple assessment of stroke severity (NSP-Stroke Scale, NSP-SS). Used for risk adjustment in outcome studies, the reliability and validity of the NSP-SS have not been assessed. We determined the reliability, concurrent and construct validity of theNSP-SS. Methods: The initial neurologic examinations of 100 consecutive patients hospitalized with ischemic stroke/TIA in a single academic medical center were reviewed. The NSP-SS was retrospectively scored twice by the same rater and independently by a second rater to assess reliability. The National Institutes of Health Stroke Scale (NIH-SS) was also scored retrospectively and used as the criterion standard for concurrent validity. Construct validity was based on discharge status. Results: The NSP-SS had moderate-substantial inter-rater (weighted kappa, κ w = 0.66, 95% CI 0.55-0.77) and intra-rater (κ w = 0.63, 95% CI 0.52-0.75) reliability. Correlation between NSP-SS and NIH-SS scores was moderate (Spearman r = 0.65, 95% CI 0.52-0.75, p < 0.0001) but some categorizations in the NSP-SS seemed inappropriate reflecting poor content validity. Each NSP-SS point was associated with a greater likelihood of poor outcome (OR = 2.1, 95% CI 1.1-3.7, p = 0.016). Based on dichotomized scores (NSP 0-2 and NIH-SS <6; mild deficits), the NSP-SS sensitivity was 70.9% (95% CI 57.9-81.2%), specificity 82.2% (95% CI 68.7-90.7%), likelihood ratio for severe stroke 4.0 (95% CI 2.1-7.6) and likelihood ratio for mild stroke 0.3 (95% CI 0.20-0.5). The dichotomized NSP-SS and NIH-SS similarly predicted poor outcome (NSP-SS >2, OR = 4.7, 95% CI 1.7-13.0, p = 0.003 vs. NIH-SS ≥6, OR = 4.4, 95% CI 1.5-13.0, p = 0.006) with excellent discrimination (C = 0.827 and 0.826, respectively). Conclusion: The NSP-SS has moderate-substantial reliability but poor content validity and poor to moderate concurrent validity as compared with the NIH-SS. In addition, it is not clear that the NSP-SS is easier to extract from medical records than the NIH-SS. Given this, and its other limitations, the utility of this scale for risk adjustment in future stroke outcome studies is questionable.

Original languageEnglish
Pages (from-to)426-430
Number of pages5
JournalCerebrovascular Diseases
Issue number5
StatePublished - Nov 2011


  • Stroke
  • Stroke outcome
  • Stroke scales

ASJC Scopus subject areas

  • Neurology
  • Clinical Neurology
  • Cardiology and Cardiovascular Medicine


Dive into the research topics of 'Concerns for the reliability and validity of the national stroke project stroke severity scale'. Together they form a unique fingerprint.

Cite this