Agreement between veterinarians and three objective evaluation systems in naturally occurring equine lameness

BACKGROUND: Subjective lameness evaluations are critical components of equine musculoskeletal health assessments. Objective approaches can supplement diagnosis and may be preferred for specific cases and scientific purposes.

OBJECTIVES: Evaluate agreements between subjective evaluation of two veterinarians and standard clinical interpretations from outputs of an AI-based smartphone application (Sleip; AI-SPA), the inertial measurement unit (IMU) system Equinosis Q Lameness Locator (LL), and the IMU system Equisym (ES).

STUDY DESIGN: In vivo experiment.

METHODS: Twenty-five research horses (10-30 years) were evaluated on a straight-line trot. Limbs were independently graded and converted to an ordinal scale that, for objective systems, were converted from system-specific data outputs. Default settings and outputs for AI-SPA and LL were utilised to grade lameness while a manual process was developed for the ES. Pairwise agreement was calculated via weighted Cohen's κ, and agreement across rater types was calculated via Gwet's Agreement Coefficient 2 (GA2).

RESULTS: Objective evaluator agreement (GA2 = 0.84, 95% Confidence Interval (CI): 0.77-0.91) was higher than subjective evaluator agreement (GA2 = 0.73, 95% CI: 0.63-0.83) across all limbs. For all five evaluators/systems, overall forelimb agreement (GA2 = 0.82, 95% CI: 0.69-0.95) was greater than overall hindlimb agreement (GA2 = 0.73, 95% CI: 0.59-0.87). Pairwise agreement scores between the objective systems were often higher than those involving veterinary evaluators. The ES system often produced the highest agreement when compared with each rater individually.

MAIN LIMITATIONS: Horses were evaluated on a straight line only. Lameness diagnosis was limited to visual observation. Outcomes for each horse's four limbs were considered independent measurements.

CONCLUSIONS: This work highlights the utility of commercially available objective evaluation systems, including the more recent ES system. Hindlimb asymmetries had lower agreement regardless of evaluator type. Objective systems had higher agreements when compared with subjective straight-line veterinary examination. The ability to uniformly assess asymmetries may assist diagnosis when compared with subjective evaluation alone.