Automatic Hand Skeletal Shape Estimation from Radiographs

Radu Paul Mihail, Gongbo Liang, Nathan Jacobs

Research output: Contribution to journalArticlepeer-review

17 Scopus citations


Rheumatoid arthritis (RA) is an autoimmune disease whose common manifestation involves the slow destruction of joint tissue, a damage that is visible in a radiograph. Over time, this damage causes pain and loss of functioning, which depends, to some extent, on the spatial deformation induced by the joint damage. Building an accurate model of the current deformation and predicting potential future deformations are the important components of treatment planning. Unfortunately, this is currently a time-consuming and labor-intensive manual process. To address this problem, we propose a fully automated approach for fitting a shape model to the long bones of the hand from a single radiograph. Critically, our shape model allows sufficient flexibility to be useful for patients in various stages of RA. Our approach uses a deep convolutional neural network to extract low-level features and a conditional random field (CRF) to support shape inference. Our approach is significantly more accurate than previous work that used hand-engineered features. We provide a comprehensive evaluation for various choices of network hyperparameters, as current best practices lack significantly in this domain. We evaluate the accuracy of our pipeline on two large datasets of hand radiographs and highlight the importance of the low-level features, the relative contribution of different potential functions in the CRF, and the accuracy of the final shape estimates. Our approach is nearly as accurate as a trained radiologist and, because it only requires a few seconds per radiograph, can be applied to large datasets to enable better modeling of disease progression.

Original languageEnglish
Article number8689041
Pages (from-to)296-305
Number of pages10
JournalIEEE Transactions on Nanobioscience
Issue number3
StatePublished - Jul 2019

Bibliographical note

Publisher Copyright:
© 2002-2011 IEEE.


  • Rheumatoid arthritis
  • conditional random field
  • convolutional neural network
  • radiograph

ASJC Scopus subject areas

  • Biotechnology
  • Bioengineering
  • Medicine (miscellaneous)
  • Biomedical Engineering
  • Pharmaceutical Science
  • Computer Science Applications
  • Electrical and Electronic Engineering


Dive into the research topics of 'Automatic Hand Skeletal Shape Estimation from Radiographs'. Together they form a unique fingerprint.

Cite this