Advit: Vision Transformer On Multi-Modality Pet Images For Alzheimer Disease Diagnosis

Xin Xing, Gongbo Liang, Yu Zhang, Subash Khanal, Ai Ling Lin, Nathan Jacobs

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

We present a new model trained on multi-modalities of Positron Emission Tomography images (PET-AV45 and PET-FDG) for Alzheimer's Disease (AD) diagnosis. Unlike the conventional methods using multi-modal 3D/2D CNN architecture, our design replaces the Convolutional Neural Net-work (CNN) by Vision Transformer (ViT). Considering the high computation cost of 3D images, we firstly employ a 3D-to-2D operation to project the 3D PET images into 2D fusion images. Then, we forward the fused multi-modal 2D images to a parallel ViT model for feature extraction, followed by classification for AD diagnosis. For evaluation, we use PET images from ADNI. The proposed model outperforms several strong baseline models in our experiments and achieves 0.91 accuracy and 0.95 AUC.

Original languageEnglish
Title of host publicationISBI 2022 - Proceedings
Subtitle of host publication2022 IEEE International Symposium on Biomedical Imaging
ISBN (Electronic)9781665429238
DOIs
StatePublished - 2022
Event19th IEEE International Symposium on Biomedical Imaging, ISBI 2022 - Kolkata, India
Duration: Mar 28 2022Mar 31 2022

Publication series

NameProceedings - International Symposium on Biomedical Imaging
Volume2022-March
ISSN (Print)1945-7928
ISSN (Electronic)1945-8452

Conference

Conference19th IEEE International Symposium on Biomedical Imaging, ISBI 2022
Country/TerritoryIndia
CityKolkata
Period3/28/223/31/22

Bibliographical note

Publisher Copyright:
© 2022 IEEE.

Keywords

  • Alzheimer's Disease
  • Multi-modalities
  • PET image
  • Vision Transformer (ViT)

ASJC Scopus subject areas

  • Biomedical Engineering
  • Radiology Nuclear Medicine and imaging

Fingerprint

Dive into the research topics of 'Advit: Vision Transformer On Multi-Modality Pet Images For Alzheimer Disease Diagnosis'. Together they form a unique fingerprint.

Cite this