Skin Lesion Segmentation Improved by Transformer-Based Networks with Inter-scale Dependency Modeling

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


Melanoma, a dangerous type of skin cancer resulting from abnormal skin cell growth, can be treated if detected early. Various approaches using Fully Convolutional Networks (FCNs) have been proposed, with the U-Net architecture being prominent To aid in its diagnosis through automatic skin lesion segmentation. However, the symmetrical U-Net model’s reliance on convolutional operations hinders its ability to capture long-range dependencies crucial for accurate medical image segmentation. Several Transformer-based U-Net topologies have recently been created to overcome this limitation by replacing CNN blocks with different Transformer modules to capture local and global representations. Furthermore, the U-shaped structure is hampered by semantic gaps between the encoder and decoder. This study intends to increase the network’s feature re-usability by carefully building the skip connection path. Integrating an already calculated attention affinity within the skip connection path improves the typical concatenation process utilized in the conventional skip connection path. As a result, we propose a U-shaped hierarchical Transformer-based structure for skin lesion segmentation and an Inter-scale Context Fusion (ISCF) method that uses attention correlations in each stage of the encoder to adaptively combine the contexts from each stage to mitigate semantic gaps. The findings from two skin lesion segmentation benchmarks support the ISCF module’s applicability and effectiveness. The code is publicly available at

Original languageEnglish
Title of host publicationMachine Learning in Medical Imaging - 14th International Workshop, MLMI 2023, Held in Conjunction with MICCAI 2023, Proceedings
EditorsXiaohuan Cao, Xi Ouyang, Xuanang Xu, Islem Rekik, Zhiming Cui
Number of pages10
StatePublished - 2024
Event14th International Workshop on Machine Learning in Medical Imaging, MLMI 2023 - Vancouver, Canada
Duration: Oct 8 2023Oct 8 2023

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume14348 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference14th International Workshop on Machine Learning in Medical Imaging, MLMI 2023

Bibliographical note

Publisher Copyright:
© 2024, The Author(s), under exclusive license to Springer Nature Switzerland AG.


  • Deep learning
  • Inter-scale context fusion
  • Skin lesion segmentation
  • Transformer

ASJC Scopus subject areas

  • Theoretical Computer Science
  • General Computer Science


Dive into the research topics of 'Skin Lesion Segmentation Improved by Transformer-Based Networks with Inter-scale Dependency Modeling'. Together they form a unique fingerprint.

Cite this