EUVIMEDEUVIMED entdecken
CrossrefInfokarte

Systematic Failure of Vision Transformers in Imbalanced Skin Lesion Classification

Serra Aksoy, Pinar Demircioglu, Ismail Bogrekci

Dermato · 2026-06-11 · 6 (2) · 22

DOI: 10.3390/dermato6020022 ↗
AbstractBibliografische Angaben

Abstract

Background/Objectives: Vision Transformers (ViTs) have demonstrated impressive performance in dealing with large-scale natural image datasets. They have started to be used in medical image classification problems as well. However, how they behave under real-world conditions, such as data scarcity and extreme class imbalance, has not been well investigated. In this study, we examine the feasibility of using a standard Vision Transformer Base model that learned from scratch how to classify skin lesion images into multiple classes using the ISIC 2019 dataset. Methods: The Vision Transformer architecture was trained from scratch using stratified splitting of the data, class-balanced cross-entropy loss, multi-seed initialization, and control of hyperparameters such as patch size and dropout rate. The evaluation of the Vision Transformer architecture was performed using a hold-out test set with metrics such as accuracy, macro-F1, weighted-F1, and analysis of the confusion matrix. Results: Across all configurations, the training exhibited substantial instability and consistent overfitting behavior, with an average accuracy gap between validation and test sets of 22.7%. Test accuracy ranged from 8.0% to 37.8%, showing high sensitivity to initialization. For minority classes, the F1-score remained very low (F1 < 0.05) even though the classes were balanced in the loss function. Conclusions: The results indicate that a standard ViT-Base model trained from scratch can exhibit pronounced instability and a tendency toward majority-class bias when applied to multi-class skin lesion classification under conditions of extreme class imbalance and data scarcity. The findings point to the limitations of using simple transformer models without pre-training or other forms of inductive bias in scarce data settings.

Wissen, das zu Ihrer Arbeit passt.

Vom kostenlosen Einstieg über die persönliche Recherche bis zur vollständigen Research Suite.

Tarife ansehen →

Sind alle Verlagsvolltexte enthalten?

Nein. Bibliografische Nachweise, Abstracts und Volltexte sind unterschiedliche Inhalte. Ob ein Volltext verfügbar ist, hängt von der Quelle, Open Access und Ihren vorhandenen Lizenzen ab. Ein EUVIMED-Tarif ersetzt keine zusätzlichen Verlagsrechte.

Tarife ansehen →