Foundation Models Permit Retinal Layer Segmentation Across OCT Devices

The segmentation of retinal layers in images from optical coherence tomography (OCT) is an important step in ophthalmological diagnosis and disease monitoring. Current CNN-based models perform well on images from the same OCT scanner on which they have been trained, but their performance can degrade drastically when images are acquired with other devices. We present the first method for OCT layer segmentation that builds on recent Vision Transformer (ViT) foundation models. We demonstrate that, compared to a state-of-the-art CNN approach, doing so significantly improves their ability to generalize to devices for which no training data was available. This highlights the potential of foundation models to enable more robust medical image analysis. We also analyze the effect of using different foundation models. Notably, more generic foundation models from computer vision permitted better generalization than an equally large foundation model that was specifically trained for OCT analysis.

  • Published in:
    German Conference on Pattern Recognition
  • Type:
    Inproceedings
  • Authors:
    Morelle, Olivier; Schultz, Thomas
  • Year:
    2024

Citation information

Morelle, Olivier; Schultz, Thomas: Foundation Models Permit Retinal Layer Segmentation Across OCT Devices, German Conference on Pattern Recognition, 2024, Morelle.Schultz.2024a,

Associated Lamarr Researchers

lamarr institute person Schultz Thomas scaled e1663922506873 - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Prof. Dr. Thomas Schultz

Principal Investigator Life Sciences to the profile