Segmentation Distortion: Quantifying Segmentation Uncertainty Under Domain Shift via the Effects of Anomalous Activations

Domain shift occurs when training U-Nets for medical image segmentation with images from one device, but applying them to images from a different device. This often reduces accuracy, and it poses a challenge for uncertainty quantification, when incorrect segmentations are produced with high confidence. Recent work proposed to detect such failure cases via anomalies in feature space: Activation patterns that deviate from those observed during training are taken as an indication that the input is not handled well by the network, and its output should not be trusted. However, such latent space distances primarily detect whether images are from different scanners, not whether they are correctly segmented. Therefore, we propose a novel segmentation distortion measure for uncertainty quantification. It uses an autoencoder to make activations more similar to those that were observed during training, and propagates the result through the remainder of the U-Net. We demonstrate that the extent to which this affects the segmentation correlates much more strongly with segmentation errors than distances in activation space, and that it quantifies uncertainty under domain shift better than entropy in the output of a single U-Net, or an ensemble of U-Nets.

  • Published in:
    Medical Image Computing and Computer Assisted Intervention
  • Type:
    Inproceedings
  • Authors:
    Lennartz, Jonathan; Schultz, Thomas
  • Year:
    2023

Citation information

Lennartz, Jonathan; Schultz, Thomas: Segmentation Distortion: Quantifying Segmentation Uncertainty Under Domain Shift via the Effects of Anomalous Activations, Medical Image Computing and Computer Assisted Intervention, 2023, 316--325, https://www.researchgate.net/publication/374348476_Segmentation_Distortion_Quantifying_Segmentation_Uncertainty_Under_Domain_Shift_via_the_Effects_of_Anomalous_Activations, Lennartz.Schultz.2023a,

Associated Lamarr Researchers

lamarr institute person Schultz Thomas scaled e1663922506873 - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Prof. Dr. Thomas Schultz

Principal Investigator Life Sciences to the profile