Open-World Semantic Segmentation Including Class Similarity
Interpreting camera data is key for autonomously acting systems, such as autonomous vehicles. Vision systems that operate in real-world environments must be able to understand their surroundings and need the ability to deal with novel situations. This paper tackles open-world semantic segmentation, i.e., the variant of interpreting image data in which objects occur that have not been seen during training. We propose a novel approach that performs accurate closed-world semantic segmentation and, at the same time, can identify new categories without requiring any additional training data. Our approach additionally provides a similarity measure for every newly discovered class in an image to a known category, which can be useful information in downstream tasks such as planning or mapping. Through extensive experiments, we show that our model achieves state-of-the-art results on classes known from training data as well as for anomaly segmentation and can distinguish
between different unknown classes.
- Published in:
IEEE Conference on Computer Vision and Pattern Recognition - Type:
Inproceedings - Authors:
Sodano, Matteo; Magistri, Federico; Nunes, Lucas; Behley, Jens; Stachniss, Cyrill - Year:
2024
Citation information
Sodano, Matteo; Magistri, Federico; Nunes, Lucas; Behley, Jens; Stachniss, Cyrill: Open-World Semantic Segmentation Including Class Similarity, IEEE Conference on Computer Vision and Pattern Recognition, 2024, https://www.computer.org/csdl/proceedings-article/cvpr/2024/530000d184/20hT6BA3c9q, Sodano.etal.2024a,
@Inproceedings{Sodano.etal.2024a,
author={Sodano, Matteo; Magistri, Federico; Nunes, Lucas; Behley, Jens; Stachniss, Cyrill},
title={Open-World Semantic Segmentation Including Class Similarity},
booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
url={https://www.computer.org/csdl/proceedings-article/cvpr/2024/530000d184/20hT6BA3c9q},
year={2024},
abstract={Interpreting camera data is key for autonomously acting systems, such as autonomous vehicles. Vision systems that operate in real-world environments must be able to understand their surroundings and need the ability to deal with novel situations. This paper tackles open-world semantic segmentation, i.e., the variant of interpreting image data in which objects occur that have not been seen during...}}