Radar Velocity Transformer: Single-scan Moving Object Segmentation in Noisy Radar Point Clouds

The awareness about moving objects in the surroundings of a self-driving vehicle is essential for safe and reliable autonomous navigation. The interpretation of LiDAR and camera data achieves exceptional results but typically requires to accumulate and process temporal sequences of data in order to extract motion information. In contrast, radar sensors, which are already installed in most recent vehicles, can overcome this limitation as they directly provide the Doppler velocity of the detections and, hence incorporate instantaneous motion information within a single measurement. In this paper, we tackle the problem of moving object segmentation in noisy radar point clouds. We also consider differentiating parked from moving cars, to enhance scene understanding. Instead of exploiting temporal dependencies to identify moving objects, we develop a novel transformer-based approach to perform single-scan moving object segmentation in sparse radar scans accurately. The key to our Radar Velocity Transformer is to incorporate the valuable velocity information throughout each module of the network, thereby enabling the precise segmentation of moving and non-moving objects. Additionally, we propose a transformer-based upsampling, which enhances the performance by adaptively combining information and over-coming the limitation of interpolation of sparse point clouds. Finally, we create a new radar moving object segmentation benchmark based on the RadarScenes dataset and compare our approach to other state-of-the-art methods. Our network runs faster than the frame rate of the sensor and shows superior segmentation results using only single-scan radar data.

  • Published in:
    IEEE International Conference on Robotics and Automation
  • Type:
    Inproceedings
  • Authors:
    Zeller, Matthias; Sandhu, Vardeep S.; Mersch, Benedikt; Behley, Jens; Heidingsfeld, Michael; Stachniss, Cyrill
  • Year:
    2023

Citation information

Zeller, Matthias; Sandhu, Vardeep S.; Mersch, Benedikt; Behley, Jens; Heidingsfeld, Michael; Stachniss, Cyrill: Radar Velocity Transformer: Single-scan Moving Object Segmentation in Noisy Radar Point Clouds, IEEE International Conference on Robotics and Automation, 2023, https://ieeexplore.ieee.org/document/10161152, Zeller.etal.2023a,

Associated Lamarr Researchers

lamarr institute person Stachniss Cyrill e1663922306234 - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Prof. Dr. Cyrill Stachniss

Principal Investigator Embodied AI to the profile