Learning depth vision-based personalized robot navigation from dynamic demonstrations in virtual reality

For the best human-robot interaction experience, the robot’s navigation policy should take into account personal preferences of the user. In this paper, we present a learning framework complemented by a perception pipeline to train a depth vision-based, personalized navigation controller from user demonstrations. Our virtual reality interface enables the demonstration of robot navigation trajectories under motion of the user for dynamic interaction scenarios. The novel perception pipeline enrolls a variational autoencoder in combination with a motion predictor. It compresses the perceived depth images to a latent state representation to enable efficient reasoning of the learning agent about the robot’s dynamic environment. In a detailed analysis and ablation study, we evaluate different configurations of the perception pipeline. To further quantify the navigation controller’s quality of personalization, we develop and apply a novel metric to measure preference reflection based on the Fréchet Distance. We discuss the robot’s navigation performance in various virtual scenes and demonstrate the first personalized robot navigation controller that solely relies on depth images. A supplemental video highlighting our approach is available online.

  • Published in:
    IEEE/RSJ International Conference on Intelligent Robots and Systems
  • Type:
    Inproceedings
  • Authors:
    de Heuvel, Jorge; Corral, Nathan; Kreis, Benedikt; Conradi, Jacobus; Driemel, Anne; Bennewitz, Maren
  • Year:
    2023

Citation information

de Heuvel, Jorge; Corral, Nathan; Kreis, Benedikt; Conradi, Jacobus; Driemel, Anne; Bennewitz, Maren: Learning depth vision-based personalized robot navigation from dynamic demonstrations in virtual reality, IEEE/RSJ International Conference on Intelligent Robots and Systems, 2023, 6757--6764, https://arxiv.org/abs/2210.01683, Heuvel.etal.2023c,

Associated Lamarr Researchers

lamarr institute person Driemel Anne e1664271117365 - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Prof. Dr. Anne Driemel

Principal Investigator Hybrid ML to the profile
lamarr institute person Bennewitz Maren - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Prof. Dr. Maren Bennewitz

Principal Investigator Embodied AI to the profile