Spatio-Temporal Consistent Mapping of Growing Plants for Agricultural Robots in the Wild

Tracking changes in growing plants is important for automating phenotyping and robots managing crops. In this paper, we propose a system that uses a 3D model of plants along crop rows to enable a robotic platform to localize itself even in the presence of heavy changes and deforming the model to adapt the scene description to the new measurements. In particular, we focus on consumer RGB-D cameras due to their cost-effectiveness and ease of deployment on real platforms. Our approach exploits modern deep-learning-based feature descriptors and geometric information to obtain matches between 3D points corresponding to temporally distant sessions. We then use the associations in a non-rigid registration pipeline to obtain the final result, an updated representation of the 3D model that reflects plant changes. Using a standard RGB-D sensor, we validate our approach on a real-world dataset recorded in a glasshouse. We obtain accurate 4D models of the plants and track the plant traits’ evolution over time. We show, through experiments, that our method is applicable to interpolate plant organs’ evolution, a helpful result for phenotypic trait measurement. We see our approach as a relevant step toward 4D reconstruction for robotic agriculture in the wild.