Panoptic Segmentation with Partial Annotations for Agricultural Robots

A detailed analysis of agricultural fields is key toward reducing the use of agrochemicals to achieve a more sustainable crop production. To this end, agricultural robots equipped with vision-based systems offer the potential to detect individual plants in the field automatically. This capability enables targeted management actions in the field, effectively reducing the amount of agrochemicals. A primary target of such vision systems is to perform a panoptic segmentation, combining the task of semantic and instance segmentation. Recent methods use neural networks for this task, which typically have to be trained on densely annotated images containing the required ground truth information for each pixel. Gathering these dense annotations is generally daunting and requires domain experts’ knowledge in the agricultural domain. In this paper, we propose a method to effectively reduce the annotation bottleneck and yet achieve high performance using partial annotations. These partial annotations contain ground truth information only for a subset of pixels per image and are thus much faster to obtain than dense annotations. We propose a novel set of losses that exploit measures from vector fields used in physics, i.e., divergence and curl, to effectively supervise predictions without ground truth annotations. The experimental evaluation shows that our approach outperforms several state-of-the-art methods targeting to reduce the amount of annotations.