Non-invasive diagnosis of nutrient deficiencies in winter wheat and winter rye using UAV-based RGB images

Better matching of the timing and amount of fertilizer inputs to plant requirements will improve nutrient use efficiency and crop yields and could reduce negative environmental impacts. Deep learning can be a powerful digital tool for on-site, real-time, non-invasive diagnosis of crop nutrient deficiencies. A drone-based RGB image dataset was generated together with ground truthing data in winter wheat (2020) and in winter rye (2021) during tillering and booting in the long-term fertilizer experiment (LTFE) Dikopshof. In this LTFE, the crops were fertilized with the same amounts for decades. The selected treatments included full fertilization including manure (NPKCa+m+s), mineral fertilization (NPKCa), mineral fertilization but no nitrogen (N) application (\_PKCa), no phosphorus (P) application (N\_KCa), no potassium (K) application (NP\_Ca), or no liming (Ca) (NPK\_), as well as an unfertilized treatment. The image dataset consisting of more than 3600 UAV-based RGB images was used to train and evaluate in total of eight CNN-based and transformer-based models as baselines within each crop-year and across the two crop-year combinations, aiming to detect the specific fertilizer treatments, including the specific nutrient deficiencies. The field observations showed a strong biomass decline in the case of N omission and no fertilization, though the effects were lower in the case of P, K, and lime omission. The mean detection accuracy within one year was 75\% (winter wheat) and 81\% (winter rye) across models and treatments. Hereby, the detection accuracy for winter wheat was highest for the NPKCa+m+s (100\%) and the unfertilized (96\%) treatments as well as the \_PKCa treatment (92\%), whereas for treatments N\_KCa and NPKCa the accuracy was lowest (about 50\%). The results were similar for winter rye. In the cross-year and cross-cereal species transfer (training on winter wheat, application on winter rye, and vice versa), the mean accuracy was about 18\%. The results highlight the potential of deep learning as a digital tool for decision-making in smart farming but also the difficulties of transferring models across years and crops.