How Small Can You Go? Compact Language Models for On-Device Critical Error Detection in Machine Translation

Ensuring the reliability of machine translation systems requires detecting critical translation errors that may alter the meaning of generated text. While large language models have shown promising performance for such tasks, their computational requirements hinder deployment in resource-constrained environments. In this work, we investigate compact language models for critical error detection in machine translation with a focus on on-device deployment. We benchmark several small-scale models and evaluate their ability to identify critical semantic errors across translation outputs while maintaining low computational overhead. Our experiments demonstrate that carefully optimized compact models can achieve competitive detection performance relative to larger systems while significantly reducing memory and compute requirements. These findings highlight the feasibility of efficient, privacy-preserving translation validation pipelines suitable for real-world edge and mobile environments.