Large Language Models ({LLMs}) excel at evaluating machine translation ({MT}), but their scale and cost hinder deployment on edge devices and in privacy-sensitive workflows. We ask: how small can you get while still detecting meaning-altering translation errors? Focusing on English->German Critical Error Detection ({CED}), we benchmark sub-2B models ({LFM}2-350M, Qwen-3-0.6B\/1.7B, Llama-3.2-1B-Instruct, Gemma-3-1B) across {WMT}21, {WMT}22, and {SynCED}-{EnDe}-2025. Our framework standardizes prompts, applies lightweight logit-bias calibration and majority voting, and reports both semantic quality ({MCC}, F1-{ERR}\/F1-{NOT}) and compute metrics ({VRAM}, latency, throughput). Results reveal a clear sweet spot around one billion parameters: Gemma-3-1B provides the best quality-efficiency trade-off, reaching {MCC}=0.77 with F1-{ERR}=0.98 on {SynCED}-{EnDe}-2025 after merged-weights fine-tuning, while maintaining 400 ms single-sample latency on a {MacBook} Pro M4 Pro (24 {GB}). At larger scale, Qwen-3-1.7B attains the highest absolute {MCC} (+0.11 over Gemma) but with higher compute cost. In contrast, ultra-small models (0.6B) remain usable with few-shot calibration yet under-detect entity and number errors. Overall, compact, instruction-tuned {LLMs} augmented with lightweight calibration and small-sample supervision can deliver trustworthy, on-device {CED} for {MT}, enabling private, low-cost error screening in real-world translation pipelines. All datasets, prompts, and scripts are publicly available at our {GitHub} repository.<\/p>\n","protected":false},"excerpt":{"rendered":"
Large Language Models ({LLMs}) excel at evaluating machine translation ({MT}), but their scale and cost hinder deployment on edge devices and in privacy-sensitive workflows. We ask: how small can you get while still detecting meaning-altering translation errors? Focusing on English->German Critical Error Detection ({CED}), we benchmark sub-2B models ({LFM}2-350M, Qwen-3-0.6B\/1.7B, Llama-3.2-1B-Instruct, Gemma-3-1B) across {WMT}21, {WMT}22, and {SynCED}-{EnDe}-2025. Our framework standardizes prompts, applies lightweight logit-bias calibration and majority voting, and reports […]<\/p>\n","protected":false},"author":12,"featured_media":0,"template":"","meta":{"_acf_changed":false,"footnotes":""},"publication-type":[30],"class_list":["post-32198","publication","type-publication","status-publish","hentry","publication-type-article"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/32198","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/types\/publication"}],"author":[{"embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/users\/12"}],"version-history":[{"count":0,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/32198\/revisions"}],"wp:attachment":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/media?parent=32198"}],"wp:term":[{"taxonomy":"publication-type","embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication-type?post=32198"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}