{"id":35130,"date":"2026-04-13T14:10:33","date_gmt":"2026-04-13T14:10:33","guid":{"rendered":"https:\/\/lamarr-institute.org\/publication\/raising-bars-not-parameters-lilmoo-compact-language-model-for-hindi\/"},"modified":"2026-06-08T13:17:36","modified_gmt":"2026-06-08T13:17:36","slug":"raising-bars-not-parameters-lilmoo-compact-language-model-for-hindi","status":"publish","type":"publication","link":"https:\/\/lamarr-institute.org\/de\/publication\/raising-bars-not-parameters-lilmoo-compact-language-model-for-hindi\/","title":{"rendered":"Raising Bars, Not Parameters: {LilMoo} Compact Language Model for Hindi"},"content":{"rendered":"<p>The dominance of large multilingual foundation models has widened linguistic inequalities in Natural Language Processing ({NLP}), often leaving low-resource languages underrepresented. This paper introduces {LilMoo}, a 0.6-billion-parameter Hindi language model trained entirely from scratch to address this gap. Unlike prior Hindi models that rely on continual pretraining from opaque multilingual foundations, {LilMoo} is developed through a fully transparent and reproducible pipeline optimized for limited compute environments. We construct a high-quality Hindi corpus ({GigaLekh}) filtered through both heuristic and learned ({LLM}-as-a-judge) methods, complemented by bilingual augmentation with curated English data. Using this dataset, we explore various training recipes for small-scale language models. Across comprehensive evaluation suites, {LilMoo} consistently outperforms comparably sized multilingual baselines such as Qwen2.5-0.5B and Qwen3-0.6B, demonstrating that well-designed language-specific pretraining can rival large multilingual models at the sub-billion-parameter range.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>The dominance of large multilingual foundation models has widened linguistic inequalities in Natural Language Processing ({NLP}), often leaving low-resource languages underrepresented. This paper introduces {LilMoo}, a 0.6-billion-parameter Hindi language model trained entirely from scratch to address this gap. Unlike prior Hindi models that rely on continual pretraining from opaque multilingual foundations, {LilMoo} is developed through a fully transparent and reproducible pipeline optimized for limited compute environments. We construct a high-quality [&hellip;]<\/p>\n","protected":false},"author":12,"featured_media":0,"template":"","meta":{"_acf_changed":false,"footnotes":""},"publication-type":[30],"class_list":["post-35130","publication","type-publication","status-publish","hentry","publication-type-article"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/35130","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/types\/publication"}],"author":[{"embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/users\/12"}],"version-history":[{"count":0,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/35130\/revisions"}],"wp:attachment":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/media?parent=35130"}],"wp:term":[{"taxonomy":"publication-type","embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication-type?post=35130"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}