{"id":38942,"date":"2026-08-21T15:27:24","date_gmt":"2026-08-21T15:27:24","guid":{"rendered":"https:\/\/lamarr-institute.org\/publication\/strong-error-analysis-for-the-stochastic-momentum-optimizer\/"},"modified":"2026-09-17T11:16:03","modified_gmt":"2026-09-17T11:16:03","slug":"strong-error-analysis-for-the-stochastic-momentum-optimizer","status":"publish","type":"publication","link":"https:\/\/lamarr-institute.org\/de\/publication\/strong-error-analysis-for-the-stochastic-momentum-optimizer\/","title":{"rendered":"Strong error analysis for the stochastic momentum optimizer"},"content":{"rendered":"<p>Stochastic gradient descent ({SGD}) optimization schemes are the methods of choice for the optimization of deep neural networks ({DNNs}) in artificial intelligence ({AI}) systems. Often not the standard {SGD} method is used but instead suitable accelerated, adaptive, and\/or normalized variants of standard {SGD} such as Adam, {AdamW}, and {MUON} are employed to train large scale {AI} systems in practically relevant settings. The acceleration (higher order convergence speed) in all these popular optimizers relies on the momentum {SGD} optimizer. In this work we provide a rigorous error analysis for the momentum {SGD} optimizer. In particular, we establish convergence rates for the momentum optimizer in terms of the size of the learning rate (step size), the size of the mini-batch, and the size of the one-point convexity constant.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Stochastic gradient descent ({SGD}) optimization schemes are the methods of choice for the optimization of deep neural networks ({DNNs}) in artificial intelligence ({AI}) systems. Often not the standard {SGD} method is used but instead suitable accelerated, adaptive, and\/or normalized variants of standard {SGD} such as Adam, {AdamW}, and {MUON} are employed to train large scale {AI} systems in practically relevant settings. The acceleration (higher order convergence speed) in all these [&hellip;]<\/p>\n","protected":false},"author":14,"featured_media":0,"template":"","meta":{"_acf_changed":false,"footnotes":""},"publication-type":[30],"class_list":["post-38942","publication","type-publication","status-publish","hentry","publication-type-article"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/38942","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/types\/publication"}],"author":[{"embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/users\/14"}],"version-history":[{"count":0,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/38942\/revisions"}],"wp:attachment":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/media?parent=38942"}],"wp:term":[{"taxonomy":"publication-type","embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication-type?post=38942"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}