{"id":35168,"date":"2026-04-13T14:10:57","date_gmt":"2026-04-13T14:10:57","guid":{"rendered":"https:\/\/lamarr-institute.org\/publication\/relative-value-learning\/"},"modified":"2026-06-08T13:18:10","modified_gmt":"2026-06-08T13:18:10","slug":"relative-value-learning","status":"publish","type":"publication","link":"https:\/\/lamarr-institute.org\/de\/publication\/relative-value-learning\/","title":{"rendered":"Relative Value Learning"},"content":{"rendered":"<p>In reinforcement learning (RL), critics traditionally learn absolute state values, estimating how good a particular situation is in isolation. Adding any constant to V(s) leaves action preferences unchanged; thus only value differences are relevant for decision making. Motivated by this fact, we propose Relative Value Learning (RV), a framework that learns antisymmetric value differences directly. We define a pairwise Bellman operator with a unique fixed point equal to true value differences, derive well-posed return targets and reconstruct generalized advantage estimation (R-GAE), resulting in an unbiased policy-gradient estimator. Empirically, integrating RV with PPO gives competitive performance on the Atari benchmark compared to standard PPO, indicating that learning relative value differences is a viable alternative to absolute critics.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In reinforcement learning (RL), critics traditionally learn absolute state values, estimating how good a particular situation is in isolation. Adding any constant to V(s) leaves action preferences unchanged; thus only value differences are relevant for decision making. Motivated by this fact, we propose Relative Value Learning (RV), a framework that learns antisymmetric value differences directly. We define a pairwise Bellman operator with a unique fixed point equal to true value [&hellip;]<\/p>\n","protected":false},"author":12,"featured_media":0,"template":"","meta":{"_acf_changed":false,"footnotes":""},"publication-type":[32],"class_list":["post-35168","publication","type-publication","status-publish","hentry","publication-type-inproceedings"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/35168","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/types\/publication"}],"author":[{"embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/users\/12"}],"version-history":[{"count":0,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/35168\/revisions"}],"wp:attachment":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/media?parent=35168"}],"wp:term":[{"taxonomy":"publication-type","embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication-type?post=35168"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}