{"id":32287,"date":"2026-01-21T17:01:38","date_gmt":"2026-01-21T17:01:38","guid":{"rendered":"https:\/\/lamarr-institute.org\/publication\/sfmocc-vision-based-3d-semantic-occupancy-prediction-in-urban-environments\/"},"modified":"2026-09-17T11:23:08","modified_gmt":"2026-09-17T11:23:08","slug":"sfmocc-vision-based-3d-semantic-occupancy-prediction-in-urban-environments","status":"publish","type":"publication","link":"https:\/\/lamarr-institute.org\/de\/publication\/sfmocc-vision-based-3d-semantic-occupancy-prediction-in-urban-environments\/","title":{"rendered":"{SfmOcc}: Vision-Based 3D Semantic Occupancy Prediction in Urban Environments"},"content":{"rendered":"<p>Semantic scene understanding is crucial for autonomous systems and 3D semantic occupancy prediction is a key task since it provides geometric and possibly semantic information of the vehicle&#8217;s surroundings. Most existing vision-based approaches to occupancy estimation rely on 3D voxel labels or segmented {LiDAR} point clouds for supervision. This limits their application to the availability of a 3D {LiDAR} sensor or the costly labeling of the voxels. While other approaches rely only on images for training, they usually supervise only with a few consecutive images and optimize for proxy tasks like volume reconstruction or depth prediction. In this paper, we propose a novel method for semantic occupancy prediction using only vision data also for supervision. We leverage all the available training images of a sequence and use bundle adjustment to align the images and estimate camera poses from which we then obtain depth images. We compute semantic maps from a pre-trained open-vocabulary image model and generate occupancy pseudo labels to explicitly optimize for the 3D semantic occupancy prediction task. Without any manual or {LiDAR}-based labels, our approach predicts full 3D occupancy voxel grids and achieves state-of-the-art results for 3D occupancy prediction among methods trained without labels.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Semantic scene understanding is crucial for autonomous systems and 3D semantic occupancy prediction is a key task since it provides geometric and possibly semantic information of the vehicle&#8217;s surroundings. Most existing vision-based approaches to occupancy estimation rely on 3D voxel labels or segmented {LiDAR} point clouds for supervision. This limits their application to the availability of a 3D {LiDAR} sensor or the costly labeling of the voxels. While other approaches [&hellip;]<\/p>\n","protected":false},"author":14,"featured_media":0,"template":"","meta":{"_acf_changed":false,"footnotes":""},"publication-type":[30],"class_list":["post-32287","publication","type-publication","status-publish","hentry","publication-type-article"],"acf":[],"publishpress_future_workflow_manual_trigger":{"enabledWorkflows":[]},"_links":{"self":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/32287","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication"}],"about":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/types\/publication"}],"author":[{"embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/users\/14"}],"version-history":[{"count":0,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication\/32287\/revisions"}],"wp:attachment":[{"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/media?parent=32287"}],"wp:term":[{"taxonomy":"publication-type","embeddable":true,"href":"https:\/\/lamarr-institute.org\/de\/wp-json\/wp\/v2\/publication-type?post=32287"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}