Towards Rhino-{AR}: A System for Real-Time 3D Human Pose Estimation and Volumetric Scene Integration on Embedded {AR} Headsets

Real-time understanding of dynamic human presence is crucial for immersive Augmented Reality ({AR}), yet challenging on resource-constrained Head-Mounted Displays ({HMDs}). This paper introduces Rhino-{AR}, a pipeline for ondevice 3D human pose estimation and dynamic scene integration for commercial {AR} headsets like the Magic Leap 2. Our system processes {RGB} and sparse depth data, first detecting 2D keypoints, then robustly lifting them to 3D. Beyond pose estimation, we reconstruct a coarse anatomical model of the human body, tightly coupled with the estimated skeleton. This volumetric proxy for dynamic human geometry is then integrated with the {HMD}’s static environment mesh by actively removing human-generated artifacts. This integration is crucial, enabling physically plausible interactions between virtual entities and real users, supporting real-time collision detection, and ensuring correct occlusion handling where virtual content respects realworld spatial dynamics. Implemented entirely on the Magic Leap 2, our method achieves low-latency pose updates (under 40 ms) and full 3D lifting (under 60 ms). Comparative evaluation against the {RTMW}3D-x baseline shows a Procrustes-Aligned Mean Per Joint Position Error below 140 mm, with absolute depth placement validated using an external Azure Kinect sensor. Rhino-{AR} demonstrates the feasibility of robust, realtime human-aware perception on mobile {AR} platforms, enabling new classes of interactive, spatially-aware applications without external computation.