A Gated Attention Transformer for Multi-Person Pose Tracking

Multi-person pose tracking is an important element for many applications and requires to estimate the human poses of all persons in a video and to track them over time. The association of poses across frames remains an open research problem, in particular for online tracking methods, due to motion blur, crowded scenes and occlusions. To tackle the association challenge, we propose a Gated Attention Transformer. The core aspect of our model is the gating mechanism that automatically adapts the impact of appearance embeddings and embeddings based on temporal pose similarity in the attention layers. In order to re-identify persons that have been occluded, we incorporate a pose-conditioned re-identification network that provides initial embeddings and allows to match persons even if the number of visible joints differ between frames. We further propose a matching layer based on gated attention for pose-to-track association and duplicate removal. We evaluate our approach on PoseTrack 2018 and PoseTrack21.

  • Published in:
    IEEE/CVF International Conference on Computer Vision Workshops
  • Type:
  • Authors:
    Döring, Andreas; Gall, Jürgen
  • Year:

Citation information

Döring, Andreas; Gall, Jürgen: A Gated Attention Transformer for Multi-Person Pose Tracking, IEEE/CVF International Conference on Computer Vision Workshops, 2023, October, https://ieeexplore.ieee.org/document/10350647, Doering.Gall.2023a,

Associated Lamarr Researchers

lamarr institute person Gall Juergen - Lamarr Institute for Machine Learning (ML) and Artificial Intelligence (AI)

Prof. Dr. Jürgen Gall

Principal Investigator Embodied AI to the profile