Action Segmentation Using 2D Skeleton Heatmaps and Multi-Modality Fusion

arXiv:2309.06462v3 Announce Type: replace
Abstract: This paper presents a 2D skeleton-based action segmentation method with applications in fine-grained human activity recognition. In contrast with state-of-the-art methods which directly take sequences of 3D skeleton coordinates as inputs and apply Graph Convolutional Networks (GCNs) for spatiotemporal feature learning, our main idea is to use sequences of 2D skeleton heatmaps as inputs and employ Temporal Convolutional Networks (TCNs) to extract spatiotemporal features. Despite lacking 3D information, our approach yields comparable/superior performances and better …

abstract applications apply art arxiv contrast convolutional cs.cv feature fine-grained fusion graph human inputs networks paper recognition segmentation state type