ChimpVLM: Ethogram-Enhanced Chimpanzee Behaviour Recognition

arXiv:2404.08937v1 Announce Type: cross
Abstract: We show that chimpanzee behaviour understanding from camera traps can be enhanced by providing visual architectures with access to an embedding of text descriptions that detail species behaviours. In particular, we present a vision-language model which employs multi-modal decoding of visual features extracted directly from camera trap videos to process query tokens representing behaviours and output class predictions. Query tokens are initialised using a standardised ethogram of chimpanzee behaviour, rather than using random or name-based …

abstract access architectures arxiv cs.ai cs.cv cs.lg decoding embedding features language language model modal multi-modal process recognition show species text type understanding videos vision visual