Sentiment analysis with adaptive multi-head attention in Transformer

arXiv:2310.14505v4 Announce Type: replace
Abstract: We propose a novel framework based on the attention mechanism to identify the sentiment of a movie review document. Previous efforts on deep neural networks with attention mechanisms focus on encoder and decoder with fixed numbers of multi-head attention. Therefore, we need a mechanism to stop the attention process automatically if no more useful information can be read from the memory.In this paper, we propose an adaptive multi-head attention architecture (AdaptAttn) which varies the number …

abstract analysis arxiv attention attention mechanisms cs.cl decoder document encoder focus framework head identify movie movıe multi-head multi-head attention networks neural networks novel numbers review sentiment sentiment analysis transformer type