Prophet Attention: Predicting Attention with Future Attention for Improved Image Captioning. (arXiv:2210.10914v1 [cs.CV])

Recently, attention based models have been used extensively in many
sequence-to-sequence learning systems. Especially for image captioning, the
attention based models are expected to ground correct image regions with proper
generated words. However, for each time step in the decoding process, the
attention based models usually use the hidden state of the current input to
attend to the image regions. Under this setting, these attention models have a
"deviated focus" problem that they calculate the attention weights based on
previous …

arxiv attention captioning future image prophet