The Bahdanau Attention Mechanism

Conventional encoder-decoder architectures for machine translation encoded every source sentence into a fixed-length vector, irrespective of its length, from which the decoder would then generate a translation. This made it difficult for the neural network to cope with long sentences, essentially resulting in a performance bottleneck.  The Bahdanau attention was proposed to address the performance […]

The post The Bahdanau Attention Mechanism appeared first on Machine Learning Mastery.

attention bahdanau encoder-decoder neural machine translation recurrent neural network