Boosting Neural Machine Translation with Dependency-Scaled Self-Attention Network. (arXiv:2111.11707v3 [cs.CL] UPDATED)

Web: http://arxiv.org/abs/2111.11707

Syntax knowledge contributes its powerful strength in Neural machine
translation (NMT) tasks. The early NMT model supposed that syntax details can
be automatically learned from numerous texts via attention networks. However,
succeeding researches pointed out that limited by the uncontrolled nature of
attention computation, the model requires an external syntax to capture the
deep syntactic awareness. Although recent syntax-aware NMT methods have bored
great fruits in combining syntax, the additional workloads they introduced
render the model heavy and slow. Particularly, …

arxiv attention boosting machine machine translation network neural neural machine translation self-attention translation