Help with seq2seq models with attention

Hi all, I'm trying to create a seq2seq model with attention and I'm a bit stuck in the decoder implementation.

I have the encoder embeddings with dimension (N, L\_e, E) and the decoder inputs with dimension (N, L\_d, E). N is the batch size; L\_e, L\_d are the encoder and decoder sequence lengths; and E is the embedding size.

I'm working in PyTorch; I would like to apply a nn.MultiHeadAttention layer to the encoder embeddings and pass them to the …

attention decoder deeplearning embeddings encoder implementation seq2seq