Seq2seq Transformer Autoencoder returns the same results when unfolded using only memory and initial value

I have numeric signals from two sensors, and I would like to create a mapping using sequence-to-sequence autoencoder. I used the transformer architecture, and it seems to be learning - the loss is getting lower both for the training and validation sets over time, and when using the decoder on both inputs - memory (that is, \`encoder(source)\`) and \`target\`, the decoder would return a value, very similar to \`target\` (and also low \`MSE\` values).

Nevertheless, in the inference stage, results …

architecture autoencoder deeplearning loss mapping memory returns sensors seq2seq training transformer transformer architecture validation value