[D] Is a single channel enough for Positional Encoding in Transformers?

Yes, with [VAPE - Vector Addition Positional Encoding](https://gist.github.com/bnqj/309550aceae3d3be423bcf0199a9afc5).

I’ve been exploring a new approach to positional encoding that I’m calling VAPE - Vector Addition Positional Encoding.

**The Method**:

* borrow some number of channels from queries and keys,
* run a cumulative (prefix) sum across sequence length on these borrowed channels (add vectors together),
* normalize - divide by the square root of the vector's magnitude,
* we now have position-aware channels,
* so concatenate them back to queries and …

channels encoding keys machinelearning positional encoding queries square together vector vectors