[R] Infinite context Transformers

I took a look and didn't see any discussion thread here on this paper which looks perhaps promising.


[https://arxiv.org/abs/2404.07143](https://arxiv.org/abs/2404.07143)


What are your thoughts? Could it be one of the techniques behind the Gemini 1.5 reported 10m token context length?

context gemini gemini 1.5 look machinelearning paper thoughts thread token transformers