[N] Google blog post "What is a long context window?" states that the long context project whose results are used in Gemini 1.5 Pro required "a series of deep learning innovations," but doesn't specify what those innovations are

From [What is a long context window?](https://blog.google/technology/ai/long-context-window-ai-models/):

>"Our original plan was to achieve 128,000 tokens in context, and I thought setting an ambitious bar would be good, so I suggested 1 million tokens," says Google DeepMind Research Scientist Nikolay Savinov, one of the research leads on the long context project. “And now we’ve even surpassed that in our research by 10x.”
>
>To make this kind of leap forward, the team had to make a series of deep learning innovations. …

context deepmind deepmind research good google google deepmind kind leads machinelearning project research research scientist thought tokens