Research directions not related to include optimization of Transformers (compute, memory, etc.)?

Honestly, I'm a bit fed up with the strong focus on squeezing the last bit of performance out of transformers. To lighten my mood I wanted to ask the community, if they've come across something interesting/different in their area.
For example, I've found ["Thinking Like Transformers"](https://arxiv.org/abs/2106.06981) by to be an enlightening fresh take.

community compute etc focus languagetechnology memory mood optimization performance research transformers