Mighty New TransformerFAM (Feedback Attention Mem)

The Latest in AI research: The video introduces a powerful new TransformerFAM (Feedback Attention Memory) by @Google a novel architecture designed to enhance Transformers by incorporating a feedback mechanism that emulates working memory.
Plus the introduction of the new Transformer BSWA (Block Sliding Window Attention).

Based on ring attention by @UCBerkeley

This design allows the Transformer to maintain awareness of its own latent representations across different blocks of data, improving its ability to process indefinitely long sequences without additional computational …

ai research architecture attention block design feedback google introduction memory novel research ring transformer transformers video