Google AI’s AdaTape is a New AI Approach With Transformer-Based Architecture

In a blog post last week, Google Research introduced AdaTape, a new AI approach with transformer-based architecture that utilizes adaptive computation. AdaTape uses its adaptive function to use create an elastic input sequence that can modulate its computational budget.

According to the accompanying research paper, AdaTape directly injects adaptivity into the input sequence instead of the model depth. It also uses an adaptive tape reading mechanism. This is to determine various tape tokens that are added to each input …

adatape architecture artificial intelligence blog budget computation computational data science elastic function google google research modulate open-data paper research research paper transformer transformers