Bottleneck T5 Text Autoencoder

Bottleneck T5 Text Autoencoder

Fascinating Colab notebook by Linus Lee demonstrating his Contra Bottleneck T5 embedding model, which can take up to 512 tokens of text, convert that into a 1024 floating point number embedding vector... and then then reconstruct the original text (or a close imitation) from the embedding again.

This allows for some fascinating tricks, where you can do things like generate embeddings for two completely different sentences and then reconstruct a new sentence that combines the weights …

ai autoencoder colab embedding embeddings floating point generativeai jupyter llms notebook python text tokens tricks vector