Transforming text into vectors: TSDAE’s unsupervised approach to enhanced embeddings

Combine TSDAE pre-training on a target domain with supervised fine-tuning on a general-purpose corpus to enhance the quality of the embeddings for a specialized domain.

Introduction

Embeddings encode text into high dimensional vector spaces, using dense vectors to represent words and to capture their semantic relationships. Recent developments in generative AI and LLM, such as context search and RAG rely heavily on the quality of their underlying embeddings. While the similarity searches use basic mathematical concepts such …

domain adaptation fine-tuning-transformer sentence-embedding transformers