[D] technical question: How is it possible that embedding models produce fixed size vectors for sentences with varying lengths?

As far as I know and studied, each token is mapped from high dimensional discrete token space into a continuous, lower dimensional space where words are embedded meaningfully based on their relationships in the training data. So 1000 tokens text produces 1000 vectors.


Now for vector databases (correct me if I'm wrong), people are storing fixed sized vectors for text with varying lengths. For example, 2 sentences one with 1000 tokens and the other is 10 tokens, each produces one …

continuous data embedded embedding embedding models machinelearning mapped relationships space technical text token tokens training training data vectors words