[P]Retri-evals: Retrieval Evaluation Pipelines

Hey all,

We've been working on building retrieval pipelines for LLMs, and like many others we questioned how changes to our pipeline (e.g. chunking, cleaning) would affect the overall outcome.

We also faced a problem of what data to evaluate against. MTEB is used academically, but using our own data would be more reliable.

Retri-evals is hoping to solve these problems. We pulled out our MTEB abstractions that let us evaluate against open source datasets, and we're going to open …

building cleaning data evals evaluation hey llms machinelearning pipeline pipelines retrieval