Jan. 16, 2024, 12:50 p.m. | /u/East_Dragonfruit7277

Machine Learning www.reddit.com

Hi everyone,

We've shared some practical insights on Retrieval Augmented Generation (RAG) with our custom data processing framework called Fondant in our latest blogpost.

Finetuning RAG is a complex task that requires a lot of time and effort. We built an example pipeline that indexes a custom knowledge base (PDF, Huggingface dataset, ...), processes the data (embedding, chunking,...) and evaluates the results. We integrated different parameters search techniques for picking the best configuration which results in the best outcome for …

automated data data processing dataset example finetuning framework huggingface insights knowledge knowledge base machinelearning optimization pdf pipeline practical processing rag retrieval retrieval augmented generation

Founding AI Engineer, Agents

@ Occam AI | New York

AI Engineer Intern, Agents

@ Occam AI | US

AI Research Scientist

@ Vara | Berlin, Germany and Remote

Data Architect

@ University of Texas at Austin | Austin, TX

Data ETL Engineer

@ University of Texas at Austin | Austin, TX

Sr. Software Development Manager, AWS Neuron Machine Learning Distributed Training

@ Amazon.com | Cupertino, California, USA