July 28, 2023, 8:01 a.m. | /u/Dustwellow

Machine Learning www.reddit.com

I have a data set that isn't that large \~200 pdfs. I have done the regular RAG approach with Langchain, extracting text, splitting into chunks, embedding with OpenAi embeddings and FAISS vector storage. However, when I do a similarity search with a question I would like answered it returns the wrong context. The documents are semi-structured information of examined bridges. A question I would like answered is f.e. 'what is the construction date of bridge X?'. When I input this …

data data set embedding embeddings faiss langchain machinelearning openai returns search set storage text vector

Software Engineer for AI Training Data (School Specific)

@ G2i Inc | Remote

Software Engineer for AI Training Data (Python)

@ G2i Inc | Remote

Software Engineer for AI Training Data (Tier 2)

@ G2i Inc | Remote

Data Engineer

@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania

Artificial Intelligence – Bioinformatic Expert

@ University of Texas Medical Branch | Galveston, TX

Lead Developer (AI)

@ Cere Network | San Francisco, US