[D] Looking for suggestions / guides on how to switch from OpenAI Embeddings and Pinecone to open-source / self-hosted architecture options.

Hi all,

I'm interested in redesigning my application to utilize an open-source embeddings model and a different vector DB. My current issue with embeddings is that processing large volumes of data into a vector DB using ada-002 is unreliable, with frequent API timeouts occurring or issues interacting with Pinecone. This is super problematic as it's difficult to track which data has / hasn't been stored correctly. I also know that many open-source embeddings models are more performant and will allow …

ada application architecture current data embeddings guides issue machinelearning openai pinecone processing self-hosted vector