[P] LLM Apps Are Mostly Data Pipelines

My colleague just wrote up an article on [LLM-based apps and how to use data engineering tools to help build them faster](https://meltano.com/blog/llm-apps-are-mostly-data-pipelines/) that I found really insightful.

It contains a complete implementation

* with scraping context data from a docs website
* chunking it, getting embeddings via the openAI API
* loading it into pinecone
* and finally a simple Q&A interface with streamlit on top of it

**Here's a quick summary:**

* LangChain and LlamaIndex are great tools for …

api context data embeddings exploration implementation langchain llamaindex loading machinelearning openai openai api pinecone scraping simple streamlit summary tools website