LLM Structured Output with Local Haystack RAG and Ollama

Haystack 2.0 provides functionality to process LLM output and ensure proper JSON structure, based on predefined Pydantic class. I show how you can run this on your local machine, with Ollama. This is possible thanks to OllamaGenerator class available from Haystack.

Sparrow GitHub repo:
https://github.com/katanaml/sparrow

Tutorial: Generating Structured Output with Loop-Based Auto-Correction:
https://haystack.deepset.ai/tutorials/28_structured_output_with_loop

0:00 Intro
0:40 OllamaGenerator
1:09 Pydantic classes
1:43 Output validator
2:54 Haystack pipeline
3:12 Ollama
3:25 Haystack pipeline
4:54 Example
6:15 Troubleshooting
6:19 Notus LLM
7:40 Result …

auto class github github repo haystack json llm loop machine ollama process pydantic rag show tutorial