Distributed Llama 2 on CPUs

A toy example of bulk inference on commodity hardware using Python, via llama.cpp and PySpark.

Why?

This exercise is about using Llama 2, an LLM (Large Language Model) from Meta AI, to summarize many documents at once. The scalable summarization of unstructured, semi-structured, and structured text can exist as a feature by itself, and also be part of data pipelines that feed into downstream machine learning models.

Specifically, we want to prove the …

apache spark generative-ai llama 2 llm nlp