Large language models (LLMs) have emerged to become an integral part of the software stack. They are available through APIs from providers like Cohere, Google Cloud, and OpenAI or as open-source models hosted on Hugging Face.
However, utilizing LLMs isn’t merely about sending prompts. Developers must consider aspects like parameter adjustments, prompt augmentation, and response moderation. LLMs are stateless, so developers must maintain conversational history for context, possibly using databases for long-term storage.