How to Evaluate an LLM's Ability to Follow Instructions

Assessing the Impact of Decoding Strategies on the Instruction Following Evaluation for Large Language Models Benchmark

Recently I’ve been intellectually obsessed with two things:

1. How do models generate text? (Trying to grok how various LLM decoding strategies impact the resulting generations)
2. And how do we gauge how good they are at it? (The minefield known as LLM evaluation)

It’s not just idle curiosity. It’s my job.

I’ve been handed this cool yet daunting task: …

evaluation hugging face large language models llm