[Research] Consistency LLMs: converting LLMs to parallel decoders accelerates inference 3.5x

Hey all! We are here to share our latest work: consistency large language models (CLLMs), which is a new family of models capable of reducing inference latency by efficiently decoding 𝑛 tokens in parallel. Your new friends for LLM serving/local deployment with faster inference speed! 🔥 Please check our blog post for demo with 3.1x speedup:

[https://hao-ai-lab.github.io/blogs/cllm/](https://hao-ai-lab.github.io/blogs/cllm/)

Compared with existing fast decoding techniques, CLLMs achieve fast parallel decoding **without the need for**:

* Draft models
* Architectural modifications/auxiliary model components …

check decoding deployment family faster hey inference inference latency language language models large language large language models latency latest llm llms machinelearning research speed tokens work