Supercharging Large Language Models: DEJAVU’s Inference Time Surpasses FasterTransformer by 2×

In a new paper Deja Vu: Contextual Sparsity for Efficient LLMs at Inference Time, a research team presents DEJAVU, a system that employs a cost-effective algorithm to predict contextual sparsity dynamically for each layer, combined with an asynchronous and hardware-aware implementation to accelerate LLM inference.

ai algorithm artificial intelligence asynchronous cost deep-neural-networks hardware implementation inference language language models large language large language model large language models layer llm llms machine learning machine learning & data science ml paper research research team sparsity team technology transformer