Splitwise improves GPU usage by splitting LLM inference phases

Expanded LLM use creates new demands on cloud GPU capacity. Splitwise presents an efficient solution by separating the two essential phases of LLM inference, achieving higher throughput within a limited power budget.

The post Splitwise improves GPU usage by splitting LLM inference phases appeared first on Microsoft Research.

budget capacity cloud cloud gpu gpu inference llm microsoft microsoft research power research research blog solution usage