[D] Running Large Language Models on CPU

Fine-tuning large language models with the aim of obtaining a small but accurate model is extremely difficult.

This is because you have to strike a balance between the model’s size and accuracy.
Researchers from IST Austria & Neural Magic seem to have found a sweet spot.

In their latest paper, they successfully applied sparse fine-tuning on MPT with remarkable performance.
The MPT model was pruned to 75% without a drop in accuracy, showing performance that is on-par with quantization approaches. …

accuracy aim austria balance cpu fine-tuning found language language models large language large language models machinelearning magic neural magic paper researchers running small spot strike