The Larger the Better? Improved LLM Code-Generation via Budget Reallocation

arXiv:2404.00725v1 Announce Type: cross
Abstract: It is a common belief that large language models (LLMs) are better than smaller-sized ones. However, larger models also require significantly more time and compute during inference. This begs the question: what happens when both models operate under the same budget? (e.g., compute, run-time). To address this question, we analyze code generation LLMs of various sizes and make comparisons such as running a 70B model once vs. generating five outputs from a 13B model and …

abstract arxiv belief budget code compute cs.ai cs.cl cs.lg cs.se however inference language language models large language large language models larger models llm llms question type via