Orca-Math: Unlocking the potential of SLMs in Grade School Math

arXiv:2402.14830v1 Announce Type: new
Abstract: Mathematical word problem-solving has long been recognized as a complex task for small language models (SLMs). A recent study hypothesized that the smallest model size, needed to achieve over 80% accuracy on the GSM8K benchmark, is 34 billion parameters. To reach this level of performance with smaller models, researcher often train SLMs to generate Python code or use tools to help avoid calculation errors. Additionally, they employ ensembling, where outputs of up to 100 model …

abstract accuracy arxiv benchmark billion cs.ai cs.cl language language models math orca parameters performance problem-solving school slms small small language models study type word