SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Models

Most of the existing Large Language Model (LLM) benchmarks on scientific problem reasoning focus on problems grounded in high-school subjects and are confined to elementary algebraic operations. To systematically examine the reasoning capabilities required for solving complex scientific problems, we introduce an expansive benchmark suite SciBench for LLMs. SciBench contains a carefully curated dataset featuring a range of collegiate-level scientific problems from mathematics, chemistry, and physics domains. Based on the dataset, we conduct an in-depth benchmarking study of representative open-source …

benchmark benchmarks capabilities college cs.ai cs.cl cs.lg elementary focus high-school language language model language models large language large language model large language models llm llms operations problem-solving reasoning school