all AI news
The cost of AI reasoning over time
April 4, 2024, 12:51 p.m. |
Simon Willison's Weblog simonwillison.net
The cost of AI reasoning over time
Karina Nguyen from Anthropic provides a fascinating visualization illustrating the cost of different levels of LLM over the past few years, plotting their cost-per-token against their scores on the MMLU benchmark.
Claude 3 Haiku currently occupies the lowest cost to score ratio, over on the lower right hand side of the chart.
Via @swyx
ai ai reasoning anthropic benchmark claude claude 3 claude 3 haiku cost generativeai haiku llm llms mmlu per plotting reasoning token visualization
More from simonwillison.net / Simon Willison's Weblog
Ham radio general exam question pool as JSON
1 day, 11 hours ago |
simonwillison.net
uv pip install --exclude-newer example
2 days, 13 hours ago |
simonwillison.net
Jobs in AI, ML, Big Data
Data Engineer
@ Lemon.io | Remote: Europe, LATAM, Canada, UK, Asia, Oceania
Artificial Intelligence – Bioinformatic Expert
@ University of Texas Medical Branch | Galveston, TX
Lead Developer (AI)
@ Cere Network | San Francisco, US
Research Engineer
@ Allora Labs | Remote
Ecosystem Manager
@ Allora Labs | Remote
Founding AI Engineer, Agents
@ Occam AI | New York