all AI news
The cost of AI reasoning over time
April 4, 2024, 12:51 p.m. |
Simon Willison's Weblog simonwillison.net
The cost of AI reasoning over time
Karina Nguyen from Anthropic provides a fascinating visualization illustrating the cost of different levels of LLM over the past few years, plotting their cost-per-token against their scores on the MMLU benchmark.
Claude 3 Haiku currently occupies the lowest cost to score ratio, over on the lower right hand side of the chart.
Via @swyx
ai ai reasoning anthropic benchmark claude claude 3 claude 3 haiku cost generativeai haiku llm llms mmlu per plotting reasoning token visualization
More from simonwillison.net / Simon Willison's Weblog
How an empty S3 bucket can make your AWS bill explode
1 day, 21 hours ago |
simonwillison.net
My approach to HTML web components
1 day, 22 hours ago |
simonwillison.net
Jobs in AI, ML, Big Data
Data Architect
@ University of Texas at Austin | Austin, TX
Data ETL Engineer
@ University of Texas at Austin | Austin, TX
Lead GNSS Data Scientist
@ Lurra Systems | Melbourne
Senior Machine Learning Engineer (MLOps)
@ Promaton | Remote, Europe
C003549 Data Analyst (NS) - MON 13 May
@ EMW, Inc. | Braine-l'Alleud, Wallonia, Belgium
Marketing Decision Scientist
@ Meta | Menlo Park, CA | New York City