Machine Learning Model Sizes and the Parameter Gap. (arXiv:2207.02852v1 [cs.LG])

We study trends in model size of notable machine learning systems over time
using a curated dataset. From 1950 to 2018, model size in language models
increased steadily by seven orders of magnitude. The trend then accelerated,
with model size increasing by another five orders of magnitude in just 4 years
from 2018 to 2022. Vision models grew at a more constant pace, totaling 7
orders of magnitude of growth between 1950 and 2022.

We also identify that, since 2020, …

arxiv gap learning lg machine machine learning machine learning model