Chinchilla Scaling: A replication attempt

arXiv:2404.10102v1 Announce Type: cross
Abstract: Hoffmann et al. (2022) propose three methods for estimating a compute-optimal scaling law. We attempt to replicate their third estimation procedure, which involves fitting a parametric loss function to a reconstruction of data from their plots. We find that the reported estimates are inconsistent with their first two estimation methods, fail at fitting the extracted data, and report implausibly narrow confidence intervals--intervals this narrow would require over 600,000 experiments, while they likely only ran fewer …

abstract arxiv compute cs.ai cs.cl data function law loss parametric plots replicate replication scaling scaling law type