Ben Sorscher: Data Pruning for Efficient Machine Learning

In this episode, Ben Sorscher, a PhD student at Stanford, sheds light on the challenges posed by the ever-increasing size of data sets used to train machine learning models, specifically large language models. The sheer size of these data sets has been pushing the limits of scaling, as the cost of training and the environmental impact of the electricity they consume becomes increasingly enormous.

As a solution, Ben discusses the concept of “data pruning” - a method of reducing the …

artificial intelligence ben sorscher data pruning data reduction data sets efficiency environmental impact large language models machine learning model-performance phd resource efficiency scaling stanford sustainability training costs