Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model

Progress in machine learning (ML) comes with a cost to the environment, given that training ML models requires computational resources, energy and materials. In the present article, we aim to quantify the carbon footprint of BLOOM, a 176-billion parameter language model, across its life cycle. We estimate that BLOOM’s final training emitted approximately 24.7 tonnes of CO2eq if we consider only the dynamic power consumption, and 50.5 tonnes if we account for all processes ranging from equipment manufacturing to energy-based …

aim article billion bloom carbon carbon footprint computational cost energy environment language language model life life cycle machine machine learning materials ml models progress resources training