Toward Cross-Layer Energy Optimizations in Machine Learning Systems

arXiv:2404.06675v1 Announce Type: new
Abstract: The enormous energy consumption of machine learning (ML) and generative AI workloads shows no sign of waning, taking a toll on operating costs, power delivery, and environmental sustainability. Despite a long line of research on energy-efficient hardware, we found that software plays a critical role in ML energy optimization through two recent works: Zeus and Perseus. This is especially true for large language models (LLMs) because their model sizes and, therefore, energy demands are growing …

abstract ai workloads arxiv consumption costs cs.ar cs.dc cs.lg delivery energy environmental environmental sustainability found generative hardware layer learning systems line machine machine learning power research role shows software sustainability systems type workloads