[P] What incremental unsolved problems are there in scaling machine learning training (distributed systems/Ray/data parallelism)?

I'm working on a class project, where we'd present a distributed systems problem and write about some improvements to an existing system. I'm interested in machine learning training and wanted to extend upon [Ray](https://github.com/ray-project/ray).

I'm trying to find a distributed systems problem that isn't fully solved in Ray, or can be optimized. I'm not looking for a solution that will be the best for every scenario, but maybe a small tradeoff improvement that can be made (ex: trade off accuracy …

accuracy distributed distributed systems etc every improvement isn machinelearning ray recovery small solution systems trade will