[P] Kubernetes plugin for mounting datasets to speed up model training

Hey r/MachineLearning!

My coworkers worked at Apple on the ML compute platform team and constantly found themselves supporting ML engineers with their large, distributed ML training jobs. ML engineers had to either use less data or they had to rewrite the training jobs to weave in more complicated *data chunking*. They also struggled to keep GPU utilization above 80% because so much time was spent waiting for data to just load: [https://discuss.pytorch.org/t/how-to-load-all-data-into-gpu-for-training/27609](https://discuss.pytorch.org/t/how-to-load-all-data-into-gpu-for-training/27609)

Inspired by the pains of that experience, they …

code data datasets deploy experience hey inside iterate job kubernetes large datasets library machinelearning open source plugin speed training