[D] Best tools for Multi-GPU model training?

Hi everyone, until recently I only had to work on problems for which a single GPU training setup would suffice. But i am working on a problem with a large dataset and I have access to multiple GPUs so I was wondering what's the best way to set this up. Going through the PyTorch documentation, they seem to suggest using torch.distributed.run along with DistributedDataParallel for distributed training although I also came across libraries in which the boilerplate stuff is abstracted …

gpu machinelearning tools training