T5 Fine-tuning for Summarization with multiple GPUs

Dear all,
I hope you are fine.

I have fine-tuned T5 (t5-small) for summarization with the English Dataset of movie reviews following the script from the hugging face course. The link is given below.


https://huggingface.co/course/chapter7/5?fw=pt


The script is working perfectly and I am getting good results for summarization but I want to fine-tune t5 for summarization using multiple GPUs.

I have observed that Nvidia Megatron-LM can be used for fine-tuning t5 for summarization using multiple GPUs but is there any …

fine-tuning gpus languagetechnology summarization