[D] What is your tech stack for research?

I am planning on working on large multiomodal training (1B parameters) for text+audio. As of now I was thinking of going with pytorch, deepspeed, wandb. What do you recommend and what do you use in general for distributed large model training?

Do you use hugginface? I felt it a bit too wrapped that it becomes messy to access the bare backbones, but haven't given it a proper try. For out of shelf models and custom dataset training that does sound …

audio deepspeed distributed felt general machinelearning parameters planning pytorch research stack tech tech stack text thinking training wandb