When will the advancement in things like GPT, Stable Diffusion and Sora plateau?

I know deep learning based models are seeing the fastest advancements ever, thanks to huge compute and data.

But people keep extrapolating and people talk about accelerating / exponential growth and the singularity being reached.

But I would argue most of these models are based off novel neural architectures and then they hyperparameter tune and train on more data, and voila, you see huge advancements and it makes it look like accelerating progress.

But once we've exhausted the different architectures …

advancement compute data deep learning deeplearning diffusion gpt growth novel people singularity sora stable diffusion talk will