[P] Optimal Performance without Static Graphs by Fusing Tensor Operation Streams

One of the most crucial aspects of current machine learning research is discovering model architectures that efficiently scale with compute resources. Transformers have emerged as the predominant architecture due to their effective utilization of contemporary hardware. However, they don't adapt their computation graphs based on the complexity of tasks, necessitating different versions for tasks of varying complexity. This approach doesn't align with the goal of having one model capable of continuous learning (lifelong learning) while remaining efficient for easy tasks. …

adapt architecture architectures complexity computation compute current graphs hardware however machine machine learning machinelearning performance research resources scale tasks tensor transformers