[P] What we learned by benchmarking TorchDynamo (PyTorch team), ONNX Runtime and TensorRT on transformers model (inference)

**TL;DR**: `TorchDynamo` (prototype from PyTorch team) plus `nvfuser` (from Nvidia) backend makes Bert (the tool is model agnostic) inference on PyTorch > **3X** faster most of the time (it depends on input shape) by just adding a single line of code in Python script. The surprising thing is that during the benchmark, **we have not seen any drawback implied by the use of this library**, the acceleration just comes for free. On the same model, TensorRT is (of course) much …

benchmarking inference machinelearning onnx pytorch team tensorrt transformers