[D] How to make AI model infernce faster?

Hi everyone,

I am currently trying to serve some LM models like GPT2 and some LLMs like llama or bloomz. I tried converting GPT2 to TensorRT (based on Nvidia's documentation) and serving with Triton server. The output is also quite good. However, if you want to continue to optimize, will there be any more methods to perform optimization? And I'm also quite curious how to make the LLM model able to serve many users at the same time. Is there …

ai model documentation faster good llama llms machinelearning nvidia serve server tensorrt triton will