New major release for nebullvm, an opensource to speed up AI inference by leveraging state-of-the-art optimization techniques (deep learning compilers, and now also quantization and quantization, and soon also sparsity, distillation, etc.)

Web: https://www.reddit.com/r/deeplearning/comments/unxcyr/new_major_release_for_nebullvm_an_opensource_to/

nebullvm is an opensource library that generates an optimize version of your deep learning model that runs 2-10 times faster in inference without performance loss by leveraging multiple deep learning compilers (openvino, tensorrt, etc.). And thanks to today's new release, nebullvm can accelerate up to 30x if you specify that you are willing to trade off a self-defined amount of accuracy/precision to get even lower response time and a lighter model. This additional acceleration is achieved by exploiting optimization techniques …

ai art compilers deep deep learning deeplearning distillation inference learning opensource optimization quantization sparsity state