Serving Deep Networks in Production: Balancing Productivity vs Efficiency Tradeoff

A recently published work provides an alternative modality for serving deep neural networks. It enables utilizing eager-mode model code directly at production workloads by using embedded CPython interpreters. The goal is to reduce the engineering effort to bring the models from the research stage to the end-user and to create a proof-of-concept platform for migrating future numerical libraries.

ai c++ deep learning deployment efficiency machine learning ml & data engineering networks news production productivity python