Optimizing deep learning inference to run on the edge?

I'm an embedded SW dev who had to help a company once to optimize their data pipeline so they could do computer vision on an edge device (an Nvidia Jetson, in case you were curious).

I'm wondering, is this a common issue that companies have? I've heard that ML inference is starting more and more to move to the edge devices instead of being run on the cloud. How do companies deal with having to optimize everything to run on …

deep learning deeplearning deep learning inference edge inference the edge