Lecture 05: Deployment (FSDL 2022)

In this video, we cover the process for turning a promising ML model into a useful ML-powered product.

00:00 Overview
01:59 First, deploy a prototype with gradio or streamlit
04:32 Model-in-server architecture
07:44 Model-in-database architecture
11:42 Model-as-a-service architecture
14:03 REST APIs for model services
16:05 Dependency management for model services
18:26 Containerization for model services with Docker
24:40 Performance optimization: to GPU or not to GPU?
26:56 Optimization for CPUs: distillation, quantization, and caching
29:58 Optimization for GPUs: Batching and …

deployment