Selective Network Linearization for Efficient Private Inference. (arXiv:2202.02340v2 [cs.CR] UPDATED)

Private inference (PI) enables inference directly on cryptographically secure
data.While promising to address many privacy issues, it has seen limited use
due to extreme runtimes. Unlike plaintext inference, where latency is dominated
by FLOPs, in PI non-linear functions (namely ReLU) are the bottleneck. Thus,
practical PI demands novel ReLU-aware optimizations. To reduce PI latency we
propose a gradient-based algorithm that selectively linearizes ReLUs while
maintaining prediction accuracy. We evaluate our algorithm on several standard
PI benchmarks. The results demonstrate up …

arxiv inference linearization network