Quality at the Tail of Machine Learning Inference

arXiv:2212.13925v3 Announce Type: replace
Abstract: Machine learning inference should be subject to stringent inference time constraints while ensuring high inference quality, especially in safety-critical (e.g., autonomous driving) and mission-critical (e.g., emotion recognition) contexts. Neglecting either aspect can lead to severe consequences, such as loss of life and property damage. Many studies lack a comprehensive consideration of these metrics, leading to incomplete or misleading evaluations. The study unveils a counterintuitive revelation: deep learning inference quality exhibits fluctuations due to inference time. …

abstract arxiv autonomous autonomous driving consequences constraints cs.ai cs.cv cs.lg cs.se driving emotion inference life loss machine machine learning mission property quality recognition safety safety-critical studies type