Beyond Inference: Performance Analysis of DNN Server Overheads for Computer Vision

arXiv:2403.12981v1 Announce Type: cross
Abstract: Deep neural network (DNN) inference has become an important part of many data-center workloads. This has prompted focused efforts to design ever-faster deep learning accelerators such as GPUs and TPUs. However, an end-to-end DNN-based vision application contains more than just DNN inference, including input decompression, resizing, sampling, normalization, and data transfer. In this paper, we perform a thorough evaluation of computer vision inference requests performed on a throughput-optimized serving system. We quantify the performance impact …

abstract accelerators analysis application arxiv become beyond center computer computer vision cs.ai cs.cv cs.dc cs.lg data deep learning deep neural network design dnn faster gpus however inference network neural network part performance performance analysis server tpus type vision workloads