Architectural Implications of Neural Network Inference for High Data-Rate, Low-Latency Scientific Applications

arXiv:2403.08980v1 Announce Type: new
Abstract: With more scientific fields relying on neural networks (NNs) to process data incoming at extreme throughputs and latencies, it is crucial to develop NNs with all their parameters stored on-chip. In many of these applications, there is not enough time to go off-chip and retrieve weights. Even more so, off-chip memory such as DRAM does not have the bandwidth required to process these NNs as fast as the data is being produced (e.g., every 25 …

abstract applications arxiv chip cs.ar cs.lg data fields inference latency low network networks neural network neural networks nns parameters process rate type