NeuroFlux: Memory-Efficient CNN Training Using Adaptive Local Learning

arXiv:2402.14139v1 Announce Type: new
Abstract: Efficient on-device convolutional neural network (CNN) training in resource-constrained mobile and edge environments is an open challenge. Backpropagation is the standard approach adopted, but it is GPU memory intensive due to its strong inter-layer dependencies that demand intermediate activations across the entire CNN model to be retained in GPU memory. This necessitates smaller batch sizes to make training possible within the available GPU memory budget, but in turn, results in a substantially high and impractical …

abstract arxiv backpropagation challenge cnn convolutional neural network cs.lg demand dependencies edge environments gpu intermediate layer memory mobile network neural network standard training type