Dr$^2$Net: Dynamic Reversible Dual-Residual Networks for Memory-Efficient Finetuning

arXiv:2401.04105v2 Announce Type: replace
Abstract: Large pretrained models are increasingly crucial in modern computer vision tasks. These models are typically used in downstream tasks by end-to-end finetuning, which is highly memory-intensive for tasks with high-resolution data, e.g., video understanding, small object detection, and point cloud analysis. In this paper, we propose Dynamic Reversible Dual-Residual Networks, or Dr$^2$Net, a novel family of network architectures that acts as a surrogate network to finetune a pretrained model with substantially reduced memory consumption. Dr$^2$Net …

abstract analysis arxiv cloud computer computer vision cs.ai cs.cv data detection dynamic finetuning memory modern networks object paper pretrained models residual resolution small tasks type understanding video video understanding vision