QuadraNet V2: Efficient and Sustainable Training of High-Order Neural Networks with Quadratic Adaptation

arXiv:2405.03192v1 Announce Type: new
Abstract: Machine learning is evolving towards high-order models that necessitate pre-training on extensive datasets, a process associated with significant overheads. Traditional models, despite having pre-trained weights, are becoming obsolete due to architectural differences that obstruct the effective transfer and initialization of these weights. To address these challenges, we introduce a novel framework, QuadraNet V2, which leverages quadratic neural networks to create efficient and sustainable high-order learning models. Our method initializes the primary term of the quadratic …

abstract arxiv cs.ai cs.lg datasets differences machine machine learning networks neural networks pre-training process sustainable training transfer type