Instabilities in Convnets for Raw Audio

arXiv:2309.05855v3 Announce Type: replace
Abstract: What makes waveform-based deep learning so hard? Despite numerous attempts at training convolutional neural networks (convnets) for filterbank design, they often fail to outperform hand-crafted baselines. These baselines are linear time-invariant systems: as such, they can be approximated by convnets with wide receptive fields. Yet, in practice, gradient-based optimization leads to suboptimal approximations. In our article, we approach this phenomenon from the perspective of initialization. We present a theory of large deviations for the energy …

abstract arxiv audio convolutional neural networks cs.lg cs.sd deep learning design eess.as fields gradient linear networks neural networks optimization practice raw systems training type