GoodBye WaveNet -- A Language Model for Raw Audio with Context of 1/2 Million Samples. (arXiv:2206.08297v1 [cs.SD])

Modeling long-term dependencies for audio signals is a particularly
challenging problem, as even small-time scales yield on the order of a hundred
thousand samples. With the recent advent of Transformers, neural architectures
became good at modeling dependencies over longer time scales, but they suffered
from quadratic constraints to scale them. We propose a generative
auto-regressive architecture that can model audio waveforms over quite a large
context, greater than 500,000 samples. Our work is adapted to learn time
dependencies by learning …

arxiv audio context language language model wavenet