R-MelNet: Reduced Mel-Spectral Modeling for Neural TTS. (arXiv:2206.15276v1 [cs.SD])

This paper introduces R-MelNet, a two-part autoregressive architecture with a
frontend based on the first tier of MelNet and a backend WaveRNN-style audio
decoder for neural text-to-speech synthesis. Taking as input a mixed sequence
of characters and phonemes, with an optional audio priming sequence, this model
produces low-resolution mel-spectral features which are interpolated and used
by a WaveRNN decoder to produce an audio waveform. Coupled with half precision
training, R-MelNet uses under 11 gigabytes of GPU memory on a single …

arxiv modeling tts