Multi-instrument Music Synthesis with Spectrogram Diffusion. (arXiv:2206.05408v2 [cs.SD] UPDATED)

An ideal music synthesizer should be both interactive and expressive,
generating high-fidelity audio in realtime for arbitrary combinations of
instruments and notes. Recent neural synthesizers have exhibited a tradeoff
between domain-specific models that offer detailed control of only specific
instruments, or raw waveform models that can train on any music but with
minimal control and slow generation. In this work, we focus on a middle ground
of neural synthesizers that can generate audio from MIDI sequences with
arbitrary combinations of …

arxiv diffusion music spectrogram