Jointist: Joint Learning for Multi-instrument Transcription and Its Applications. (arXiv:2206.10805v2 [cs.SD] UPDATED)

In this paper, we introduce Jointist, an instrument-aware multi-instrument
framework that is capable of transcribing, recognizing, and separating multiple
musical instruments from an audio clip. Jointist consists of the instrument
recognition module that conditions the other modules: the transcription module
that outputs instrument-specific piano rolls, and the source separation module
that utilizes instrument information and transcription results.

The instrument conditioning is designed for an explicit multi-instrument
functionality while the connection between the transcription and source
separation modules is for better …

applications arxiv learning transcription