MuLan: A Joint Embedding of Music Audio and Natural Language. (arXiv:2208.12415v1 [eess.AS])

Music tagging and content-based retrieval systems have traditionally been
constructed using pre-defined ontologies covering a rigid set of music
attributes or text queries. This paper presents MuLan: a first attempt at a new
generation of acoustic models that link music audio directly to unconstrained
natural language music descriptions. MuLan takes the form of a two-tower, joint
audio-text embedding model trained using 44 million music recordings (370K
hours) and weakly-associated, free-form text annotations. Through its
compatibility with a wide range of …

arxiv audio embedding language music natural natural language