Generating Multimodal Augmentations with LLMs from Song Metadata for Music Information Retrieval

Abstract

In this work we propose a set of new automatic text augmentations that leverage Large Language Models from song metadata to improve on music information retrieval tasks. Compared to recent works, our proposed methods leverage large language models and copyright-free corpora from web sources, enabling us to release the knowledge sources collected. We show how combining these representations with the audio signal provides a 21% relative improvement on five of six datasets on genre classification, emotion recognition and music tagging, achieving state-of-the-art in three (GTZAN, FMA-Small and Deezer). We demonstrate the benefit of injecting external knowledge sources by comparing them withintrinsic text representation methods that rely only on the sample's information.