Note: This bibliographic page is archived and will no longer be updated. For an up-to-date list of publications from the Music Technology Group see the Publications list .

Cross-collection evaluation for music classification tasks

Title Cross-collection evaluation for music classification tasks
Publication Type Conference Paper
Year of Publication 2016
Conference Name 17th International Society for Music Information Retrieval Conference (ISMIR 2016)
Authors Bogdanov, D. , Porter A. , Herrera P. , & Serra X.
Conference Start Date 07/08/2016
Abstract Many studies in music classification are concerned with obtaining the highest possible cross-validation result. However, some studies have noted that cross-validation may be prone to biases and that additional evaluations based on independent out-of-sample data are desirable. In this paper we present a methodology and software tools for cross-collection evaluation for music classification tasks. The tools allow users to conduct large-scale evaluations of classifier models trained within the AcousticBrainz platform, given an independent source of ground-truth annotations, and its mapping with the classes used for model training. To demonstrate the application of this methodology we evaluate five models trained on genre datasets commonly used by researchers for genre classification, and use collaborative tags from as an independent source of ground truth. We study a number of evaluation strategies using our tools on validation sets from 240,000 to 1,740,000 music recordings and discuss the results.
preprint/postprint document