RepoVizz: Online Multimodal Recordings' Repository & Visualizer
RepoVizz is a data repository and visualization tool for structured storage and user-friendly browsing of music performance multi-modal recordings. The primary purpose of RepoVizz is to offer means for the scientific community to gain on-line access to a music performance multi-modal database shared among researchers.
RepoVizz is designed to hold synchronized streams of heterogeneous data (audio, video, motion capture, physiological measures, extracted descriptors, etc.), annotations, and musical scores. Data streams are stored as single-channel PCM files. The motion-sequence format MJPEG is chosen for video. Annotations are stored as text files. MusicXML is used for musical scores. Data is structured by customizable XML skeleton files enabling meaningful retrieval. Skeleton files are first created during data gathering, and provide means to (re-)organize acquired data in any desired hierarchical structure, as they only hold pointers to stored data files. Once a dataset is created and uploaded to the server, each skeleton file defines a view. Multi-track data visualization is done remotely via a powerful HTML5-based environment that enables web-driven editing (add annotations, extract descriptors) and downloading of datasets.
A first instance of RepoVizz has been created within the EU FET-Open Project SIEMPRE, devoted to studying social interaction in ensemble performance.