Essentia
Essentia is an audio analysis and music matching tool. It provides an extensible collection of algorithms mainly used to extract features from audio files and to compare music pieces. It is written in clean, object-oriented C++, has Python bindings and also features a small scripting language designed to make it very easy to create and use feature extractors. The audio analysis part of Essentia has essentially 2 major modes of working:- The standard mode allows the user the load the whole audio into memory and then has access to any part of the audio for analysis. This is very useful for development or for rapidly trying out new features, etc...
- The streaming mode works on audio that is sequentially processed as if it was flowing through the algorithms. This mode is multi-threaded, meaning you get all the benefits of a multi-core CPU, and also needs a lot less memory than the standard mode.
Together with Essentia we have developed the Gaia music matching tool. Gaia indexes music and audio representations that require dedicated similarity metrics, which are not well suited to be indexed with neither the XML search engines nor traditional relational databases. The information that feeds the matching engine is extracted automatically with the Essentia library yet it accepts additional data such as editorial metadata and others as input. It is written in (highly-optimized) C++ and also has Python bindings for easier databases manipulation and experimentation. It can work as a stand-alone services as well as a library. Gaia supports a variety of representations, similarity metrics and search strategies.
Examples of applications that use Essentia are:
