mltk: machine listening toolkit

Michael Simpson

Sound is a multiplicity of qualities and features. What if a tool allowed us to easily extract and use more of these data points in real-time?<br />
<br />
The <i>Machine Listening Toolkit</i> is a project aimed at easing the use of computational audition for creative purposes.


The Machine Listening Toolkit is a software toolkit for streamlining the use of machine listening algorithms. The toolkit was designed to be used in real-time applications and provides access to utilities and data structures that help alleviate the related burdens. MLTK currently includes an openFrameworks add-on, a collection of learning resources, and a standalone tool that can be used to visually explore representations of the data and data flows.

The dashboard allows users to select, configure, and explore a vast array of relevant algorithms and have their output rendered in real-time as a graphic visualization. The dashboard can display a matrix of real-time visualizations which the user can arrange freely. The simultaneous view allows differences between graphs to become apparent and observed. This is useful as a way of better understanding the algorithms and their relationships to each other but also provides a real-time indication of how the algorithms perform and seem most appropriate for a certain task given the sound. The graphs can individually be explored in 3D to help reveal historical trends in the data visually. Visualizations can be selected on-the-fly to make their data stream available to external applications using OSC. Data can also be exported into flat files in several common data file formats.



Thesis Presentation Video