ITP Camp 2024

Leverage real-time speech synthesis in your next time-based media project!

Date: June 08, 2024 4:30-6:30pm


Session Leaders: Yonatan Rozin


Format: Hybrid (In-person with online access)


Tags: #speech #linguistics #synthesis #sound #performance #live #audio #Max #MSP #Web-Audio


session slides

Recommended for anyone interested in live sound, speech synthesis and exploring new, homemade creative tools!

Parametric speech synthesis models the acoustics and parameters at play when humans speak using our physical mouths. With a basic foundational knowledge of linguistics and some patience, you can make a parametric speech synthesizer say pretty much whatever you want! One such parametric voice synthesizer is Pink Trombone, a fantastic open-source interface designed for speech therapy and research. Check it out here:

https://dood.al/pinktrombone

I believe synthesized speech has lots of untapped potential in experimental live performance, and have spent the past month passively developing 2 homemade, VERY MUCH WIP tools that build on Pink Trombone. I'm very excited to introduce them here, show you all how they work and perhaps discuss some potential creative applications of them!

The first tool allows you to create and edit "speech animations" - essentially keyframes for speech parameters such as tongue position, air flow and voice tension. We'll use this tool to create some short speech sequences and in doing so, establish a basic understanding of linguistics and the patterns of speech at play when we talk.

The second tool brings Pink Trombone into Max/MSP with benefits including Midi speech control and easy integration into your existing Max projects.

No programming experience required to start! Familiarity with JSON is beneficial but not required. Access to Max/MSP is recommended. Download it for free here before the session if you want to follow along: https://cycling74.com/downloads, but note you won't be able to save your progress without a paid license.