ITP Camp 2024

Generative AI for (real-time) Interactive Installations - Session 2

Date: June 21, 2024 1:30-3pm


Session Leaders: Francisco Barretto


Format: In-person only


Tags: #generative ai #text2image #image2image #real-time #interactive #touchdesigner


In this 2-part session, we will explore the use of Generative AI for creating interactive installations, with a specific focus on Stable Diffusion and Stream Diffusion within TouchDesigner.

We aim to provide a comprehensive overview of key machine-learning models, APIs, and strategies for interactive installations involving GenAI by engaging in hands-on practical examples in TouchDesigner.

The workshop will be structured as follows:

[Session 1 - 06/14 - 01:30pm] https://itp.nyu.edu/camp/2024/session/158

  1. Introduction to Generative AI Generative AI concepts, key models, and StableDiffusion.

  2. Local Generative AI Set up local StableDiffusion, integrate into TouchDesigner, address latency and FPS.

[Session 2 - this session]

  1. Introduction to RT Generative AI StableDiffusion, StreamDiffusion, StreamV2V

  2. Real-Time Generative AI StreamDiffusion + TouchDesigner for real-time applications.

By the end of this session series, participants will have an understanding of generative AI and some practical experience with state-of-the-art tools.

  • This session is Mac-user-friendly and designed for GenAI beginners, requiring no prior knowledge of this topic. Attendees should bring their computers, as we will set up the necessary environment during the session. Basic Touchdesigner is recommended.