« Intro to Computational Media - Proposal for Final Project - Video Manipulation for Live Performance | Main | tester »
December 12, 2005
LIVE Video Tracking and Manipulation for Theatre Performance
Goal: To create a video-tracking tool for real-time video manipulation.
Context: Cross-media theatre. Specifically for performance where the performers are lit behind a scrim onto which live video is front-projected.
(see our website for visual examples)
How does it work? Two live cameras are used as input. One is an infrared camera which is used for tracking brightness. The other is a regular video camera that provides the live video content for display.
Control is shared between the video operator and the performer onstage. The video operator will turn the system on, and will select which mode is being used. The performer will wear an infrared light as a wearable prop, such as a ring or a bracelet. The infrared camera tracks the position of the light through space, and the display video responds to this movement. In this way, the performer can manipulate the live projected video in real time by moving her hand through space.
THE VIDEOS LINKED BELOW WERE ALL SHOT IN THE ITP LOUNGE USING BRIGHTNESS TRACKING
Mode 1: Rearranging. The performer can "grab" small sections of the video and move them across the image, and then "drop" them over another part of the video. The small sections that are moved remain live after they have been dropped. There is a choice as to whether an empty space is exposed where video has been "grabbed", or not. If there is space, this can be used by the performer as a window in the scrim to look through.
Mode 2: Audio response. The video responds in real time to audio input. The louder the audio input, the greater the fragmentation of the video.
Mode 3: Spotlight. The immediate area around the point that follows the performer's hand remains live video, while the rest of the image is frozen.
Questions:
Slow speed of image. The Processing core does not deal extremely well with live video processing. There were additional modes that I wrote which had to be left out because they slowed down the image too much.
Unreliability of video tracking. Video tracking requires a very controlled environment. Although it may well be possible to create a suitable environment in a theatre context, I wonder at what expense, in terms of lighting and video projection? Also, I doubt it would be worth it in terms of touring, as the amount of time it might take to get the tracking working in each new venue might be disproportionate to the value of the effect.
What does it really add, if the audience is not doing it themselves? The tracking has to be integrated into the aesthetic of the show, and watching the performer manipulate the video must be beautiful or relevant to content, or it just becomes demonstration of media technology.
Perhaps better to fake it? The audience does not care how you do it, as long as it looks right and adds to their experience?
Posted by edpurver at December 12, 2005 03:04 AM