Suspend and Expand is an interactive sound project using the Kinect to map physical gesture to surround sound panning. It is intended for an 8 channel ambisonic audio environment, but can be the installation can be reproduced for any surround sound environment. There is also a headphone implementation using head rotation to interact with sound perspective.
The Kinect code for gesture tracking was written using the SimpleOpenNI Library in Processing, and information is processed for spatial audio panning in max/msp.
For head pose tracking over headphones, the code was adapted from Kyle Mcdonald’s openFrameworks wrapper, and sent via OSC to max/msp.
For surround sound hand panning with the Kinect, the hand location is being compared to the torso, creating the virtual center of the room at the user’s center of mass. As the hand moves 3-dimensionally around the body, so does the sound.
I was also interested in exploring (less successfully–the code at this point is very slow), using colored blob detection to interact with other virtual sources other than the torso. In this version, a user can move their hands around an object to control a 3D sonic image (think crystal ball…)
My final project for Nature of Code is an extension of my midterm and Thesis Project, and involves interacting with immersive 3D audio environments using head rotation. I envision this project as an installation, but also as research for possible interactive audio applications (ie. gaming). In the examples below, the Kinect is being used to sense the user’s head rotation. This information is applied to panning virtual audio sources over headphones.
My midterm project for Nature of Code is an extension of my Thesis Research. In the following two videos, I am interacting with an ambisonic speaker environment ( a type of surround sound setup) using the Kinect sensor. The angle and vector difference of my hands/torso (in the first video) and head/torso ( 2nd video) spatialize audio at X Y and Z coordinates.
wait twenty seconds.
with as little motion as possible, press play
10.
close your eyes. focus on a pitch. hum.
wait 15 seconds. keep humming the pitch. quit the browser.
glissando down as low as possible, creating low pulse with voice. continue as long as breath allows.
Manhattan Ave Street End Park is a quiet waterfront at the northern end of Greenpoint on Newtown Creek, and estuary dividing Brooklyn and Queens. The park serves as a refreshing touch to the expansive and dilapidated industrial scenery along the waterfront. Newton Creek, once a prosperous ecosystem, has recently been designated as a Superfund site after a 30 million gallon oil spill.
This video is a somewhat melancholy (understatement) depiction of Newton Creek, culminating with a hopefull vision of the future.
I am proposing a sound awareness project that employs the natural sounds of the area, creating an extended and subtle sonic environment. I find myself attracted to the inevitable synthesis of natural and industrial sound. Even in seemingly natural and remote environments, one can often hear the ambiance of industry, be it a jet, a bridge, a tunnel, etc. Manhattan Ave Street End Park, displaying a more extreme version of industry, is a prime example (both sonically and visually) of this synthesis.
The goal of the installation is in two parts:
1) to highlight the transition from natural to polluted
2) to blend natural and synthetic sound
As a devout San Franciscan, I am using the Wave Organs (an Exploratrium installation) as inspiration for my project. The Wave Organs are pipes coming up from the bay that project a number of “bandpass filtered” water sounds.
A gate/fence, comprised of a series of hollow pipes, surrounds Manhattan Ave Street End Park. My goal is to turn the hollow pipes into speakers, projecting the amplified sound of the polluted water.
I plan to use this section of the gate for the installation (excuse my drawing skills):
The pipe on the left will play the signal coming directly from the water. A microphone will pick up that sound and relay over to the next pipe (1), and a microphone on pipe (1) will relay the sound over to pipe (2), and so on and so forth (think Lucier and Tudor). At pipe (5), the fully saturated sound will be mixed with the fully saturated sound of a Buchla Synthesizer sample (a patch made to sound similar to the filtered water sound). The original Buchla Sample starts at the right and works its way up to pipe (5) in the same fashion.
There are spaces between each pipe where the listener can observe the transition, pollution, and transformation of the sounds.
How I Will Make This:
I will use a hydrophone to pick up the sound of the water.
I will use sound exciters to convert the pipes ito a series of speakers. Below is a picture of a past project I installed using these devices. Sound exciters are transducers without a resonating body. They turn whatever object they vibrate into the speaker itself.
I will need 2 6 channel amplifiers to power the sound exciters.
To power everything I plan on using something like a car battery.
Bootleg Version:
I did a mock up version of the installation in my living room. I taped together a series of copper pipes, attached a sound exciter to the center, and projected amplified water sounds. I recorded the sound through the resonance of the pipe, put that recording into my computer, and played back the recorded version (again, think Lucier), until natural resonant frequencies of the pipe emerged. I find this process especially interesting in conveying the “pollution” of the natural water sounds; as the sample becomes polluted, it inversely takes on the natural sounds of the pipe. Below are the recorded and re-recorded takes (organized respectively):
And here is the orignal Buchla Synthesizer sample (that would undergo the same process):
Conclusion:
I am motivated to pursue this project for a number of reasons. My primary focus, which has long been a personal creative pursuit, is to provide a sound environment that combines the natural and the industrial. In past projects I have recorded the sounds of creeks under bridges and tunnels and combined those soundscapes further with mechanical sounds (mixed into the preamp input of a Buchla synth). I am particularly excited in using Newtown Creek, which has over time become a synthesis of the natural and industrial via pollution. Life has almost ubiquitously become a combination of these experiences, New York City being the shining example. The purpose of this project is not to “save” Newtown Creek, rather to highlight these experiences. And who knows, maybe 50 years from now the hollow steel gates will open up to a diving board.
Manhattan Ave Street End Park is a quiet waterfront at the northern end of Greenpoint on Newton Creek. The park serves as a refreshing touch to the expansive and dilapidated industrial scenery along the creek’s waterfront. Newton Creek, once a prosperous ecosystem, has recently been designated as a Superfund site after a catastrophic oil spill. In researching the site, I found a number of proposals to provide an ecologically sound design for the waterfront area.
This video is a somewhat melancholy (understatement) depiction of Newton Creek, culminating with a hopefull vision of the future.
I am envisioning a project that highlights the natural sounds of the area, creating an extended sonic environment; the soundscape will be a synthesis of acoustic industrial sound and amplified natural sound. As a devout San Franciscan, I am using the Wave Organs (an Exploratrium installation) as inspiration for my project. The Wave Organs are pipes coming up from the bay that project a number of “bandpass filtered” water sounds.
A gate/fence, comprised of a series of hollow pipes, surrounds Manhattan Ave Street End Park. My goal, in short, is to amplify the sound of the water through these hollow pipes creating an extended acoustic experience. As you can see in the above picture, there are breaks between each pipe. I will create an enclosure for the disconnected space that can be altered; this will give the audience control over the resonance of the “organ.”
Here is an example of an Iphone recording inside one of the pipes:
How I Will Make This:
I will use hydrophones to pick up the sound of the water. I will potentially place remote controlled boats in the water to provide additional sound stimulus (both acoustic and amplified).
I will use sound exciters to convert the pipes into a series of speakers. Below is a picture of a past project I installed using these devices. Sound exciters are transducers without a resonating body. They turn whatever object they vibrate into the speaker itself.
To power everything I plan on using something like a car battery.
Conclusion:
I am motivated to pursue this project for a number of reasons. My primary focus, which has long been a personal creative pursuit, is to provide a sound environment that combines the natural and the industrial. In past projects I have recorded the sounds of creeks under bridges and tunnels and combined those soundscapes further with mechanical sounds (mixed into the preamp input of a Buchla synth). I am particularly excited in using Newton Creek, which has over time become a synthesis of the natural and industrial via pollution. Life has almost ubiquitously become a combination of these experiences, New York City being the shining example. The purpose of this project is not to “save” Newton Creek, rather to highlight these experiences. And who knows, maybe 50 years from now the hollow steel gates will open up to a diving board.
The idea for this project is to elastically mount eight speaker enclosures. When the speakers are stable, the sound level is low and even. When one is moved its volume (and audio content) begins to fluctuate. Since all the speakers are physically connected by some kind of elastic material, pushing one will have an effect on the rest. The size of the speaker will most likely correspond to the output frequency, although this is all subject to change a bit.
Purpose:
My goal for this project is to create an interactive spatial sound piece. The sound will be perceived spatially, but also the interaction with the objects is spatial. Ideally, accelerometers will measure movement through space, and the proximity between each object will be measured. The concept of space will be investigated on microscopic and macroscopic levels.
Questions:
How much input is too much? How simple should I keep it? What kind of materials would work best? Should all the objects be the same, just differ in size?
