The Color Play user arranges colorful plastic wedges of varying widths to compose a sound loop. Each color represents a pitch and the width of the wedge determines rhythm. The goal is to experience a voluntary feeling of synesthesia – hear pitch by seeing color.

The relationship between sound and color has been a curiosity throughout history. The Color Play was originally inspired by Newton, who created a color wheel with corresponding pitches.

The design is based on a record player, and features removable trays, interchangeable color pieces, and a knob to change the speed of the spin. A light up arrow points to the color sensor made from an RGB LED and a photocell. Because each color absorbs and reflects unique colors of light, it shines Red, Green, and Blue light at the target and records the amount of light that returns. The Arduino sends a midi note to Ableton based on these values.

The Color Play is a project by Louise Foo and Natasha Dzurny from Introduction to Physical Computing at the Interactive Telecommunications Program at NYU. Please contact us or go to our project website to find out more.

(video documentation coming soon)

Process

From the very beginning we wanted to make colors control sound. We were both interested in human vision / hearing and perception and we thought it could be interesting to make a device that translates a visual composition into a musical composition

We brainstormed and played with different ideas for interfaces of color controlling sound

Inspiration

We found inspiration in old record players and toy record players

In the beginning we imagined having a bunch of different discs with different patterns, that you could play on the device.

I looked into the history of visual music, also called color music. It is a genre in the art history, where artist have been experimenting with devices which can translate sounds or music into a related visual presentation. What we wanted to do was the opposite – translating composition of colors into music.

I found this post about the history of optical synthesis by Derek Holzer. It was inspiring to read about the development of the technology of synthesizing sound from light.

We also found great inspiration in works by other contemporary artists experimenting with sensing rotating discs and re-contextualizing nostalgic devices such as the record player .This first one is an ITP project by Inessah Selditz

Since the ancient greek philosopher Plato first linked color to sound, humans have been mapping sound to color in a number of ways. Here are some of the color-sound mappings we’ve found.

In the school of Bauhaus Gertrud Grunow would teach her theory of harmonisation, that was a circle on the floor divided into twelve points. The individual positions of the circle corresponded with the chromatic sequence of notes from C to B and were associated with certain body postures and colors.

The concept  (reinventing the wheel)

We learned that Newton actually had notes assigned to each color in his famous color circle. Here is a drawings from his book Opticks of 1704, showing the colors correlated with musical notes. The spectral colors from red to violet are divided by the notes of the musical scale.The circle completes a full octave.

We decided to make Newton’s color (and sound) circle into a physical device – and bingo! he concept of Color Play was born long before us!

The first Color Play

We came up with a concept of a device that looks a bit like a record player. Each color in a color circle is triggering a sound on a musical scale. In this first version we imagined the sensor (perhaps a camera) sensing the colors from above.

We bult a cardboard model of the device, to play test it

We hooked up a midi keyboard and tried to play what we thought it would sound like when people would spin it back and forth and do scratching

We talked to people about questions like where it felt natural to have knobs, and if they would rather have the device sitting on the wall. The most important lesson we learned from play testing was that people really liked to create there own compositions and that they liked the record-player like design idea.

Checking the sound – color aesthetics

At some point we were considering if we should aim for a chromatic scale, with 12 notes or if we should stick to Newtons 8 notes, so we sketched some color and sound compositions made with a C Major Scale, and 8 rainbow colors, to see if the music and the visual compositions would be interesting enough. After the test, we decided that we liked the simplicity of this aesthetics and that we wanted to wait with a chromatic scale for a later version of the product. In this way, going with the C Major Scale – we will always get a composition that “sounds good”

Building the tray

Making it spin

Making Color Play into a MIDI device

For the first version we played notes with the Arduino Tone library, but we wanted to be able to play different sounds, so we made Color Play into a MIDI device with help from MIDI output LAB and the Arduino To DAW LAB. The Arduino inside Color Play is sending MIDI notes to Ableton based on the color readings.

At some point we experimented with adding beats, but we decided that we really liked the simplicity of the color-sound concept.

The wintershow

Color Play was part of the ITP wintershow 2012 (documentation coming soon)

I am sure that people that have been working with color detection, will agree with Josef Albers’ statement that

“Color is the most relative medium in art” 

Since color is reflected light, and because the light is ever changing in a room, and because the reflected light is affected by other sources of reflected light, that reflects reflected light, it can be a very tricky to work with!

 “Colors and Hues are defined, as are tones in music, by wave length” 

I decided to read the book “Interaction of Color”, by Josef Albers – student and teacher at Weimar Bauhaus, because I found it to be relevant to me and Natasha’s Color Play project. For our project we have built our own color sensor by a light sensor and an RGB LED. Im am just recently beginning to understand how the sensor actually works – how it much like the human vision, reads waves of light physically bouncing off of a surface.

Because our project deals with a color – sound relationship, I read the book with these color-sound glasses on. Reading the book in that context totally made sense, because in “Interaction of Color”, Josef Albers uses several musical analogies such as harmony and harmonization, dissonance, consonance, interval, composition, arrangement and more to explain his color theory and it was great to learn more about Albers thoughts on how the senses of vision and hearing relate. He compares how hearing music depends on the recognition of the in-betweens of the tones, of their placing and of their spacing and to how the concern should be the interaction of color and seeing what happens between colors.

He also talks a lot about the differences in how we perceive color from sound, how we are able to hear a single  tone, but that we almost never see a single color. He states that if one is not able to distinguish the difference between a higher tone and a lower tone, one should probably not make music, and explains how that conclusion can’t really be applied to color, since very few people can distinguish higher and lower light intensity between different hues.

According to Josef Albers, what is the most distinct difference between a musical composition and a color composition is, that music is perceived over time, where colors can be seen in any direction and at any speed.

“different basic conditions of these media result in different behavior” 

Bauhaus was founded with the idea of Gesamtkunstwerk, creating a ‘total’ work of art in which all arts, including architecture would eventually be brought together. To me, ITP could be the Bauhaus of today. This possibility of synthesis of the arts is what I find to be so great and special about ITP. Here you meet people with skills in all the different fields of the arts and beyond.

 “Just as the knowledge of acoustics does not make one musical… so does no color system by itself develop one’s sensitivity for color… What counts here is not so called knowledge, but seing. Seing here implies fantasy and imagination” 

working on the mechanic of the project: motors, speed, torque, construction, design, functionality…

video

Cutting the prototype of the tray and the colored pieces of composition components in the laser cutter

Today in office hours with Nick, he noticed that the reading from our DIY color sensor, was kind of random and not consistant and he found out that the circuit of the photocell was wrong… his advice was to get that right first of all…

After getting that right, and after isolating the sensor inside some electrical tape, to focus the light, it seemed like the sensor was actually working and I started writing down the readings, which seemed to make sense as R, G and B values. The red colored paper would reflect a higher red value and so on…

So then I started working on the If statement and mapped the values to tones… It worked! I was really positively surprised with how well we can sense  so many different shades of colors! I am so happy that now we basically have the color sensor we dreamt of! YAY! Next step is to figure out how to get a faster reading…

Click the image for video documentation:

Me and  Natasha tried to do the H bridge Lab three times, with no luck. First of all we got the same error message as I did with the last motor lab (where my Arduino browned out) After that we tried with another Arduino and that also didn’t work…

Since my last blogpost we found out that our RGB color LED didn’t cycle trough the red, green and blue LED as it was supposed to, but turned on all of them on at once. We found out because it didn’t look like it was doing so, and we checked it by adding a bigger delay to the cycle. Then we made a new simpler circuit with just the LED to make sure we had the wiring right. The code would send out HIGH to the R G and B leg, and one leg would go to ground. We switched that around, so the R,G and B leg was getting LOW and the one leg was going to power. That worked!

I was very happy when  the RGB LED did exactly what the code told it to do!

After getting the RGB LED and switches wiring right I went back to the tone output lab, and added a speaker to the circuit and a C major scale to the code, to test the speaker…

Tomorrow I signed up for office hours with one of the residens to translate this piece of pseudo code to move on with the project:

IF reading of R is between this number and that number AND reading of G is between the this number and that number AND reading of B is between this number and that number THEN play the tone C… (and so on)

Throughout time people have been connecting colors with sound based on different logic.

This example is based on Newtons musical color wheel. We are still deciding which system to use for our project…