Final Project Progress: visualizing sound

Inspired by Alice. The process of turning sound into numbers and numbers into visuals is tough for exactly the reasons we talked about in class. Sound has many aspects to it that change from instant to instant. Fourier Tranformations? Spectrum size? Changes in waves over time? If you think I remember ANY trigonometry, you’re wrong.

Recall my dream for Amon Tobin-style crowdsourced music visuals, tersly summarized below:

badassdj

This weeks I got the basics of minim down. I am exploring use of the getBand(), noise(), and averaging sound bytes over time. The code below plays the very basic “jingle” file so we can observe how changes in the sound of the music are reflected across the soundbytes. Different axis feature different equalizer-type representations of the sound.

Notice how the equalizers tend to have a long-tail situation in terms of frequency.

Screen Shot 2014-04-21 at 12.48.59 PM

My inner economist thinks the next step is to take the (inverse?) natural log of this equalizer value and see how it looks.

import ddf.minim.analysis.*;
import ddf.minim.*;
import processing.serial.*;
import ddf.minim.ugens.*; 

float x; 
float y;
float l1; 
float l2; 
float l3; 
float l4;  
float l5;
float level; 
float average;

Minim       minim;
AudioPlayer jingle;
FFT         fft;

void setup()
{
  float y = 0;
  float x = 0;
  l1 = 0; 
  l2 = 0; 
  l3 = 0; 
  l4 = 0; 
  l5 = 0;
  size(600, 600, P3D);
  smooth();
  colorMode(RGB);

  minim = new Minim(this);

  // specify that we want the audio buffers of the AudioPlayer
  // to be 1024 samples long because our FFT needs to have 
  // a power-of-two buffer size and this is a good size.
  jingle = minim.loadFile("jingle.mp3", 1024);

  // loop the file indefinitely

  jingle.loop();

  // create an FFT object that has a time-domain buffer 
  // the same size as jingle's sample buffer
  // note that this needs to be a power of two 
  // and that it means the size of the spectrum will be half as large.
  fft = new FFT( jingle.bufferSize(), jingle.sampleRate() );
}

void draw()
{
  background(0);

  // perform a forward FFT on the samples in jingle's mix buffer,
  // which contains the mix of both the left and right channels of the file
  fft.forward( jingle.mix );

  for (int i = 0; i < fft.specSize(); i++)
  {
    // draw the line for frequency band i, scaling it up a bit so we can see it
    x+=0.02;
    y+=1;
    float n = noise(x);
    float t = noise(y);
    float p = noise(i);
    pushMatrix();
    stroke(70, 3, 11, 20);
    translate(310, -140);
    rotate(.5);
    fill (60, 170, 205, t*4);
    rect(t*(width)-50, n*height+10, 50*n, height/2*n);
    rect( n*height+10, t*(width)-50, height/2*n, 50*n);
    popMatrix();
    stroke(255);
    level = fft.getBand(i);
    l1 = level;
    //float logLevel = -1/log(level);
    print(average + " | "+ p + "\n");

    if (average <= 12) {
      stroke(50, 160, 200);
      rect(40, 30, 150, 450);
    };

    if (average <= 18 && average >= 12) {
      stroke(50, 160, 200);
      rect(60, 130, 180, 450);
    };

    if (average <= 40 && average >= 18) {
      stroke(50, 160, 200);
      rect(400, 30, 550, 450);
    };

    if (average > 40) {
      stroke(50, 160, 200);
      rect(350, 130, 120, 600);
    };

    line( i, height, i, height - level*50);
    stroke(255);
    line(i, 0, i, p*100);
    line(average, i, 0, i);
    l5 = l4;
    l4 = l3;
    l3 = l2;
    l2 = l1;
    average = 100*((l1 + l2 + l3 +l4 + l5)/5);
  };
}

 

Leave a Reply