Overdue #1: ICM Midterm

So for my ICM midterm, I made headway into my planned final project “Thought Bubbles.”

The overall project (which will be blogged about incessantly since it is both my ICM and PhysComp final) is as follows:

1. The User pushes a record button and gives a sound bite of about 1.5 seconds (there will be a prompt).
2. User blows through a fake but model “bubble wand.” Real soap bubbles appear at this point.
3. Whenever a bubble pops, the program will play out of hidden speakers the persons voice, and each subsequent popped bubble will be a random selection from all of the voices previously recorded.

For my ICM midterm, I programmed the Bubble tracking in order to figure out when the bubbles have been “popped.” In this implementation, the bubbles are tracked using OpenCV blob tracking. With a plain background, the camera catches the light reflected from the bubble and recognizes it as an object enough to track it across the screen. The bubbles are denoted on the screen by a blue circle that is 1/5 in length and width of the total pixels occupied by the bubble. I will add a video of this in action as soon as I can find a suitable (free) video capture solution for mac.

Since the Blob tracking class in OpenCV is an arrayList, as of now the bubbles “pop” whenever an object disappears because I have the program recognize that the length of the array has shrunk. In the future, I hope to add a sort of “object permanence” to the bubbles so that the program recognizes bubbles as the program progresses (I have a few ideas, and plan on asking a lot of questions to create this, but any suggestions are helpful).

This is approximately the idea I presented at my ICM midterm, however, I would like to give credit to my class for giving me these ideas:
1. Multiple voices in the array (so it isn’t just one voice played back repeatedly)
2. Prompting the voice response (rather than placing someone on the spot)

Here is what my code looks like (I have not touched this part since the midterm, but I have been working on the sound area):

//Things to add
//Bubble logic (see bottom for explanation)
//blob size is going to be passed out from Arduino,
//such that blobs only fall within a certain range based on bubblesize
// (bubble size will be based on motor speed)

import hypermedia.video.*;

OpenCV vid;
Blob[] bubbles;

String bubblething;

//int[] bubblelocationx;
//int[] bubblelocationy;
int[] totalbubbles = new int[20];   //an array int list to keep track of total
                                    //bubbles on the screen

boolean popped;

void setup() {
  size(640, 480);
  vid = new OpenCV(this);
  vid.capture(width,height);
  smooth();
}

void draw() {
  println(frameRate);
  vid.read();
  image(vid.image(), 0, 0);

  vid.threshold(80);

  bubbles = vid.blobs(100, 10000, 100, false);

  for (int i = 0; i < bubbles.length; i++) {
    //println(bubbles[0].area);
    fill(0, 26, 87);
    ellipse(bubbles[i].centroid.x, bubbles[i].centroid.y,
    bubbles[i].length/5, bubbles[i].length/5);

    fill(0);
    bubblething = "Bubble " + i;
    text(bubblething, bubbles[i].centroid.x + 10,
    bubbles[i].centroid.y + 10);

    totalbubbles[0] = bubbles.length;
    for (int k = 1; k < totalbubbles.length; k++) {
      totalbubbles[k] = totalbubbles[k-1];
      if (totalbubbles[0] - totalbubbles[19] != 0) {
          popped = true;
          fill(255, 0, 0);
          text("Bubble Popped!", width - 100, 20);
          }
       }
     }
} 

void keyPressed() {
 vid.remember();
} 

void bubblepop() {
     if (popped) {
         fill(255, 0, 0);
         text("Bubble Popped!", width - 50, 20);
     }
}
/* Logic for future bubblethings
If the difference between the bubbles' center current location
(bubbles[i].centroid.x, also bubbles[i].centroid.y)
and the previous position is greater than
a guessed/perceived movement rate (20 or something about that),
then the bubble popped (unless the bubble floated off screen)
This would also work since the bubbles jump position when the
objects change...could be dangerous since the lighting levels need
to be  controlled.
if (20 > abs(bubbles[i].centroid.x - bubbles[i].centroid.x (last)){
 set popped = true;
 }

 */

So that is what I have so far concerning the tracking. Keep tight for updates! (My next ICM blog post is due by next Wednesday, so it is coming up.