http://evasivhologram.wordpress.com/

I would like to say a special Thank you to Christian Bovine for helping me build the enclosure. He showed me the basics of wood working, and without him, my project wouldn't be complete!

This is the monster that will run around the screen. He will do the following things:
a. run away from you. He runs away and flaps his arms as his feet move unreasonably fast.
b. if no one is trying to catch him, he stands around for a while.
c. after standing around, he'll get bored and start waving at you.
d. after a while, he'll taunt you, by sticking his tounge out at you.

The animation responds to mouse-x, y positions. It is a series of GIF images that are pulled dynamically in processing, depending on the state of the creature and his relationship with the user.

Motion Tracking
I decided to start all over. I discovered a reason not to use jitter: the JMyron library in processing! This library uses DirectX for video capturing, and so all my problems are solved. I am more comfortable with code than with graphical programming (Jitter), so I decided to start all over for the motion tracking part.

A JMyron is a camera object that uses directX and that allows us to specify things like how sensitive a camera is how often it adapts, etc. I relied heavily on the code examples that came with the library. But this is the basic logic: I take a picture of the current background before startting the motion tracking part. Then I call the differenceImage function of the camera object that will take a difference of the "control" image and the new image and finds the difference between. This differenced image is what I use for tracking. The differenced image will create white spots/areas where the differences are, and the JMyron will track white in the Matrix. All the points of change are then averaged to get a center point, or the average location of motion.

Camera Location
The camera is now located at the ceiling of the box, pointing downwards at a 20 degree angle. The plexi will be angled at 45 degree, and the monitor will be angled to create the illusion of a hollogram in the camera's direct line of sight.

Todo:
Now I need to build an enclosure to hide everything. I also need to make a button to reset things, and maybe (if I have time) allow the user to scroll through different hollographic characters using a variable restistor.

import JMyron.*;

JMyron m;//a camera object

//variables to maintain the floating green circle
float objx = 640; //160;
float objy = 350; //120;
float objdestx = 640; //160;
float objdesty = 350; //120;

//animated creature clips
AniSprite waving, walking, teasing, standing;
//float xpos, ypos;
float drag = 30.0;

int TEASE_STATE = 500;
boolean teaseState = false;
int teaseframes = 31;
int currentSt = 0;

int CAMERA_WIDTH = 320;
int CAMERA_HEIGHT = 240;

void setup()
{
size(1280, 800); //960);
background(0,0,0); //255, 204, 0);
frameRate(30);
waving = new AniSprite("wave", 17);
walking = new AniSprite("step", 9);
teasing = new AniSprite("tease", teaseframes);
standing = new AniSprite("stand", 11);

//capturing junk
m = new JMyron();//make a new instance of the object
m.start(CAMERA_WIDTH, CAMERA_HEIGHT);//start a capture at 320x240
m.trackColor(255,255,255,256*3-100);//track white
m.update(); //called once per frame, updates vision processing and camera updating
m.adaptivity(10); //how fast/often retina image adapts!
m.adapt();// immediately take a snapshot of the background for differencing
println("Myron " + m.version());
rectMode(CENTER);
noStroke();


}

void draw()
{

m.update();//update the camera view
drawCamera();

int[][] centers = m.globCenters();//get the center points
println("centers size = " + centers.length);
//draw all the dots while calculating the average.
float avX=0;
float avY=0;
for(int i=0;i fill(80);
rect(centers[i][0],centers[i][1],5,5);
avX += centers[i][0];
avY += centers[i][1];
}
if(centers.length-1>0){
avX/=centers.length-1;
avY/=centers.length-1;
}

//draw the average of all the points in red.
fill(255,0,0);
rect(avX,avY,5,5);

//update the location of monster
if(!(avX==0&&avY==0)&¢ers.length>0){
objdestx = avX*3.75;
objdesty = avY*3.75;
println("X = " + avX + ", Y = " + avY);
println("objx = " + objx);
println("objdestx - ojx = " + ( (objdestx - objx)/10.0f));
} //else there is no movement, pull out teasing here

objx += (objdestx-objx)/10.0f;
objy += (objdesty-objy)/10.0f;

objx = constrain(objx, 0, width - walking.getWidth());
objy = constrain(objy, 0, height - walking.getHeight());

/*
//DRAWING THE CREATURE
//think of difx as the new direction and drag as the speed /
float difx = objdestx - objx; //mouseX - xpos;
float dify = objdesty - objy; //mouseY - ypos;
if(abs(difx) > 1.0) {
objx = objx + difx/drag;
objx = constrain(objx, 0, width);
//xpos = xpos + difx/drag;
//xpos = constrain(xpos, 0, width);
}

if(abs(dify) > 1.0) {
objy = objy + dify/drag;
objy = constrain(objy, 0, width);
//ypos = ypos + dify/drag;
//ypos = constrain(ypos, 0, width);
}*/

background(0,0,0);

println("currentSt = " + currentSt);

// Display the sprite at the position xpos, ypos
/* if(mousePressed) {
//background(153, 153, 0);
//waving.display(xpos-waving.getWidth()/2, ypos);
//waving.display(objx, objy);
teasing.display(objx-waving.getWidth()/2, objy);
}
else */
if(centers.length == 0){

currentSt++;
if(currentSt == TEASE_STATE){
teaseState = true;
currentSt = 0;
}

if(teaseState){
if(currentSt <= teaseframes) teasing.display(objx, objy);
else {
currentSt = 0;
teaseState = false;
waving.display(objx, objy);
}
}
else {
//make monster wave at the user, but only after an interval of boredeom (just standing)
if((currentSt < 150) || ((currentSt > 300) && (currentSt < 400)) ) {
standing.display(objx, objy);
}else waving.display(objx, objy); //objx-waving.getWidth()/2, objy);
}

} else {
//background(255, 204, 0);
walking.display(objx, objy); //xpos-waving.getWidth()/2, ypos);
}
}

void drawCamera(){
//for debug:
//m.cameraImage returns reg image
//m.retinaImage returns image adapting to camera
//m.globsImage returns globs preview
//m.differenceImage returns image diff
int[] img = m.differenceImage(); //get the normal image of the camera
loadPixels();
//for(int i=0;i // pixels[i] = img[i]; //draw each pixel to the screen
//}
updatePixels();
}

/*
void mousePressed(){
m.settings();//click the window to get the settings
}*/

public void stop(){
m.stop();//stop the object
super.stop();
}

// Class for animating GIFs

class AniSprite
{
PImage[] ani;
int frame;
int numFrames;

AniSprite(String imageName, int frameCount) {
numFrames = frameCount;
ani = new PImage[numFrames];
loadImages(imageName);
}

void loadImages(String name) {
for(int i=0; i String imageName = name + i + ".gif";
ani[i] = loadImage(imageName);
}
}

void display(float xpos, float ypos)
{
frame = (frame+1)%numFrames;
image(ani[frame], xpos, ypos);
}

int getWidth() {
return ani[0].width;
}

int getHeight(){
return ani[0].height;
}
}

It took a day to figure out a way to hack into the hardrive without logging into my computer, using the command prompt. I manually copied every file, by file, that I needed into an external drive. It was tedious, annoying, but I'm thankful I didn't lose all my work.

Baby steps/Phases

1. enclosure
2. hollogram
3. hand detection
4. videos of creature (or maybe animation)
5. max/msp/jitter

Failures

0. plate glass
1. projection
2. mirrors (spherical/regular flat)
3. IR sensors in the 10 - 80 cm range

Status

1. New setup using a PC monitor
2. 45 degree angled plexi

Problems

1. Detection mechanism: setup isn't accurate enough!!!
Proposed Compromise: will probably use Video Tracking (pcomp book) + IR in the back end.
2. Interaction w/ the creature: where hands enter in a realistic manner
Proposed Compromise: Black cloth, sleeves, no gloves :)

The Midi lab was probably the most confusing, complicated lab I've done. I worked with Sinan and we were amibitious. We initially wanted to use my PC and Adobe Audition to read the Midi channel and play whatever signals we were sending it. The programs would detect signals, but for some reason we couldnt' get that setup to work. Probably because neither of us had any expereince with the program.

So we resorted to the Midi synthesizer box. The code for the lab didn't work for us, and after debugging for like an hour, we just wrote our own code that was simple and straightforward that allows a user to play using a potentiometer.

I will probably need to work with Midi for my final project because it's faster the serial communication. :(

We decided to create an oracle. A bowl of water that will be displaying random, beautiful images until a user comes up to it, waves his/her hand over the bowl of water, asking it a question. The oracle will then respond by displaying a vision/image answering the user's qeury.

The first image is the setup, the second image is the 3 byte output of the arduino, after it recieves a byte input through the serial port.

I'm using serial port COM4 for communication between the processing language and the arduino. TODO: Customize the processing program for another interesting graphic application. Maybe a game w/ a joystick?

11:00 am - L Train, 1st avenue. People were using the machines to charge their MTA cards. People were swiping their cards.

11:15 am - L Train, moving. Almost 1 in every 2 people have headphones on. I'm assuming they're iPods.

11:40 am - M5 Train. There is a sign at the front of the bus that has 6 colored segments that light up when someone pays the bus fee. The "Adult" segment lights up when someone swipes their MTA card or pays in coins.

12:00 pm - M5 Train. People were swiping their MTA cards, to pay for the ride. People were pressing the tape to stop the bus.

12:15 pm - M5 Train, heading uptown. The radio informing the bus driver of the traffic, allowing communication in the bus network.

1:00 pm - M5 Train. A lady in front of me is talking on her cell phone.

1:15 pm - M5 Train. A man w/ headphones boards the bus. Followed by another gentleman. Probably some sort of mp3 player.

1:16 pm - M5 Train. Another woman is chatting animatedly on her cell phone.

4:00 pm - My apartment. The radio is blasting in my roommate's bedroom.

5:00 pm - My apartment. My roommate is on her cell phone. I'm on my laptop, using the wireless router to connect to the internet.

After the lab, I had tried to make the servo motor move like a wheel, only to discover this is not how a servo motor moves. So, instead we decided to work with another variable resistr - the flex sensor alongside a vibrator. We wanted the flex sensor to control the intensity of the vibrator. The more you flex, the higher the vibration. If the flex sensor is straight, then the vibrator is still

We were thinking we might use a vibrator in our midterm project idea #1 - the hospital-friendly pet; whenever a child would hold it/pet it, it would purr/vibrate.

I did this lab and Lab 4 together, since it was the most logical next step.

Once the platform was configured, programming the microcontroller was simple. I attemped to do the combination lock at first, since I didn't read Carlyn's email at that moment of time...

Plan a :: Combo Lock
I was working with Christian at the time, and his idea was to create a combination lock that requires 2 switches to be on, and the 3rd switch needed to be clicked 3 times consecutively to open the lock. We signified this w/ 2 yellow LEDs lighting up. Any wrong combo would result w/ a 3rd green LED emiting light. The difficulty of this combo lock was:
1. Our switch was the "state" switch (for lack of the tech-term), in which you do not know the state of it - hrigh or low. So we needed a way to consider 3 clicks, or rather 3 toggled state changes in sequence.
2. The Arduino's main function is the loop function wich iterates in the range of milliseconds. So in order to sample the state of our switch in time, and click it before the loop iterates, we need to force a delay.

So the main algorithm behind the program was having 3 state variables for the three switches, plus an additional previous state (prevState) variable which will save the last state of the 3rd switch., so we can check for the 3 toggled state changes. Also I added a delay of 500 ms.

This algorithm worked, but when the combo was "unlocked" the LEDs would blink very briefly, b/c of the new loop iteration. But it wasn't a good design b/c of the speed of sampling and coinciding it with the clicks.

Bottom line: I would not use a regular on/off or toggle switch for a combo lock. I just had no other resources.
View Combo Lock File

Plan b :: Binary Counter
When I finally read Carlyn's email, I decided the best use I could make of 3 switches and 5 LEDs was to create a binary counter/timer that would go up to 2^5, then restart. These are the specs:
1. Switch 1 - turned on the binary counter and sustains it. if switch 1 is open, then the counter will pause, but save state.
2. Switch 2 - would turn on/off the serial communiation between the microcontroller and the pc. If turned on, the couner will print in the terminal, the current count in binary (e.g. 10101) ; if turned off, nothing is sent to the terminal.
3. Switch 3 - was a reset button, which will clear all counters and turn off the LEDs.

View Binary Counter File

I've learned that I'm extremely clumsy w/ the solderng iron. I kept soldering the pins together, or breaking something, or just ruining it. Once THAT was out of the way, the rest of the lab was very straight forward. We put the LEDs in series and in parallel, and we used a potentiometer to vary the voltage being fed to the LEDs.

One mistake we made is we used a larger resister than the 220 Ohm. So of course nothing was lighting up. But we would measure the voltage across the LED and we got a reading, but no light. We finallly traced it to the resistor.

Measuring voltage and amperage was simple, once you figured out how to switch b/t the two types of readings on the multimeter.