Earlier Projects from the class:
These were inspiration to the general idea of building applications for therapists to use for rehabilitation or training.
Progress made by midterm:
Working Demo (unedited):
Earlier Projects from the class:
These were inspiration to the general idea of building applications for therapists to use for rehabilitation or training.
Progress made by midterm:
Working Demo (unedited):
In my research for finals, almost all interactive floor projections using Kinect (that didn’t use proprietary software), used blob detection as the basis for user interaction. I wanted to create a stomp like interaction for which I definitely needed skeletal tracking data.
I struggled for a couple of weeks trying to come up with the perfect solution before I came across the heaven sent solution – I altered and used Andrew Lazarow’s code that accurately mapped skeletal tracking data into my floor projection.
Demo – Stomp test:
Set Up:
Projector and Kinect mounted on ceiling.
What the Kinect RGB Camera sees from that angle:
Make sure the Kinect sees all of the projection and has enough room to see people moving over that space.
More documentation and code to follow as I complete this project.
Big props to Andrew Lazarow, Matthew Tennie and Patrick Hebron for helping put this project together.
Monster Design by Liangjie Xia
There has been no escape from the monsters for me. For my final project I have decided to take this project further and make it into Kinect Interactive Floor Projection.
- Midterm skeletal tracking demo video
What’s next:
- Monster Sequencer – Randomly select/highlight next monster to squish
- Each dying monster explodes in a particle system. Those particles/boids/bubbles then follow the user, clouding the ability to kill the next monster
- Only way to avoid the particle cloud would be to drag it outside the boundary. Higher drag force outside the boundary gives the user a chance to then jump on the next monster. (Adapted from week 8 HW)
- Music experiments : Progressive song vs SoundFx + Sound track
I’m still trying to decide what would be more engaging from a sound design perspective.
- Projection decisions : wall v/s floor v/s elevated platform.
I will also run into the problem of shadows cast on the floor by the user. Matthew Tennie and Andrew Lazarow have a professional/robust solution for this problem that requires 2 (or 3?) projectors and camera to combat that, but for this purpose of a final project, I think I’m just going to cheat with the concept by sequentially choosing monsters only going up the rows.
- Based on the final projection decision , I’ll have to decide how I use the Kinect: blob detection vs skeletal tracking.
Problems to tackle:
1) How does the particle cloud block or inhibit the monster from dying
2) Trial and error – Best position for the Kinect hook up.
3) How to effective sequence the monsters
Projection Test and Proof of Concept:
Ps. Yin has the cutest stomp!
Super ambitious list:
- Cloud of particles, start clustering and monsters start re-spawning
- Level 2
- GA for monsters
(Probably not gonna happen… )
This week I worked with Claire Mitchell to build a crude model of robotic elbow that interfaced with the kinect to mimc the user’s elbow.
The elbow is a combination hinge and pivot joint. The hinge part of the joint lets the arm bend like the hinge of a door; the pivot part lets the lower arm twist and rotate. Several muscles, nerves, and tendons (connective tissues between muscles and bones) cross at the elbow.
I read Childhood and Adolescent Sports-Related Overuse Injuries by KYLE J. CASSAS and AMELIA CASSEttArI-WAYHS published in the American Family Physician in 2006.
I specifically looked at a common condition in young atheletes popularly known as ‘little league elbow’ or a throwing injury in children called Medial Apophysitis.
Repeated pulling can tear ligaments and tendons away from the bone. The tearing may pull tiny bone fragments with it in the same way a plant takes soil with it when it is uprooted. This can disrupt normal bone growth, resulting in deformity.
I found a useful link for young pitchers that lists by age the the types of pitches and number of innings young athletes should play.
This week in Redial some of us took up the daunting task of switching to our own servers. Here’s a detailed account of how it went down..
Register with Rackspace
Launch Server
Configure Sever – Run Scripts
Create new user
Create new Sinatra projectfolder
Create new asterisk user
Run new_sinatra_app
(Outgoing Calls)
Register with flowroute
Configure sip.conf – here
“show sip registry” on Asterisk should show “1 sip registration”
(Incoming Calls)
Register with IPKall
Configure sip.conf with new context for incoming calls – here
Use the same name for incoming calls context and extentions.conf
Call number provided by IPKall to test dialplans
Install cloud9 IDE
Edit the ~/cloud9/config.js file to change username and password
Install Tinyphone from github – check git clone git://github.com/itp-redial/tinyphone/
What this n00b learned the hard way:
- HIT THE PLAY button on simple server on cloud9 for arduino to be able to connect
- While running ruby script to make new Sinatra app, Run it in the new sinatra project folder instead of the scripts folder
- Add everything you need to in the sip.conf and extention.conf files before # comments like (#include /etc/asterisk/userconf_sip.conf) That probably signifies the end of the file..?
- In the event of sinatra errors for outgoing calls chmod 777 /var/spool/asterisk/outgoing
- Cloud9 ‘warning eaddr’ error means that it needs to manually be killed and restarted
$ ps aux | grep node
you PID 1.5 0.2 44172 8260 pts/2 S 15:25 0:00 node app.js
$ kill -2 PID
Respawn Complete!
Working DaftPunk Dialplan:
Calling Arduino:
Previously this worked with Sinatra – mySql – Ajax. I changed the backend to work with node.js. This still needs some debugging, I don’t know why this slows down after a bit.
Arduino Calls you:
Push the button. Receive a call.
For Nature of Code’s Steering Behavior assignment this week, I wanted to create a create a competitive environment. Here are the rules:
I created a system of snakes(inspired from the old school Nokia game) that have these rules:
1) they follow food
2) they get bigger with each bite (size of score = size of snake)
3) going out of the wall/boundary slows them down
4) on crossing each other, the bigger snake kills the small one
More on Craig Reynolds’ Steering Behavior: here
Code for this projectint Num = 20;
int number =0;
PVector food;
PVector mouse;
float d = 25;
ArrayListvehicles;
void setup() {
size(640, 360);
frameRate(30);
vehicles = new ArrayList();
for (int i = 0; i < Num; i++) {
vehicles.add(new Vehicle(random(width), random(height)));
}
food = new PVector(random(0, width-d*2), random(0, height-d*2));
mouse = new PVector(0, 0);
smooth();
}
void draw() {
background(0);
stroke(255);
fill(80);
rectMode(CENTER);
rect(width/2, height/2, width-d*2, height-d*2);
mouse.x = mouseX;
mouse.y = mouseY;
for (Vehicle v : vehicles) {
v.seek(food);
v.update();
v.display();
v.eat();
v.kill(vehicles);
v.boundaries();
// for (int k = 0; k < Num; k++) {
//println(“score of ” + (k+1) + ” is “+ Vehicles[i].score);
//}
}
//Remove dead snakes
Iterator it = vehicles.iterator();
while (it.hasNext ()) {
Vehicle v = it.next();
int i = vehicles.size();
number++;
if (number > i){
number = 0;
}
if (v.dead) {
it.remove();
println(“Arraylist size = ” + i);
}
//
// if (!v.eaten) {
// println(“eaten”);
// println(“score of”+ number + “=” + v.score);
// }
}
}
// Nature of Code HW – Week 8
// Hunger Games
// Adapted from Daniel Shiffman’s ‘Seek_Arrive’ example
class Vehicle {
ArrayListhistory = new ArrayList();
PVector location;
PVector velocity;
PVector acceleration;
float r;
float maxforce; // Maximum steering force
float maxspeed; // Maximum speed
float deadspeed = 0;
int score = 0;
boolean eaten = false; //boolean to check if food was eaten
boolean dead = false;
Vehicle(float x, float y) {
acceleration = new PVector(0, 0);
velocity = new PVector(random (0, 1), random (0, 1));
location = new PVector(x, y);
r = 3.0;
maxspeed = 4;
maxforce = 0.1;
}
// Method to update location
void update() {
// Update velocity
velocity.add(acceleration);
// Limit speed
velocity.limit(maxspeed);
location.add(velocity);
// Reset accelerationelertion to 0 each cycle
acceleration.mult(0);
history.add(location.get());
if (history.size() > score) {
history.remove(0);
}
}
void applyForce(PVector force) {
// We could add mass here if we want A = F / M
acceleration.add(force);
}
// A method that calculates a steering force towards a target
// STEER = DESIRED MINUS VELOCITY
void seek(PVector target) {
PVector desired = PVector.sub(target, location); // A vector pointing from the location to the target
// Normalize desired and scale to maximum speed
desired.normalize();
desired.mult(maxspeed);
// Steering = Desired minus velocity
PVector steer = PVector.sub(desired, velocity);
steer.limit(maxforce); // Limit to maximum steering force
applyForce(steer);
}
void display() {
beginShape();
stroke(255);
noFill();
for (PVector v: history) {
rect(v.x, v.y, 5, 5);
}
endShape();
// Draw a triangle rotated in the direction of velocity
float theta = velocity.heading2D() + PI/2;
//fill(175);
stroke(255);
pushMatrix();
translate(location.x, location.y);
rotate(theta);
beginShape();
vertex(0, -r);
vertex(-r, r);
vertex(r, r);
endShape(CLOSE);
popMatrix();
}
void eat() {
//Display food
fill(175);
stroke(255);
ellipse(food.x, food.y, 30, 30);
//distance calculation
// for(int i=0;i float k = PVector.dist(food, location);
if ( k < 15) {
eaten = true;
score++;
// println (score);
float counter = 1;
for (int j = 0; j fill(175, 0, 0);
stroke(255);
ellipse(food.x, food.y, 50, 50);
// }
}
//New location for food ..
if (eaten == true) {
food.x = random(0, width-d*2);
food.y = random(0, height-d*2);
eaten = false;
}
}
}
void boundaries() {
PVector desired = null;
if (location.x < d) { desired = new PVector(deadspeed, deadspeed); } else if (location.x > width -d) {
desired = new PVector(deadspeed, deadspeed);
}
if (location.y < d) { desired = new PVector(deadspeed, deadspeed); } else if (location.y > height-d) {
desired = new PVector(deadspeed, deadspeed);
}
if (desired != null) {
desired.normalize();
desired.mult(deadspeed);
PVector steer = PVector.sub(desired, velocity);
steer.limit(maxforce);
applyForce(steer);
}
}
// How to make them deactivate each other?
void kill (ArrayListvehicles) {
for (Vehicle other : vehicles) {
//for (PVector loc : other.history) {
float d = PVector.dist(location, other.location);
if (d > 0 && d < r + other.r) {
// println(“COLLISION!”);
// BOTH DIE
//dead = true;
// other.dead = true;
// WHO DIES???
if (score < other.score || score == other.score ) {
dead = true;
println(“dead!”);
}
// set a variable = true, finished = true or dead = true;
// look at the particle system examples
}
}
/*if (dead){
println(“dead!”);
}*/
// score = 0;
// r=0;
}
// If the distance is greater than 0 and less than an arbitrary amount (0 when you are yourself)
// float d2 = PVector.dist(location, other.location);
// if ( d2 < 1 && a < b) {
//}
}
I’m not sure why I want this, but I do..
Over springbreak I was thinking about cameras and image formats and the one thing I wanted to research was.. Have spherical plates been developed and experimented with instead of flat ones to capture images..?

Principles of Panoramic/Spherical Photography so far include:
- Images Taken in succession
- Overlap between images
- Image stitching
- Edge Blending
Experiment with spherical panoramic photography.
When we started visualizing joint angles using the kinect in the biomechanics class, it got me thinking about building more games and visualizing tools for rehab/physical therapy. For my midterm, I developed a tiny game that lets you kill monsters with your right hand and your right knee. Here’s a rough demo that as a presented as my midterm..
I’d like to take this project further by using systems/creatures/forces taught in nature of code and some physical therapy programs to design games that take away the boredom of rehab exercise program.
You might not want to want to flex and extend that knee 20 times a day, but for killing monsters.. yeah!
*Monster Design by Liangjie Xia