The Kinect camera from Microsoft XBox is a pretty neat piece of equipment. It combines an IR and RGB camera in one device, but also projects a grid of IR light, and interprets the information it receives back, to create a depth map of any scene in front of it. Basically this means this camera can see in 3D, it can sense depth, which was a limitation of computer vision for quite a long time, as cameras were better suited to tracking sideways and up/down motion, instead of forwards and backwards.
There has been a lot of experimentation with the Kinect once people realized it was a USB device and it would probably be possible to access the camera data from the USB port. Last November, in a matter of days after the product was released, Adafruit Industries sponsored a contest for the first person to successfully “hack” the Kinect, to create a driver / library so that people could use the Kinect as a camera in any way they wished, not just to play an XBox Live game. Since then a community of Kinect hackers has emerged, sharing code, libraries and videos of their experiments with the device. Daniel Shiffman, a professor at ITP, made a library so that the Kinect can be used with Processing, the Java-based language that many of use here to program. If you’re into OpenFrameworks, Zach Lieberman and his crew created a driver to use the Kinect with C++.
Also, Carnegie Mellon just posted a lot of projects made with Kinect in a class taught by Golan Levin, called Interactive Art and Computational Design. Can I take this class?
I have been thinking a lot about what to do for the Kinetic Project in Sustainable Energy. The idea is to find something in your life that you use or do which requires electricity, whether from an outlet or battery, and figure out a way to generate electricity to power whatever it is using your body. One of the most simple ways to generate a fair amount of energy is to turn a DC motor or Stepper Motor around and around. A motor is basically an electromagnet inside of a wire coil, and the motion of moving a magnet around a conductive material like coiled wire generates electricity. This is the reverse process of sending electricity to a motor, which gets it to turn.
For the Kinetic Project I am working with Gabriella Levine and Emily Webster to create hand-powered sound. We have a bunch of ideas for the output, like AM radio, a simple speaker powered by the AC signal coming out of a stepper, walkie talkies. Hopefully we’ll know soon what we want to make, but we’re fairly certain that we’ll use either a hand crank to turn our generator(s), or perhaps a pull cord starter (like the ones on a chain saw) in order to turn the generator more times with each gesture.
We might also incorporate the 555 timer circuits that Eric Rosenthal introduced in Basic Analog Circuits last Friday. Here is a list of a few of the many projects you can make with a 555 timer. The basic idea (as I understand it) is that the 555 IC creates an oscillating signal, which you can use for many purposes, like blinking two LEDs at different rates, giving power to an Arduino pin at specific intervals, or creating a sound wave. Eric Rosenthal pointed me to this project that uses a 555 timer to make an AM radio. Hackaday.com is also running a 555 timer contest right now so there are a lot of other neat projects that people are submitting.
My idea was to try and have the computer measure how many “true” red, green and blue pixels were in its view. I decided to visualize the number of RGB pixels as a sort of meter on the side of the screen. The threshold for how similar a pixel’s value is to 255 is fairly large because in tests with a low threshold I wasn’t seeing any RGB pixels. This could have to do with my not understanding the dist() function, or just the nature of color in live video. Oh, and the other issue I had was my computer’s camera auto-irising after the sketch started running. This makes for “inaccurate” readings and I would like to find a way to disable auto adjustments in the future.
Oh, and the video capture program did something weird to the frame rate, so I look a little stop-motiony.
Here’s my code:
//RGB Color Meter//Genevieve Hoffman//Comp Cameras Spring 2011
import processing.core.PApplet;
import processing.video.Capture;publicclass colorTracking extends PApplet {
Capture camera;
int threshold =120;
int aveX, aveY;//what we're looking for
float objectR =255;
float objectG =0;
float objectB =0;
boolean debug =true;
int lastTime, ellapsedTime;//checking for performance
int rCount =0;
int gCount =0;
int bCount =0;public void setup(){
size(320,240);
camera =new Capture(this,320,240,30);
camera.settings();}public void draw(){if(camera.available()){
ellapsedTime = millis()- lastTime;//time since last frame
lastTime = millis();//reset timer for checking time next frame
camera.read();
int totalFoundPixels =0;//avg location of change pixels
int sumX =0;//need the sum of all the x, y and total
int sumY =0;//loop through the pixelsfor(int row=0; row<camera.height; row++){for(int col=0; col<camera.width; col++){
int offset = row*width + col;
int thisColor = camera.pixels[offset];//pull out individual colors for both pixels
float r = red(thisColor);
float g = green(thisColor);
float b = blue(thisColor);//find dist between colors//float diff = dist(r, g, b, objectR, objectG, objectB);
float diffR = dist(r,g,b,255,0,0);
float diffG = dist(r,g,b,0,255,0);
float diffB = dist(r,g,b,0,0,255);if(diffR < threshold){
rCount++;
println(rCount);}if(diffG < threshold){
gCount++;}if(diffB < threshold){
bCount++;}}
image(camera,0,0);//draw red square
fill(255,0,0);
smooth();
noStroke();
int rowCounterR = height;for(int i=0; i < rCount; i++){
int modulo = i %10;
rect(modulo,rowCounterR,2,2);if(modulo ==0){
rowCounterR -=2;}}//draw green square
fill(0,255,0);
smooth();
noStroke();
int rowCounterG = height;for(int i=0; i < gCount; i++){
int modulo = i %10;
rect(modulo+10,rowCounterG,2,2);if(modulo ==0){
rowCounterG -=2;}}//draw blue square
fill(0,0,255);
smooth();
noStroke();
int rowCounterB = height;for(int i=0; i < bCount; i++){
int modulo = i %10;
rect(modulo+20,rowCounterB,2,2);if(modulo ==0){
rowCounterB -=2;}}}
rCount =0;
gCount =0;
bCount =0;}}public void mousePressed(){//if they click, use that picture for the new thing to follow
int offset = mouseY * width + mouseX;//pull out the same pixel from the current frame
int thisColor = camera.pixels[offset];//pull out the individual colors for both pixels
objectR = red(thisColor);
objectG = green(thisColor);
objectB = blue(thisColor);
println("Chasing new color "+ objectR +" "+ objectG +" "+ objectB);}public void keyPressed(){//for adjusting things on the flyif(key=='s'){
camera.settings();}elseif(key=='-'){
threshold--;
println("Threshold "+ threshold);}elseif(key=='='){
threshold++;
println("Threshold "+ threshold);}elseif(key=='d'){
background(255);
debug =!debug;
println("Debug "+ debug);}elseif(key=='t'){
println("Time Between Frames "+ ellapsedTime);}}}
In light of our debate in last Tuesday’s class about whether reality (and the brain) is blended or discrete, as well as the nature of analog and digital systems, an ad caught my attention as I was reading the interview with Stanislas Dehaene. Scientific American (with the Foundational Questions Institute) is announcing an essay contest for “leading thinkers” to debate the question “Is reality digital or analog?” Rules for entries can be found here. Check back soon to find out the answer since the deadline for entries is February 15th. Although, as the Scientific American blogger’s “mathematician brother” points out, “the question is biased: it presents a binary choice!”
I had to do a fair amount of research this week to really try and understand what Augmented Reality (AR) is and why it’s cool. I think I’m still on the fence but I’m coming around after seeing some the of things it can do, and its ability to change peoples’ experiences of the real world. Here are few examples of what’s possible, though there are many many more.
Talking Business Card:
the cuter version:
Rock Paper Scissors / AR Magazine
Esquire’s AR cover…RDJ just shouldn’t be contained to still images
AR Sightseeing
Augmented Reality Tutoring / Study Aides
AR Helicopter Video Gaming
I saw this demoed at a Sharper Image over Thanksgiving and it was pretty badass.
AR Corporate Schwag – this category is what a lot of AR seems to be used for by businesses, which is to offer a little something extra to delight the customer. For companies that aren’t specifically gaming or entertainment oriented AR is a nice means of cheap viral advertising:
This iPhone app, called TwittARound, let’s you see where people are tweeting from in your area.
Here’s a list of 10 “Amazing AR iPhone apps” from Mashable.com. Highlights include Worksnug, which lets itinerant freelancers without a web connection find places to work with free WiFi. (I really could have used an app like this for the past month when I was without internet due to TimeWarner clerical errors…) I’m not sure which apps actually use AR markers, or how they use them, but they do demonstrate how AR will most likely be used in the future, since so many people have pocket sized computer cameras on their body at all times.
Although many of these examples are pretty neat, it took me watching this TED talk of Blaise Aguera y Arcas demoing Microsoft Bing’s mapping software to get excited about the possibilities of AR and how we will use it in the future. I’ve been thinking about ways to add information to Google Maps, or other mapping programs, that isn’t currently being quantified. My ICM final project last semester attempted to map darkness in New York City at night, but in a minimalist way. It would be nice to have Google Street View display night time images so that people could get a sense for how their surroundings shift appearance over time. Microsoft seems to be on this.
For the assignment this week, I pretty much stuck to using the example code, but played around with inserting my own images. As Clay Shirky would say, I think I’m still in the Lolcatz phase with my understanding AR’s applications, but it’s a fun place to start.
This week’s assignment in Fast…Web was to tweak a WordPress Parent Theme to make your own Child Theme. I had tried to do something similar to make a quick and dirty website to publicize an installation for a friend over the break. I found a theme that I liked, Glossy-Stylo, which let’s you use an image for the background and looks pretty minimal, but I had had trouble changing the php so that it didn’t display the date under the post header, and didn’t show the comments. I guess in some ways I could have tried to make a page instead of a post, but I didn’t think of that at the time. Anyway, I took this assignment as an opportunity to try and disable some of those features on the theme.
Here is a screenshot of the original theme:
And here is one with the changes I made (as well as a new background image, which might be cheating but does make a pretty big difference):
I tried to find a way to make the transparent background behind the text a bit darker, but I had trouble finding both the CSS id as well as what it could be called in the php. Hopefully we can go over that in class tomorrow. Anyway, this assignment made me feel a lot more comfortable messing around with php. At first I tried commenting out the lines I didn’t want displayed, but it made forward slashes appear where I didn’t want them. In the end I just ended up deleting the lines. I wonder if there is a way to comment those out…
Anyway, here is a link to the child theme so you can take a look for yourself.
