I have two directions I am thinking of going in for my website final project in this class.
Light-Wake:
A friend of mine asked me to collaborate with him to make a net art piece using James Joyce’s Finnegans Wake as source material. At this point we’re not entirely sure how we want the viewer / user to engage with the text, but we want to make a database of all the words in the text, and allow the viewer to encounter them randomly / interactively / non-linearly.
The text is quite complicated, and has been described by some as psychedelic. It is the last book Joyce wrote and it took him 17 years to finish it from when he started writing. Many of the words he used are completely made up, and the book was widely criticized and misunderstood when it was first published. The structure of the novel is comprised of 17 chapters, divided into four books. Here is an existing website that lets people encounter the book according to chapter and page number, with a glossary at the foot of each page.
Our intention, at least at this point, is to have each word in the database appear randomly, perhaps drawing connections between words according to alphabet, syllable length, or even meaning. Also at play in making this piece, neither of us have read the text in full.
Darkness Map:
Last semester I collaborated with two students at ITP, Rune Madsen and Scott Wayne Indiana, to make studies for a project called the Darkness Map, which measures and maps the levels of darkness in New York City at night. For our proof of concept executed last semester, Scott and I walked on each side of the street with a camera pointed at the other side and took video for the length of each block, keeping track of where we started and stopped. Rune made a program to analyze the average brightness of each frame of video, and plot the data as a histogram. I took these block-long histograms into Photoshop and organized them as best I could into a representation of the neighborhoods we walked.
The next step in where we would like to take the project would be to make a mobile phone app that would GPS-tag video clips, generate the histograms, and then layout them out “automatically” by the GPS location, not manually. People would be able to download the application, collect data around their neighborhoods, or wherever they chose to walk, and then upload these videos to YouTube, or some other database website. The histograms would be added to the map via the Google Maps API or some other system to orient the histograms correctly, augmenting the existing map with information about the amount and quality of darkness at night.
I plan to build the mobile application soon, and would like to take the opportunity with this assignment to make the website that people would be adding their content to, and seeing the progress as the “dark areas” on the map get filled in.
I remember coming across this article in the New York Times recently, which describes developments in computer vision, especially facial recognition. There are algorithms now for detecting whether a person is smiling or not, which I think is pretty exciting. I wonder what other human emotions might become recognizable to computers, and what that will mean for the future of Human Computer Interaction.
The article also brings up the dark side of face tracking, which I am very concerned about as well. We already live in a society obsessed with surveillance, and it worries me what will happen when distributed cameras will be able to recognize you and track your route around the city, your habits, etc. Eric Schmidt is quoted saying that Google decided to keep facial recognition out of the their mobile app Goggles, which allows users to take a picture of something they encounter, like the Empire State Building or Annie’s Mac and Cheese, and receive information about it. Google realizes the safety risk involved if people were able to take pictures of strangers and then receive information about their name, address, etc. It worries me that at some point other priorities will trump their decision to “avoid encouraging stalking.”
This week’s assignment for Computational Cameras was to research and/or imagine 10 different applications for Face Tracking.
Here are some examples I found by various artists and technologists:
Golan Levin’s Re:Face, uses Face Tracking to make a digital Exquisite Corpse.
Parag Kumar Mital’s piece Memory uses face tracking to embed viewers’ faces into a video sculpture.
Ian Ozsvald’s face detecting HeadroidI is a servo-controlled video camera that follows faces to keep them in view.
A few of my own ideas for face tracking applications:
Thought Bubbles appear over people’s heads
Blushing – detecting embarrassment in participants, and augmenting their coloring to exaggerate it.
Identity Transfer – taking images of previous visitors to an installation, then superimposing them over the current person’s face.
An interactive projected art object that appears differently if it detects a person’s presence in front of it.
Little dark clouds rain over people’s heads, but if they start smiling then the rain stops and the sun comes out.
A video mirror that has a delay built in so that a person’s face is about 5 seconds behind the rest of their body.
An installation connected to Google Picasa so that when people stand in front of a live camera feed, it detects their identity and pulls photos from Picasa where they’re tagged to create a unique environment constructed out of these photos. Also of note, documentation of Picasa’s Face Movie feature, which displays photos in a slideshow by aligning a person’s face in each photo.
For this week’s assignment I chose to try and implement the delayed video mirror idea. I worked from the example code, as well as code I found on a Processing forum in which a key press saves the background image and superimposes that over a person’s face.
Here’s what I’ve got working so far:
//Computational Cameras - Spring 2011//Face Tracking, Video Mirror Delay//Genevieve Hoffman, Feb. 2//Based on code from Dan Shiffman and Processing user midcon - http://forum.processing.org/topic/face-detection-manipulation-with-opencv
import hypermedia.video.*;
import java.awt.Rectangle;
import processing.video.*;
Capture video;
PImage[] history;
OpenCV opencv;
OpenCV opencvOld;
int w =320;
int h =240;
int counter =0;
void setup(){
size(640,240);
history =new PImage[100];
video =new Capture(this, w,h,30);for(int i =0; i < history.length; i++){
history[i]= createImage(w,h,RGB);}
opencv =new OpenCV(this);
opencv.allocate(w, h);
opencv.cascade( OpenCV.CASCADE_FRONTALFACE_ALT );
opencvOld =new OpenCV(this);
opencvOld.allocate(w, h);
opencvOld.cascade( OpenCV.CASCADE_FRONTALFACE_ALT );}
void draw(){if(video.available()){
video.read();
history[counter % history.length].copy(video,0,0,w,h,0,0,w,h);
history[counter % history.length].updatePixels();
counter =(counter +1);
opencv.copy(video);
Rectangle [] faces = opencv.detect();
int before =(counter -50)+ history.length;
opencvOld.copy(history[before % history.length]);
Rectangle [] facesOld = opencvOld.detect();
image(video,0,0);
image(history[before % history.length],320,0);// println (faces.length);/*
// Drawing a Red rectangle over current face detection, and green rectangle over old face
noFill();
stroke(255,0,0);
if (faces.length > 0) {
rect( faces[0].x,faces[0].y,faces[0].width,faces[0].height);
}
stroke(0,255,0);
if (facesOld.length > 0) {
rect( facesOld[0].x+w,facesOld[0].y,facesOld[0].width,facesOld[0].height);
}
*/if(faces.length >0&& facesOld.length >0){copy(history[before % history.length], facesOld[0].x, facesOld[0].y, facesOld[0].width,facesOld[0].height, faces[0].x,faces[0].y,faces[0].width,faces[0].height);}}}
