If you have a web cam you can got to http://itp.nyu.edu/dano/face to hover over a web with other people looking at that some page.  Your video image follows your cursor around the page tuning in other video images more sharply when they are close to you on the page.  This needs a Flash 10 plugin.

I have a Master’s Degree in Interactive Telecommunications so I get a lot of calls to help people with their technical systems.  These calls are mostly from my Mom wanting to watch TV and they mostly result from her not understanding the universal remote.  My life on the family AV squad would be helped if I could see what she sees when she calls.  I thought it would be nice to give her a hat with a camera in it.  My first thought was one of those swell network cameras. They are small wireless and I would not have to do any of the netwoking.  Unfortunately my Mom does not have an Internet connection.  My next thought was to use some the solutions that the lifecasting folks are using like Qik but I don’t think that can stream to another phone and she usually need help when I am out and about.  Also I kind of wanted learn how to program for Android so I decided to do it myself on one of those.  I did it as a one day project for ITP’s 4in4.

Before you say how ridiculous it makes people look remember that she is usually in the privacy of her own home and does not have to wear it all the time like the lifecasters.  I only got as far as uploading pictures to the server so you have to watch on a constantly refreshing web browser.  Eventually I would like to be able to point on my phone’s touch screen and have that be reflected by a laser pointer on the camera side.  The code is mostly mushing together the Camera Preview example from the Android api examples and this tutorial for uploading files.  I will past the code below.  I looked at some of the getting started stuff but I got  YALF (yet another language fatigue) thinking about activities and views and intents.  Somehow I got it done without understanding any of that.  My biggest headache was in getting the manifest right when I added another app to the a single project in Eclipse.  In the end the only way I could get it to work was to hand edit the xml in the last tab rather.
package com.osullivision;

import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.net.HttpURLConnection;
import java.net.MalformedURLException;
import java.net.URL;
import java.net.URLEncoder;

import android.app.Activity;
import android.content.Context;
import android.graphics.PixelFormat;
import android.hardware.Camera;
import android.os.Bundle;
import android.util.Log;
import android.view.KeyEvent;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.Window;

public class CameraHat extends Activity implements Camera.PreviewCallback, Camera.ShutterCallback, Camera.PictureCallback {

final String TAG = “CameraHat”;

Camera mCamera;

private Preview mPreview;

boolean freeToSend = true;

ShutterThread myShutterThread;

class ShutterThread extends Thread {

public void run(){
while (true){
if (freeToSend){
freeToSend = false;
Log.v(TAG,”thread ready”);
takePicture();
}
}
}
}
public void takePicture(){
mCamera.takePicture(this, null, this);
}
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);

// Hide the window title.
requestWindowFeature(Window.FEATURE_NO_TITLE);

// Create our Preview view and set it as the content of our activity.
mCamera = Camera.open();
Camera.Parameters p = mCamera.getParameters();
int CAM_W = 160;
int CAM_H = 120;
p.setPictureSize(CAM_W, CAM_H);
p.setPictureFormat(PixelFormat.JPEG);
//p.setPreviewFormat(PixelFormat.JPEG);
mCamera.setParameters(p);
//mCamera.setPreviewCallback(this);
mPreview = new Preview(this);
setContentView(mPreview);
myShutterThread = new ShutterThread();
myShutterThread.start();
}

// ———————————————————————-
@Override
public boolean onKeyDown(int keycode, KeyEvent event) {
Log.v(TAG, “KeyDown” + keycode);
if (keycode == KeyEvent.KEYCODE_CAMERA || keycode == 8) {
mCamera.takePicture(this, null, this);
mCamera.startPreview();
}

super.onKeyDown(keycode, event);
return true;
}

// private Camera.ShutterCallback sillyDelay = new Camera.ShutterCallback() {
public void onPreviewFrame(byte[] data, Camera camera) {
// TODO Auto-generated method stub

if (freeToSend){

freeToSend = false;
Log.v(TAG,”preview”);
new Uploader(“http://itp.nyu.edu/~dbo3/up.php”, data, “androidTest.jpg”);
}
}

public void onShutter() {
Camera.Parameters p = mCamera.getParameters();
int CAM_W = 160;
int CAM_H = 120;
p.setPictureSize(CAM_W, CAM_H);
p.setPictureFormat(PixelFormat.JPEG);
//p.setPreviewFormat(PixelFormat.JPEG);
mCamera.setParameters(p);
}

// };

// private Camera.PictureCallback doSomeThingWithIt = new Camera.PictureCallback() {

public void onPictureTaken(byte[] data, Camera camera) {
Log.v(TAG, “Took Picture” + data.length);
mCamera.stopPreview();
new Uploader(“http://itp.nyu.edu/~dbo3/up.php”, data, “androidTest.jpg”);
mCamera.startPreview();
// doPost(“http://itp.nyu.edu/~dbo3/upity.php”, test.getBytes());

}

// };

class Preview extends SurfaceView implements SurfaceHolder.Callback {
SurfaceHolder mHolder;

Preview(Context context) {
super(context);

// Install a SurfaceHolder.Callback so we get notified when the
// underlying surface is created and destroyed.
mHolder = getHolder();
mHolder.addCallback(this);
mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
}

public void surfaceCreated(SurfaceHolder holder) {
// The Surface has been created, acquire the camera and tell it where
// to draw.

mCamera.setPreviewDisplay(holder);

}

public void surfaceDestroyed(SurfaceHolder holder) {
// Surface will be destroyed when we return, so stop the preview.
// Because the CameraDevice object is not a shared resource, it’s very
// important to release it when the activity is paused.
mCamera.stopPreview();
mCamera = null;
}

public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
// Now that the size is known, set up the camera parameters and begin
// the preview.
//Camera.Parameters parameters = mCamera.getParameters();
///parameters.setPreviewSize(w, h);
//mCamera.setParameters(parameters);
mCamera.startPreview();
}

}

public class Uploader extends Thread {
String urlString;

byte[] payload;

String fileNameOnServer;

public Uploader(String _urlString, byte[] _payload, String _fileNameOnServer) {
urlString = _urlString;
payload = _payload;
fileNameOnServer = _fileNameOnServer;
start();
}

public void run() {
HttpURLConnection conn = null;
DataOutputStream dos = null;
DataInputStream inStream = null;

String lineEnd = “\r\n”;
String twoHyphens = “–”;
String boundary = “—————————–29772313742745″;

try {
// —————— CLIENT REQUEST

URL url = new URL(urlString);
// Open a HTTP connection to the URL
conn = (HttpURLConnection) url.openConnection();

// Allow Inputs
conn.setDoInput(true);
// Allow Outputs
conn.setDoOutput(true);
// Don’t use a cached copy.
conn.setUseCaches(false);
// Use a post method.
conn.setRequestMethod(“POST”);
conn.setRequestProperty(“Connection”, “Keep-Alive”);
// conn.setRequestProperty(“Cookie”, “JSESSIONID=”+PlayList.getSessionId());
conn.setRequestProperty(“Content-Type”, “multipart/form-data;boundary=” + boundary);

dos = new DataOutputStream(conn.getOutputStream());

dos.writeBytes(twoHyphens + boundary + lineEnd);
// dos.writeBytes(“Content-Disposition: form-data; name=\”fileNameOnServer\”"+lineEnd+URLEncoder.encode(fileNameOnServer,”UTF-8″) + lineEnd);
dos.writeBytes((“Content-Disposition: form-data; name=\”data_file\”; filename=\”" + lineEnd + URLEncoder.encode(fileNameOnServer, “UTF-8″) + “\”\r\n”));
dos.writeBytes(“Content-Type: ” + “image/jpg” + ” \r\n”);
dos.writeBytes(lineEnd);
//Log.v(TAG, fileNameOnServer);

Log.v(TAG, “Headers are written”);

// create a buffer of maximum size
dos.write(payload);

dos.writeBytes(lineEnd);
dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);

// close streams
Log.v(TAG, “File is written”);

dos.flush();
dos.close();

} catch (MalformedURLException ex) {
Log.v(TAG, “error: ” + ex.getMessage(), ex);
}

catch (Exception ioe) {
Log.v(TAG, “error: ” + ioe.getMessage(), ioe);
}

// —————— read the SERVER RESPONSE

try {
inStream = new DataInputStream(conn.getInputStream());
String str;

while ((str = inStream.readLine()) != null) {
Log.v(TAG, “Server Response” + str);
}
inStream.close();
freeToSend = true;
Log.v(TAG, “FreeToSend ” + freeToSend);

} catch (Exception ioex) {
Log.v(TAG, “error: ” + ioex.getMessage(), ioex);
}

}
}

}

I used a stretch sensor attached to a bra.  I put in a temperature sensor while I was at it.  Both of them are simple variable resistors that are very easy to use with microcontrollers.  I used a Lilypad microcontroller which is specially designed to be sewn into clothes.  There is a voltage regulator because I only had a 9 volt battery for power but it is not really necessary if you  used something like this or this instead.  Anyway instead of using a breadboard, I connected everything together by using a row of female headers bending all the ground connection to one side and the power to the other, data connections down the middle, eventually hot glue on everything.  It probably would have been easier to use this. 

On the software side I did the example in mobile proccessing but didn’t use their library for the bluetooth connection but instead just pasted in my own class called BTThread.  To keep the code simple I wanted to avoid the whole part where one device “discovers” other devices and services.  Instead I use my laptop to find the address of the bluetooth module (it will look like “00A096179A85″) and 1 hardcoded it.  I also put some lines in there for logging the data but I commented it out because if might slow you down.   The cellphone code and also the tiny arduino code for the Lilypad is below.
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

import javax.microedition.io.Connector;
import javax.microedition.io.StreamConnection;
public class BreathBra extends PMIDlet{
  String BT_ADDRESS = “00A096179Ac5″;  //use your laptop to figure out the hardware address of the blue smirf.

  BTThread blueTooth;

  int xpos;
  PFont font ;
  int recordNum  ;
  String accumulatedFromBT = “”;

  public void setup(){
    font = loadFont(“ArialMT-12.mvlw”);

    // String[] savedPlace = loadStrings(“HBSavedPlace”);
    //if (getLength(savedPlace) > 0) {
    //   recordNum =  Integer.parseInt( savedPlace[0].trim());
    //   }

    blueTooth = new BTThread();
    blueTooth.start();

  }
  public void draw(){

  }
  void blueToothEvent(int input){
    accumulatedFromBT = accumulatedFromBT + (char) input;
    if (input == 10) {
      println(“In From BlueTooth: ” + accumulatedFromBT);
      String[] parts = split(accumulatedFromBT,”,”);
      accumulatedFromBT = “”;  //reset accumulation
      //send a byte out in case other side is waiting to be prompted
      blueTooth.send(65);

      if (getLength(parts) > 1) {
        try {
          int breath = Integer.parseInt((parts[0].trim()))-100;
          int heat = Integer.parseInt((parts[1].trim()))-400;
          //  background(0);
          fill(255, 255, 255);
          rect(0,0,width,30); //clear the place where you are writing text
          text (breath + ” ” + heat , 10,20);
          /// saveStrings(“HBVal”+recordNum,parts);
          // saveStrings(“HBSavedPlace”,new String[] {
          //  String.valueOf(recordNum)                                                                                          }
          // );  //recorde the last place you saved
          recordNum++;
          fill(0, 255, 255);
          ellipse(xpos, breath, 2, 2);
          fill(255, 0, 255);
          ellipse(xpos, heat , 2, 2);
          xpos = xpos + 2;
          if (xpos >= width) {
            xpos = 0;
            background(255,0,0);
          }
        }
        catch (NumberFormatException e) {
          println(“Not a line of Numbers”);
        }
      }

    }
  }

  int getLength(Object[] o)  { //don’t pay attention, servicing a bug in m.p.
    return o.length; //ideally you could just say yourObject.length
  }
  class BTThread extends Thread {
    StreamConnection btConnection = null;
    OutputStream btOutputStream = null;
    /* thread to accumulate bytes  */

    public void run() {
      textFont(font);

      try {
        //surrounds hardware address in bluetooth stuff. “:01″ is for the serial service
        String BTurl = “btspp://” + BT_ADDRESS + “:01;authenticate=false;encrypt=false;master=false”;
        btConnection = (StreamConnection) Connector.open(BTurl);
        try {
          InputStream btInputStream = btConnection.openInputStream();
          btOutputStream = btConnection.openOutputStream();
          ///send a byte out in case other side is waiting to be prompted
          send(65);
          while (true) {
            //read in a byte at a time, this stops for the byte, that is why we are in a thread
            int input = btInputStream.read();
            blueToothEvent(input);
          }
        }
        catch (IOException e) {
          println(“Stopped Listening To BlueTooth”);
        }
      }
      catch (IOException e) {
        println(“Sorry but could not connect. to ” + BT_ADDRESS);
      }
      disconnect();
    }

    void send(int _output){
      try{
        btOutputStream.write( _output);
        btOutputStream.flush();
      }
      catch (IOException e) {
        println(“Stopped Listening To BlueTooth”);
      }
    }

    void disconnect() {
      try {

        btConnection.close();
      }
      catch (Exception e) {
        println(“Trouble Closing Connection.”);

      }

    }

  }
}

ARDUINO:

int analogReading;  //variable int size because adc number potentially as big as 1024

void setup()
{
   beginSerial(9600);  //set up communication back to pc
   //don’t forget to press “serial monitor button”
}

void loop()
{
  analogReading = analogRead(0); //can only use pins 0-5
  Serial.print(analogReading,DEC);
  Serial.print(44,BYTE);
  analogReading = analogRead(1); //can only use pins 0-5
  Serial.print(analogReading,DEC);
  Serial.println(10,BYTE); //talk back to pc
}

It is kind of a kick to see your heart beat.  Polar is a company that makes lots of stuff for athletes to watch their heart beat.  The data usually goes to something like a watch and i think it is pretty hard to get your own hands on the it.  I made this example for students in my Rest of You class to be able to use their heart control somthing or just be able to see the same data in their own visualization. 

You can this fantastic module from Sparkfun or alternatively this cheaper chip.   If you get the fancier one, the serial output pins line up perfectly the “Blue Smirf“ bluetooth module (also from Sparkfun).  Sparkfun is making this stuff so easy I somehow feel guilty.  Anyway I wrote some code to get the readings in and show them on the cell phone.  I did the example in mobile proccessing but didn’t use their library for the bluetooth connection but instead just pasted in my own class called BTThread.  To keep the code simple I wanted to avoid the whole part where one device “discovers” other devices and services.  Instead I use my laptop to find the address of the bluetooth module (it will look like “00A096179A85″) and 1 hardcoded it.  I also put some lines in there for logging the data but I commented it out because if might slow you down.   Here is the code:
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;

import javax.microedition.io.Connector;
import javax.microedition.io.StreamConnection;
public class MobileHeart extends PMIDlet{
  String BT_ADDRESS = “00A096179A85″;  //use your laptop to figure out the hardware address of the blue smirf.

  BTThread blueTooth;

  int xpos;
  PFont font ;
  int recordNum  ;
  String accumulatedFromBT = “”;
  String  filePrefix = “mhb”;  //distinguishes log file from other apps.

  int timeOfLastReading = 0;
  int interval = 1000;  //how often you want to read

  public void setup(){
    font = loadFont(“ArialMT-12.mvlw”);

    String[] savedPlace = loadStrings( filePrefix + “SavedPlace”);
    //if (getLength(savedPlace) > 0) {
    // recordNum =  Integer.parseInt( savedPlace[0].trim());
    //}

    blueTooth = new BTThread();
    blueTooth.start();

  }

  public void draw(){
    if (millis() – timeOfLastReading > interval && blueTooth != null){
      blueTooth.send(71);
      blueTooth.send(49);
      blueTooth.send(13);
      timeOfLastReading = millis();
    }

  }
  void blueToothEvent(int input){
    accumulatedFromBT = accumulatedFromBT + (char) input;

    if (input == 13) {
      println(“In From BlueTooth: ” + accumulatedFromBT);
      String[] parts = split(accumulatedFromBT,” “);
      accumulatedFromBT = “”;  //reset accumulation
      if (getLength(parts) > 2) {
        try {
          int status1 = Integer.parseInt((parts[0].trim()));
          int status2 = Integer.parseInt((parts[1].trim()));
          int heart = Integer.parseInt((parts[2].trim()));
          //  background(0);
          fill(255, 255, 255);
          rect(0,0,width,130); //clear the place where you are writing text
          text (“Heart” + heart, 10,120);
          /*  saveStrings( filePrefix + “Val”+recordNum,parts);
           saveStrings( filePrefix + “SavedPlace”,new String[] {
           String.valueOf(recordNum)                                                                                          }
           );  //recorde the last place you saved
           */
          //recordNum++;

          fill(255, 0, 255);
          ellipse(xpos, heart+ height/2, 2, 2);
          xpos = xpos + 2;
          if (xpos >= width) {
            xpos = 0;
            background(255,0,0);
          }
        }
        catch (NumberFormatException e) {
          println(“Not a line of Numbers”);
        }
      }

    }
  }

  int getLength(Object[] o)  { //don’t pay attention, servicing a bug in m.p.
    return o.length; //ideally you could just say yourObject.length
  }
  class BTThread extends Thread {
    StreamConnection btConnection = null;
    OutputStream btOutputStream = null;
    /* thread to accumulate bytes  */

    public void run() {
      textFont(font);

      try {
        //surrounds hardware address in bluetooth stuff. “:01″ is for the serial service
        String BTurl = “btspp://” + BT_ADDRESS + “:01;authenticate=false;encrypt=false;master=false”;
        btConnection = (StreamConnection) Connector.open(BTurl);
        try {
          InputStream btInputStream = btConnection.openInputStream();
          btOutputStream = btConnection.openOutputStream();
          while (true) {
            //read in a byte at a time, this stops for the byte, that is why we are in a thread
            int input = btInputStream.read();
            blueToothEvent(input);
          }
        }
        catch (IOException e) {
          println(“Stopped Listening To BlueTooth”);
        }
      }
      catch (IOException e) {
        println(“Sorry but could not connect. to ” + BT_ADDRESS);
      }
      disconnect();
    }

    void send(int _output){
      try{
        if (btOutputStream != null){
          btOutputStream.write( _output);
          btOutputStream.flush();
        }
      }
      catch (IOException e) {
        println(“Stopped Listening To BlueTooth”);
      }
    }

    void disconnect() {
      try {

        btConnection.close();
      }
      catch (Exception e) {
        println(“Trouble Closing Connection.”);

      }

    }

  }
}

I am looking for holes in the skin color to find the eyes. When I get back today I will segement out just the holes for the eyes.

Here is the code so far:

import java.awt.AlphaComposite;
import java.awt.Color;
import java.awt.Font;
import java.awt.Graphics2D;
import java.awt.List;
import java.awt.Rectangle;
import java.awt.event.KeyEvent;
import java.awt.event.KeyListener;
import java.awt.image.BufferedImage;
import java.awt.image.Raster;
import java.awt.image.WritableRaster;
import java.util.ArrayList;
import java.util.Arrays;

import blobs.Blob;
import blobs.BlobFinder;
import blobs.BlobInPixelSource;
import blobs.BrightBlob;

import pFaceDetect.PFaceDetect;
import processing.core.PApplet;
import processing.core.PFont;
import processing.core.PImage;
import util.VariableAdjuster;
import vxp.AxisPixelSource;
import vxp.PixelSource;
import java.awt.AlphaComposite;
import java.awt.Color;
import java.awt.Font;
import java.awt.Graphics2D;
import java.awt.Rectangle;
import java.awt.event.KeyEvent;
import java.awt.image.BufferedImage;
import java.awt.image.WritableRaster;
import java.util.ArrayList;
import java.util.Arrays;

import processing.core.PApplet;
import util.VariableAdjuster;
import vxp.AxisPixelSource;
import vxp.PixelSource;

import blobs.Blob;
import blobs.BlobFinder;
import blobs.BlobInPixelSource;
import blobs.SkinBlob;

public class DontBlink extends PApplet {

PFont myFont;

int w = 320;

int h = 240;

Rectangle seedArea = new Rectangle(0,0,320,240);

FaceFinder face;
PFaceDetect facecv;
PixelSource ps ;

public void setup() {
size(800, 600);

myFont = createFont(“Arial”, 12);
ps = new AxisPixelSource(“128.122.151.189″, 320, 240, false);
face = new FaceFinder(ps);
facecv = new PFaceDetect(this, ps.getVideoWidth(),ps.getVideoHeight(), “haarcascade_frontalface_default.xml”);

}

public void draw() {
// background(0);
BufferedImage img = ps.getImage();
image(new PImage(img), 0, 0, 320, 240);
//facecv.findFaces(new PImage(img));
//int[][] res = facecv.getFaces();
//if (res.length > 0) {
// println(“Sendit”);
//Rectangle seedArea = new Rectangle(res[0][0],res[0][1],res[0][2],res[0][3]);
img= face.vxpanalyze(seedArea);
image(new PImage(img), 0, 0, 320, 240);
//noFill();
//rect( seedArea.x,seedArea.y, seedArea.width, seedArea.height);

// rect(x, y, w, h);
/*
* for (int i = res.length – 1; i > res.length – 2; i–) { int x = res[i][0]; int y = res[i][1]; int w = res[i][2]; int h = res[i][3]; rect(x, y, w, h); BufferedImage bi = video.getImage().getSubimage(x, x, w, h); new Sender(mySocket, bi); }
*/
//}

// setComposite(AlphaComposite.getInstance(AlphaComposite.SRC_OVER, 0.01f));

}

static public void main(String _args[]) {
PApplet.main(new String[] { “DontBlink” });

}
public void keyPressed(KeyEvent e){
face.keyPressed(e);
}

public class FaceFinder{
BufferedImage myImage;

BufferedImage backgroundImage;

WritableRaster braster;

BufferedImage displayImage;

BlobFinder skinFinder = new BlobFinder(w, h);

VariableAdjuster variableAdjuster;

PixelSource ps;

SkinBlob19 skinBlob;

public int consolidationX = 5;

public int consolidationY = 5;

ArrayList<Blob> existingBlobs = new ArrayList<Blob>();

// ArrayList<Blob> oldEnoughBefore = new ArrayList<Blob>();
public int tooSmall = 1000;

public FaceFinder(PixelSource _ps){
backgroundImage = new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB);
braster = backgroundImage.getRaster();
displayImage = new BufferedImage(w, h, BufferedImage.TYPE_INT_ARGB);
ps = _ps;

//protoBlob = new Blob19(aps);
skinBlob = new SkinBlob19(_ps);

//seedArea = new Rectangle(0, 0, 320, 240);
variableAdjuster = new VariableAdjuster(this, null, ps.getDeviceName());
//variableAdjuster.addObject(protoBlob, “ProtoBlob”);
variableAdjuster.addObject(skinBlob, “SkinBlob”);
//variableAdjuster.addObject(bf, “BlobFinder”);
variableAdjuster.addObject(skinFinder, “SkinFinder”);
}
public int dist(int a, int b, int c, int a1, int b1, int c1) {
return (int) Math.sqrt(Math.pow(a – a1, 2) + Math.pow(b – b1, 2) + Math.pow(c – c1, 2));
}

public void keyPressed(KeyEvent _e) {
variableAdjuster.keyPressed(_e);
}

public BufferedImage vxpanalyze(Rectangle _seedArea) {
if (ps.grabFrame()) {
long now = System.currentTimeMillis();
myImage = ps.getImage();

Graphics2D myGraphics = displayImage.createGraphics();
myGraphics.drawImage(myImage, 0, 0, null);
skinFinder.clearDebugImage();
ArrayList foundBlobs = skinFinder.findBlobs(skinBlob, true, _seedArea);
foundBlobs = BlobFinder.killSmallBlobs(foundBlobs, tooSmall);
skinFinder.consolidate(foundBlobs, consolidationX, consolidationY, false);
BlobFinder.drawRects(Color.PINK, myGraphics, foundBlobs);
skinFinder.lookForContinuity(existingBlobs, foundBlobs, true, false);

myGraphics.drawImage(skinFinder.getDebugImage(), 0, 0, null);
myGraphics.drawImage(variableAdjuster.getImage(), 0, 0, null);

}
return displayImage;
// return myImage;
}

public class SkinBlob19 extends BlobInPixelSource {

public float skinRedLower = .35f;

public float skinRedUpper = .55f;

public float skinGreenLower = .26f;

public float skinGreenUpper = .35f;

public SkinBlob19(PixelSource _ps) {
super(_ps);
}

// public Object clone(){
// if (debugImage == null){
// return new Skin(ps) ;
// }
// return new Skin(ps);
// }
public boolean doesPixelQualify(int[] _rgb) {
int total = (_rgb[0] + _rgb[1] + _rgb[2]);
float nr = (float) _rgb[0] / total;
float ng = (float) _rgb[1] / total;

return (( nr < skinRedUpper && nr > skinRedLower && ng < skinGreenUpper && ng > skinGreenLower));
}

public void shiftRed(float _redShift) {
skinRedUpper = skinRedUpper + _redShift;
skinRedLower = skinRedLower + _redShift;
}

public void shiftGreen(float _greenShift) {
skinGreenUpper = skinGreenUpper + _greenShift;
skinGreenLower = skinGreenLower + _greenShift;
}

public float[] getRanges() {
float[] a = { skinRedLower, skinRedUpper, skinGreenLower, skinGreenUpper };
return a;
}

}
}

}

Allows people to congregate in a layer on top of your web browser. Using a webcam attached to your computer the application grabs your face and sends it to other people looking at the same web page. Your face is displayed on other peoples’ browser at the place on the page you are most interested in as expressed by your cursor location. The web page becomes a substrate for gathering. In the future I would like to degrade the resolution of faces of people interested in things further away on the page and change to peer to peer architecture for the actual transmission of the images. This was done as the Thursday project in the 7 projects in 7 days (5 in 5 for me, … er make that 4 in 5) festival at ITP.

Tech: OpenCV processing libraries for the face detection but I would probably change that. I used IP cameras for the video capture. It also used a new capability in Java 1.6 for overlaying on on other applications. Unfortunately it only allows for a global translucency for the frame, not pixel by pixel transparency. It does allow for not rectilinear shapes so I might change it to that. Below is the code:

Client:

import java.awt.*;
import java.awt.image.BufferedImage;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.UTFDataFormatException;
import java.net.Socket;
import java.net.UnknownHostException;

import javax.swing.*;

import com.sun.image.codec.jpeg.ImageFormatException;
import com.sun.image.codec.jpeg.JPEGCodec;
import com.sun.image.codec.jpeg.JPEGEncodeParam;
import com.sun.image.codec.jpeg.JPEGImageDecoder;
import com.sun.image.codec.jpeg.JPEGImageEncoder;
import com.sun.image.codec.jpeg.TruncatedFileException;

import pFaceDetect.PFaceDetect;
import processing.core.PApplet;
import processing.core.PImage;
import processing.net.Server;
import vxp.AxisPixelSource;
import vxp.CaptureAxisCamera;

public class FaceTheWeb extends PApplet {
Server myServer;

int endOfMessageChar = 10;

Socket mySocket;

int socketPort = 9001;

String ip = “localhost”;

AxisPixelSource video;

boolean reply = true;

static String[] args;

int otherx, othery;

PImage other;

int myConnectionNumber = -1;

PFaceDetect face;

// PImage img;
public void setup() {
size(screen.width,screen.height);
setLocation(0,0);
JFrame.setDefaultLookAndFeelDecorated(true);
com.sun.awt.AWTUtilities.setWindowOpacity(this.frame, 0.5f);
video = new AxisPixelSource(“128.122.151.189″, 320, 240, false);

face = new PFaceDetect(this, video.getVideoWidth(), video.getVideoHeight(), “haarcascade_frontalface_default.xml”);

try {
mySocket = new Socket(ip, socketPort);
} catch (UnknownHostException e) {
println(“Could not find host address”);
} catch (IOException e) {
println(“Could not connect to host”);
}
println(“Connecting as client at ” + socketPort + ” on ” + ip);
new TeleListener(mySocket);

}

static public void main(String _args[]) {
PApplet.main(new String[] { “FaceTheWeb” });

}

public void draw() {
background(255,255,255,0);

if (other != null) {
image(other, otherx, othery, other.width, other.height);
}

}

public void sendPicture() {
if (mySocket != null && video != null) {
BufferedImage img = video.getImage();

face.findFaces(new PImage(img));
int[][] res = face.getFaces();
if (res.length > 0) {
// println(“Sendit”);
int x = res[0][0];
int y = res[0][1];
int w = res[0][2];
int h = res[0][3];
// rect(x, y, w, h);
BufferedImage bi = video.getImage().getSubimage(x, y, w, h);
new Sender(mySocket, bi);
/*
* for (int i = res.length – 1; i > res.length – 2; i–) { int x = res[i][0]; int y = res[i][1]; int w = res[i][2]; int h = res[i][3]; rect(x, y, w, h); BufferedImage bi = video.getImage().getSubimage(x, x, w, h); new Sender(mySocket, bi); }
*/
}
}

}

public void mousePressed() {
sendPicture();
}

public void incoming(PImage _img, int _connectionNumber) {
// println(“INcoming”);
if (_img != null) other = _img;

}

public class TeleListener extends Thread {

DataInputStream dis;

DataOutputStream dout;

int connectionNumber = 0;

TeleListener(Socket _c) {

try {
dis = new DataInputStream(_c.getInputStream());
dout = new DataOutputStream(_c.getOutputStream());
} catch (IOException e) {
println(“Couldn’t Connect”);
}

start();
}

public void run() {

while (true) {
int size = 0;
int connectionNumber = 0;

try {
String header = null;
try {

header = dis.readUTF();
} catch (UTFDataFormatException e) {
System.out.println(” UTFDataFormatException exeption “);
continue;
}
if (header == null) break;
try {
String[] messageParts = header.split(“,”);
size = Integer.parseInt(messageParts[1]);
otherx = Integer.parseInt(messageParts[2]);
othery = Integer.parseInt(messageParts[3]);

} catch (RuntimeException e) {
System.out.println(” RuntimeException exeption “);
continue;
}

if (size == 0) continue;

// make a buffer that size to accept the incoming bytes
byte[] incoming = new byte[size];
dis.readFully(incoming);
PImage myImage = null;
myImage = byteArrayToPImage(incoming);

incoming(myImage, connectionNumber);
// println(connectionNumber + ” Client got” + size);
} catch (NumberFormatException e) {
System.out.println(“Text to number problem.”);
break;
} catch (IOException e) {
System.out.println(“Nothing came in.”);
break;
}

}
}
}

public class Sender extends Thread {
DataOutputStream dos;

DataInputStream dis;

BufferedImage img;

Sender(Socket _c, BufferedImage _image) {
try {
dos = new DataOutputStream(_c.getOutputStream());
dis = new DataInputStream(_c.getInputStream());
} catch (IOException e) {
println(“Couldn’t Connect”);
}
img = _image;
start();
}

public void run() {

byte[] imageInBytes = imageToByteArray(img);
// tell them how many bytes to expect
String outSize = String.valueOf(imageInBytes.length);
String header = “enclosure,” + outSize + “,” + mouseX + “,” + mouseY;

try {
DataOutputStream dos = new DataOutputStream(mySocket.getOutputStream());
dos.writeUTF(header);
dos.flush();
dos.write(imageInBytes);
dos.flush();
} catch (IOException e) {
e.printStackTrace();
}

}

}

public byte[] imageToByteArray(BufferedImage _image) {

// bit if a hack to convert PImage to a Buffered Image, draw it on the buffered image
// ((BufferedImage) image).getRGB(x, y, w, h, img.pixels, offset, img.width);

// BufferedImage bi = new BufferedImage(_image.width, _image.height, BufferedImage.TYPE_INT_RGB);

// bi.setRGB(0, 0, _image.width, _image.height, _image.pixels, 0, _image.width);
// compress and turn into a byte array
// okay just to copy and paste this method

ByteArrayOutputStream baOut = new ByteArrayOutputStream();
JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(baOut);
JPEGEncodeParam param = encoder.getDefaultJPEGEncodeParam(_image);
param.setQuality(0.9f, false);
encoder.setJPEGEncodeParam(param);
try {
encoder.encode(_image);
baOut.flush();
} catch (ImageFormatException e) {
System.out.println(“could not form image” + e);
} catch (IOException e) {
System.out.println(“could not encode image” + e);
}

return baOut.toByteArray();
}

public PImage byteArrayToPImage(byte[] _pic) {
// decompress into a buffferedImage
// okay just to copy and paste this method
ByteArrayInputStream in = new ByteArrayInputStream((byte[]) _pic);
JPEGImageDecoder decoder = JPEGCodec.createJPEGDecoder(in);
BufferedImage snapShot = null;
try {
snapShot = decoder.decodeAsBufferedImage();
} catch (ImageFormatException e) {
System.out.println(“Could not make array into a jpeg. ” + _pic.length);

} catch (TruncatedFileException e) {
System.out.println(“Truncated File” + _pic.length);
} catch (IOException e) {
System.out.println(“IO Making file” + _pic.length);
}

if (snapShot == null) return null;
return new PImage(snapShot);
}

}

Server:

import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.InetAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.ArrayList;

/*
* Created on Feb 7, 2005
*
*/

/**
* @author Dan O’Sullivan
*
* Allows conections to come in and spawns thread to take care of them
*/
public class FaceToWebServer {
int lastConnectionNumber = 0;

ArrayList allConnections = new ArrayList();

public static void main(String[] args) {
new FaceToWebServer();
}

FaceToWebServer() {
ServerSocket frontDoor;
int portNum = 9001;
try {
frontDoor = new ServerSocket(portNum);
System.out.println(“Set up a door at ” + InetAddress.getLocalHost() + ” ” + frontDoor.getLocalPort());
while (true) {
System.out.println(“Waiting for a new connection”);
Socket newSocket = frontDoor.accept();// sits here and waits for // someone to knock on the door
System.out.println(newSocket.getRemoteSocketAddress() + ” knocked on the door and I let them in”);
// farm it out immediately to another thread
ServerThread newServerThread = new ServerThread(newSocket);
allConnections.add(newServerThread); // add it to the list of connections
newServerThread.setConnectionID(lastConnectionNumber);
lastConnectionNumber++;
newServerThread.start();// this calls the run method in the thread
}
} catch (IOException e) {
System.out.println(“Had a problem making a front door” + e);
}
}

synchronized public void tellEveryone(ServerThread _me, String _header, byte[] _enclosure) {

for (int i = 0; i < allConnections.size(); i++) {

ServerThread thisConnection = (ServerThread) allConnections.get(i);
if (thisConnection == _me) continue; //don’t send back to yourself
//insert uniqueName into the header
String outHeader = _header+ “,” + _me.getConnectionID() + “,” + thisConnection.getConnectionID() ;
System.out.println(“Server Distributes ” + outHeader);
new ServerSenderThread(thisConnection.dout,outHeader,_enclosure);

}
}

synchronized public void removeMe(ServerThread _which) {
allConnections.remove(_which);

}

/**
* @author Dan O’Sullivan A thread that listens text and relays it to other people Try to add and introduction that asks for their name and then preppends name to all comments Try adding functionality where you can wisper to a specific person
*/
public class ServerThread extends Thread {
Socket mySocket;

DataInputStream din;

public DataOutputStream dout;

int connectionID =0;

int panPosition = 0;

boolean stillRunning = true;

ServerThread(Socket _socket) {
mySocket = _socket;

// trade the standard byte input stream for a fancier one that allows for more than just bytes
try {
din = new DataInputStream(mySocket.getInputStream());
dout = new DataOutputStream(mySocket.getOutputStream());
} catch (IOException e) {
System.out.println(“couldn’t get streams” + e);
}
}

public void run() {
try {

while (stillRunning) {
String header = null;
byte[] enclosure = null;

header = din.readUTF();

if (header == null) {
killMe();
break;
}

// protocol is type,size,who
String[] headerParts = header.split(“,”);
int size = Integer.parseInt(headerParts[1]);

enclosure = new byte[size];
din.readFully(enclosure);
header = headerParts[0] + “,” + headerParts[1] + “,” + headerParts[2] + “,” + headerParts[3] ;

tellEveryone(this,header, enclosure);
//System.out.println(“server got ” + header );
}
} catch (IOException e) {
killMe();
System.out.println(“Connection Lost”);

}

}

public void killMe() {
System.out.println(“Removing Connection”);
stillRunning = false;
removeMe(this);
}

public void setConnectionID(int _cn){
connectionID = _cn;
}

public int getConnectionID(){
return connectionID;
}

}

/**
* @author Dan O’Sullivan A thread that listens text and relays it to other people Try to add and introduction that asks for their name and then preppends name to all comments Try adding functionality where you can wisper to a specific person
*/
public class ServerSenderThread extends Thread {

DataOutputStream sendOut;
byte[] sendEnclosure;
String sendHeader;

ServerSenderThread(DataOutputStream _dout, String _header, byte[] _enclosure) {
sendOut = _dout;
sendHeader = _header;
sendEnclosure = _enclosure;
start();
}

public void run() {
try {
sendOut.writeUTF(sendHeader);
sendOut.flush();
if (sendEnclosure != null){
sendOut.write(sendEnclosure);
sendOut.flush();
}
} catch (IOException e) {
System.out.println(“Problem sending”);
}

}

}
}

For determining the popularity of different video segments on public displays. It uses a camera and computer vision to notice how many faces are watching the screen. It takes advantage of a shortcoming of the OpenCV processing libraries for the face detection which only detect faces only at very limited angles. It it finds a face you can be assured that they are facing only the screen. Shawn Van Every created a video drop box for people to leave video to be seen on various screens around the floor. We wanted to be able to cull a best of list from the drop box so Shawn VanEvery and I thought face tracking might be interesting. It is currently hardwired to a particular video and only shows the “Attention Quotient” on the screen. If I had more time, I would integrate it with a database of videos and register the attention into that database. This was done as the Tuesday project in the 7 projects in 7 days (5 in 5 for me) festival at ITP.

Tech: I used IP cameras for the video capture. Below is the code:

import java.awt.Color;
import java.awt.Component;
import java.io.File;

import javax.swing.JFrame;

import pFaceDetect.PFaceDetect;
import processing.core.PApplet;
import processing.core.PFont;

import quicktime.QTException;
import quicktime.QTSession;
import quicktime.app.view.MoviePlayer;
import quicktime.app.view.QTFactory;
import quicktime.io.OpenMovieFile;
import quicktime.io.QTFile;
import quicktime.std.StdQTException;
import quicktime.std.movies.Movie;
import vxp.CaptureAxisCamera;

public class VoteWithYourFace extends PApplet{

Movie qtMovie;
MoviePlayer qtPlayer;
Component qtCanvas;
JFrame videoFrame ;
String mediaFileName = “C:\\ALLFOLDERS\\ps3\\video\\others\\get_smart-tlr1_h.320.mov”;
private CaptureAxisCamera video;
private PFaceDetect face;
PFont myFont;
int attention ;
int opportunities;
static public void main(String _args[]) {
PApplet.main(new String[] { “VoteWithYourFace” });
}

public void setup(){
try {
QTSession.open();

System.out.println(“QTSession Open”);

} catch (QTException qte) {
System.out.println(“Sorry NOT: QTSession Open”);
}
try {
OpenMovieFile omf = OpenMovieFile.asRead(new QTFile(mediaFileName));
qtMovie = Movie.fromFile(omf);
} catch (Exception e) {
System.out.println(“Problem opening file”);
}

try {

qtPlayer = new MoviePlayer(qtMovie);
qtCanvas = (QTFactory.makeQTComponent(qtMovie)).asComponent();

} catch (Exception e) {
System.out.println(“Trouble Making Movie”);
}

setBackground(Color.BLACK);

try {
setSize(qtMovie.getNaturalBoundsRect().getWidth(), qtMovie.getNaturalBoundsRect().getHeight());
qtMovie.setTimeValue(0);
qtMovie.start();
} catch (StdQTException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
videoFrame = new JFrame();

videoFrame.add(qtCanvas);
videoFrame.setUndecorated(true);
videoFrame.setSize(getWidth(), getHeight());
videoFrame.setVisible(true);
videoFrame.setLocation(100,100);
video = new CaptureAxisCamera(this,”128.122.151.189″,width,height,false);
face = new PFaceDetect(this,width,height,”haarcascade_frontalface_default.xml”);
myFont = createFont(“Arial”,24);
}

public void draw() {
background(0);
video.read();
face.findFaces(video);
opportunities++;
int [][] res = face.getFaces();
attention = attention + res.length;
String AQ = String.valueOf((float) attention/opportunities);
AQ = AQ.substring(0,Math.min(AQ.length(),4));

textFont(myFont);
text(“Attention Quotient: ” + AQ,10,100);
//image(video,0,0);
// drawFace();
}

void drawFace() {
int [][] res = face.getFaces();
if (res.length>0) {
for (int i=0;i<res.length;i++) {
int x = res[i][0];
int y = res[i][1];
int w = res[i][2];
int h = res[i][3];
rect(x,y,w,h);
}
}
}

}

As people stand in front of a mirror, this applications uses face detection technology to find the faces in the video and then display them with just the faces switched. Each person looks normal except they have someone elses eyes, nose and mouth. This far I go before. In this excersize I wanted it to work for a crowd standing in front of the large screen display at IAC, Gehry building. This required that the faces be shared across multiple application so today I did the networking for that. If I had more time I would add an oval alpha mask so the faces blend better.  This was done as a Monday project in the 7 projects in 7 days (5 in 5 for me) festival at ITP.

Tech: I used the OpenCV processing libraries for the face detection. I used IP cameras for the video capture. Below is the code:

Client:

import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.io.ByteArrayInputStream;
import java.io.ByteArrayOutputStream;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.UTFDataFormatException;
import java.net.Socket;
import java.net.UnknownHostException;

import pFaceDetect.PFaceDetect;
import processing.core.PApplet;
import processing.core.PImage;
import processing.net.Server;
import vxp.CaptureAxisCamera;

import com.sun.image.codec.jpeg.ImageFormatException;
import com.sun.image.codec.jpeg.JPEGCodec;
import com.sun.image.codec.jpeg.JPEGEncodeParam;
import com.sun.image.codec.jpeg.JPEGImageDecoder;
import com.sun.image.codec.jpeg.JPEGImageEncoder;
import com.sun.image.codec.jpeg.TruncatedFileException;

public class FaceLift extends PApplet {

Server myServer;

int endOfMessageChar = 10;

Socket mySocket;

int socketPort = 9001;

String ip = “localhost”;

CaptureAxisCamera video;

boolean reply = true;

static String[] args;

int otherx, othery, otherw, otherh;

PImage other;

int myConnectionNumber = -1;

// PImage img;

static public void main(String _args[]) {
PApplet.main(new String[] { “FaceLift” });
}

PFaceDetect face;

public void setup() {
size(320, 240);
video = new CaptureAxisCamera(this, “128.122.151.189″, width, height, false);

face = new PFaceDetect(this, width, height, “haarcascade_frontalface_default.xml”);
frameRate(15);

noFill();
try {
mySocket = new Socket(ip, socketPort);
} catch (UnknownHostException e) {
println(“Could not find host address”);
} catch (IOException e) {
println(“Could not connect to host”);
}
println(“Connecting as client at ” + socketPort + ” on ” + ip);
new TeleListener(mySocket);

}

void drawFace() {
int[][] res = face.getFaces();
if (res.length > 0) {
for (int i = 0; i < res.length; i++) {
int x = res[i][0];
int y = res[i][1];
int w = res[i][2];
int h = res[i][3];
rect(x, y, w, h);
}
}
}

public void draw() {
background(0);
video.read();
//if (video.available()) {
face.findFaces(video);
image(video, 0, 0);
drawFace();
if (other != null) {

PImage img = createImage(otherw, otherh, ARGB);
int centerx = width/2;
int centery = height/2;
for(int y=0; y < img.height; y++) {
for(int x=0; x < img.width; x++) {
int alpha = (int) (100*dist(x,centerx,y,centery)/width/2);
img.pixels[y*otherw+x] = color(0, 90, 102, alpha);
}

}

other.mask(img);
/*BufferedImage oval = new BufferedImage(otherw, otherh, BufferedImage.TYPE_INT_ARGB);
Graphics2D ovalGraphics = oval.createGraphics();
ovalGraphics.setBackground(new Color(255, 0, 0, 0));
ovalGraphics.clearRect(0, 0, otherw, otherh);
ovalGraphics.setColor(new Color(255, 255, 255, 255));
ovalGraphics.fillOval(0, 0, otherw, otherh);
image(new PImage(oval), otherx, othery, otherw, otherh);
*/
image(other, otherx, othery, otherw, otherh);
}
//sendPicture();
//}
}

public void sendPicture() {
if (mySocket != null && video != null) {

int[][] res = face.getFaces();
if (res.length > 0) {
//println(“Sendit”);
otherx = res[0][0];

othery = res[0][1];
otherw = res[0][2];
otherh = res[0][3];
// rect(x, y, w, h);
BufferedImage bi = video.getImage().getSubimage(otherx, othery, otherw, otherh);
new Sender(mySocket, bi);
/*
* for (int i = res.length – 1; i > res.length – 2; i–) { int x = res[i][0]; int y = res[i][1]; int w = res[i][2]; int h = res[i][3]; rect(x, y, w, h); BufferedImage bi = video.getImage().getSubimage(x, x, w, h); new Sender(mySocket, bi); }
*/
}
}

}

public void mousePressed() {
sendPicture();
}

public void incoming(PImage _img, int _connectionNumber) {
//println(“INcoming”);
if (_img != null) other = _img;

}

public class TeleListener extends Thread {

DataInputStream dis;

DataOutputStream dout;

int connectionNumber = 0;

TeleListener(Socket _c) {

try {
dis = new DataInputStream(_c.getInputStream());
dout = new DataOutputStream(_c.getOutputStream());
} catch (IOException e) {
println(“Couldn’t Connect”);
}

start();
}

public void run() {

while (true) {
int size = 0;
int connectionNumber = 0;

try {
String header = null;
try {

header = dis.readUTF();
} catch (UTFDataFormatException e) {
System.out.println(” UTFDataFormatException exeption “);
continue;
}
if (header == null) break;
try {
String[] messageParts = header.split(“,”);
size = Integer.parseInt(messageParts[1]);

} catch (RuntimeException e) {
System.out.println(” RuntimeException exeption “);
continue;
}

if (size == 0) continue;

// make a buffer that size to accept the incoming bytes
byte[] incoming = new byte[size];
dis.readFully(incoming);
PImage myImage = null;
myImage = byteArrayToPImage(incoming);

incoming(myImage, connectionNumber);
// println(connectionNumber + ” Client got” + size);
} catch (NumberFormatException e) {
System.out.println(“Text to number problem.”);
break;
} catch (IOException e) {
System.out.println(“Nothing came in.”);
break;
}

}
}
}

public class Sender extends Thread {
DataOutputStream dos;

DataInputStream dis;

BufferedImage img;

Sender(Socket _c, BufferedImage _image) {
try {
dos = new DataOutputStream(_c.getOutputStream());
dis = new DataInputStream(_c.getInputStream());
} catch (IOException e) {
println(“Couldn’t Connect”);
}
img = _image;
start();
}

public void run() {

byte[] imageInBytes = imageToByteArray(img);
// tell them how many bytes to expect
String outSize = String.valueOf(imageInBytes.length);
String header = “enclosure,” + outSize;

try {
DataOutputStream dos = new DataOutputStream(mySocket.getOutputStream());
dos.writeUTF(header);
dos.flush();
dos.write(imageInBytes);
dos.flush();
} catch (IOException e) {
e.printStackTrace();
}

}

}

public byte[] imageToByteArray(BufferedImage _image) {

// bit if a hack to convert PImage to a Buffered Image, draw it on the buffered image
// ((BufferedImage) image).getRGB(x, y, w, h, img.pixels, offset, img.width);

// BufferedImage bi = new BufferedImage(_image.width, _image.height, BufferedImage.TYPE_INT_RGB);

// bi.setRGB(0, 0, _image.width, _image.height, _image.pixels, 0, _image.width);
// compress and turn into a byte array
// okay just to copy and paste this method

ByteArrayOutputStream baOut = new ByteArrayOutputStream();
JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(baOut);
JPEGEncodeParam param = encoder.getDefaultJPEGEncodeParam(_image);
param.setQuality(0.9f, false);
encoder.setJPEGEncodeParam(param);
try {
encoder.encode(_image);
baOut.flush();
} catch (ImageFormatException e) {
System.out.println(“could not form image” + e);
} catch (IOException e) {
System.out.println(“could not encode image” + e);
}

return baOut.toByteArray();
}

public PImage byteArrayToPImage(byte[] _pic) {
// decompress into a buffferedImage
// okay just to copy and paste this method
ByteArrayInputStream in = new ByteArrayInputStream((byte[]) _pic);
JPEGImageDecoder decoder = JPEGCodec.createJPEGDecoder(in);
BufferedImage snapShot = null;
try {
snapShot = decoder.decodeAsBufferedImage();
} catch (ImageFormatException e) {
System.out.println(“Could not make array into a jpeg. ” + _pic.length);

} catch (TruncatedFileException e) {
System.out.println(“Truncated File” + _pic.length);
} catch (IOException e) {
System.out.println(“IO Making file” + _pic.length);
}

if (snapShot == null) return null;
return new PImage(snapShot);
}

void getParams() {
if (args != null) { // running as an application
if (args.length > 0) socketPort = Integer.parseInt(args[0]);
if (args.length > 1) {
ip = args[1];

}
} else { // running as an applet

String isItThere = getParameter(“socketPort”);
if (isItThere != null) socketPort = Integer.parseInt(isItThere);
isItThere = getParameter(“ip”);
if (isItThere != null) ip = isItThere;
}
}

}

Server:

import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.net.InetAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.ArrayList;

/**
* @author Dan O’Sullivan
*
* Allows conections to come in and spawns thread to take care of them
*/
public class FaceServer {
int lastConnectionNumber = 0;

ArrayList allConnections = new ArrayList();

public static void main(String[] args) {
new FaceServer();
}

FaceServer() {
ServerSocket frontDoor;
int portNum = 9001;
try {
frontDoor = new ServerSocket(portNum);
System.out.println(“Set up a door at ” + InetAddress.getLocalHost() + ” ” + frontDoor.getLocalPort());
while (true) {
System.out.println(“Waiting for a new connection”);
Socket newSocket = frontDoor.accept();// sits here and waits for // someone to knock on the door
System.out.println(newSocket.getRemoteSocketAddress() + ” knocked on the door and I let them in”);
// farm it out immediately to another thread
ServerThread newServerThread = new ServerThread(newSocket);
allConnections.add(newServerThread); // add it to the list of connections
newServerThread.setConnectionID(lastConnectionNumber);
lastConnectionNumber++;
newServerThread.start();// this calls the run method in the thread
}
} catch (IOException e) {
System.out.println(“Had a problem making a front door” + e);
}
}

synchronized public void tellNextPerson(ServerThread _me, String _header, byte[] _enclosure) {
ServerThread nextPerson = null;

for (int i = 0; i < allConnections.size(); i++) {

ServerThread thisConnection = (ServerThread) allConnections.get(i);
if (thisConnection == _me) { //don’t send back to yourself
int nextIndex = i+1;
if (nextIndex >= allConnections.size()) nextIndex = 0;
nextPerson = (ServerThread) allConnections.get(nextIndex);
String outHeader = _header;
// System.out.println(“Server Distributes ” + outHeader);
new ServerSenderThread(nextPerson.dout,outHeader,_enclosure);
break;
}
}

}

synchronized public void removeMe(ServerThread _which) {
allConnections.remove(_which);

}

/**
* @author Dan O’Sullivan A thread that listens text and relays it to other people Try to add and introduction that asks for their name and then preppends name to all comments Try adding functionality where you can wisper to a specific person
*/
public class ServerThread extends Thread {
Socket mySocket;

DataInputStream din;

public DataOutputStream dout;

int connectionID =0;

int panPosition = 0;

boolean stillRunning = true;

ServerThread(Socket _socket) {
mySocket = _socket;

// trade the standard byte input stream for a fancier one that allows for more than just bytes
try {
din = new DataInputStream(mySocket.getInputStream());
dout = new DataOutputStream(mySocket.getOutputStream());
} catch (IOException e) {
System.out.println(“couldn’t get streams” + e);
}
}

public void run() {
try {

while (stillRunning) {
String header = null;
byte[] enclosure = null;

header = din.readUTF();

if (header == null) {
killMe();
break;
}

// protocol is type,size,who
String[] headerParts = header.split(“,”);
int size = Integer.parseInt(headerParts[1]);

enclosure = new byte[size];
din.readFully(enclosure);
header = headerParts[0] + “,” + headerParts[1] ;

tellNextPerson(this,header, enclosure);
//System.out.println(“server got ” + header );
}
} catch (IOException e) {
killMe();
System.out.println(“Connection Lost”);

}

}

public void killMe() {
System.out.println(“Removing Connection”);
stillRunning = false;
removeMe(this);
}

public void setConnectionID(int _cn){
connectionID = _cn;
}

public int getConnectionID(){
return connectionID;
}

}

/**
* @author Dan O’Sullivan A thread that listens text and relays it to other people Try to add and introduction that asks for their name and then preppends name to all comments Try adding functionality where you can wisper to a specific person
*/
public class ServerSenderThread extends Thread {

DataOutputStream sendOut;
byte[] sendEnclosure;
String sendHeader;

ServerSenderThread(DataOutputStream _dout, String _header, byte[] _enclosure) {
sendOut = _dout;
sendHeader = _header;
sendEnclosure = _enclosure;
start();
}

public void run() {
try {
sendOut.writeUTF(sendHeader);
sendOut.flush();
if (sendEnclosure != null){
sendOut.write(sendEnclosure);
sendOut.flush();
}
} catch (IOException e) {
System.out.println(“Problem sending”);
}

}

}
}

I worked with Tom Igoe, Stephen Lewis, and Joseph O’Connell to develop this. I worked mostly on the video tracking software.