Errors:

Using the Capture object in Snow Leopard may get an error like: java.lang.reflect.InvocationTargetException at java.awt.EventQueue.invokeAndWait... FIX: When you go to the "Run" instead of using the usual "Run Applet" pull down to "Run Configurations," and within that over to the "Arguments" tab and with that down to the "VM Arguments" box. Just put in -d32 and run. You should only need to do this once.
Be sure if the processing starts with something like "class BlaBla extends PApplet" that you save your processing sketch named as "BlaBla"
Processing 2.0 uses GStreamer library which requires .start() on the Capture object and doesn't like .settings()

Index

Psuedo Code for All Tracking

setup()

Establish contact with camera by creating a Capture object and putting it into a variable (in this case named video)
- Three ways
  - video = new Capture(this, width, height);
  - video = new Capture(this, width, height, Capture.list()[6]); //get a specific the camera number (do println(Capture.list() to discover number)
  - video = new Capture(this,width,height,"Sony HD Eye for PS3 (SLEH 00201)");//get a specific the camera name (do println(Capture.list() to discover name)
Optionally get dialog box with video.settings();

mainloop()

This can either be inside draw() with a videoObj.available() test or inside captureEvent(Capture _video);
Get a frame of video from the camera using videoObj.read()
Set up nested loops for looking through all the rows and all the columns
- Inside the loops pull out a single pixel from the array of all pixels (sometimes two pixels if you are comparing this frame against a previous)
- Separate out some color or brightness information from that pixel
- Ask a question. The question will vary depending on whether you care about brightness, color, one pixel or many, comparing frames or only the current frame
  - Depending on the answer to the question, set some variables or add to a shape or change the color of an output pixel
Sometimes after looping through all the pixels you will display your conclusions, for instance draw an ellipse at the brightest point.

event handlers()

You might use keyPressed to adjust variables on the the fly
You might use mousePressed to pick new colors off the screen

Dan Shiffman and Patrick Hebron are working on something which may (or may not) become a replacement for Processing's core video library. It is built on top of OpenCV and includes a Movie and Capture class which replicate Processing's existing core video library (no MovieMaker).

We're looking for folks to bang away with it and report back any findings. Note at present it only works on Mac OS X 10.5/10.6 and it does not support sound playback in movie files.

You can download it here:

http://code.google.com/p/processing-video-opencv/downloads/list

To use it simply install to your Processing libraries folder and change:

import processing.video.*; to: import processing.video2.*;

You can file bug reports via e-mail or here:

http://code.google.com/p/processing-video-opencv/issues/list

Finally, we are looking for help with building windows and linux versions of the library if anyone has experience building OpenCV on those platforms, please let me know.

Turn Processing into Java

After the imports but before the variables put "public class BlaBla extends PApplet{" and then put another matching bracket at the bottom.
If you are using the processing editor make sure you save your sketch as "BlaBla." If you are in Eclipse make sure the .java file is named "BlaBla.java."
Add public before the void of all your functions, like "public void setup()" instead of void setup

Run as an Application Instead of an Applet

Add This function
- static public void main(String _args[]) {
- PApplet.main(new String[] { "--present", "InsertNameOfYourMainClass" });
- }
Remember to insert the name of your class.
In Eclipse's "run as", application will automatically appear as an option.

Processing Code for Looking for the Single Best Pixel

import processing.video.*;

Capture video;//regular processing libary

float goalRed = 17; float goalGreen = 123; float goalBlue = 184;

void setup() {

  size(640, 480);
  video = new Capture(this, 640, 480);

}

 void mousePressed(){
  //click to pick a color to chase
  int thisPixel = video.pixels[mouseY* width + mouseX];
  goalRed = red(thisPixel);
  goalGreen = green(thisPixel);
  goalBlue =  blue(thisPixel);
  println("Goal" + goalRed+ " " + goalGreen + " " + goalBlue);

}

void draw() {

  if (video.available()) {
    video.read();
    float closestSoFar = 65000; //start out very far away
    int winnerX = 0;
    int winnerY = 0;
    for(int row = 0; row < video.height; row++){
      for(int col = 0; col < video.width; col++){
        int offset = row*width + col;
        int thisPixel = video.pixels[offset];
        float r = red(thisPixel);
        float g = green(thisPixel);
        float b = blue(thisPixel);
        float closeness = dist(r,g,b,goalRed, goalGreen, goalBlue);
        //check to see if this beats the world record for best match so far

        if (closeness < closestSoFar ){
          winnerX = col;
          winnerY = row;
          closestSoFar = closeness; //up the world record to this match
        }
      }
    }
    image(video,0,0);
    fill(255,0,0);
    ellipse(winnerX, winnerY,30,30); //draw a circle over the winner
  }

}

void keyPressed() {

  if (key == 's') {
    video.settings();
  }

}

Processing Code for Tracking Brightest Rectangle

import processing.video.Capture;

Capture video;// regular processing libary

int threshold = 200; //255 is white, 0 is black

void setup() {

  size(640, 480);
  video = new Capture(this, width, height);
  video.settings();

}

void draw() {

  if (video.available()) {
    video.read();
    int leftMost = width;  //starts out as the right most
    int topMost = height; // starts out as the bottom most
    int rightMost = 0; // starts out left most
    int bottomMost = 0; //sarts out top most
    //enter into the classic nested for statements of computer vision
    for (int row = 0; row < video.height; row++) {
      for (int col = 0; col < video.width; col++) {
        //the pixels file into the room long line you use this simple formula to find what row and column the sit in 
        int offset = row * width + col;  
        int thisPixel = video.pixels[offset];
        float b = brightness(thisPixel);
        if (b > threshold) {  //we are looking for all the pixels that are brighter than the threshold
          video.pixels[offset] = 255; //color it black for debuggin
          //if the pixel qualifies and is outside the current boundaries, stretch the boundaries
          if (col < leftMost) leftMost = col;
          if (col > rightMost) rightMost = col;
          if (row > bottomMost ) bottomMost = row;
          if (row < topMost) topMost = row;
        }
      }
    }
    image(video, 0, 0); //draw the video image (optional)
    //draw the rectangle around the bright pixels
    stroke(255, 0, 255);
    fill(0, 255, 0, 0);
    rect(leftMost, topMost, rightMost-leftMost, bottomMost-topMost);
  }

}

void keyPressed() {

  //for adjusting things on the fly
  if (key == 's') {
    video.settings();
  } 
  else if (key == '-') {
    threshold--;
    println("Threshold " + threshold);
  } 
  else if (key == '=') {
    threshold++;
    println("Threshold " + threshold);
  }

}

Processing Code for Tracking a Color

import processing.video.Capture;

Capture video;// regular processing libary

int threshold = 20; //255 is white, 0 is black

int aveX, aveY; //this is what we are trying to find

float objectR =255;

float objectG = 0;

float objectB = 0;

boolean debug = true;

int lastTime, ellapsedTime; //for checking performance

void setup() {

  size(640, 480);
  println(Capture.list());
  video = new Capture(this,width,height);
  //video.settings();

}

void draw(){

  if (video.available()){
    ellapsedTime = millis() - lastTime;  //find time since last time, only print it out if you press "t"
    lastTime = millis();  //reset timer for checking time next fram
    video.read();
    int totalFoundPixels= 0;  //we are going to find the average location of change pixes so
    int sumX = 0;  //we will need the sum of all the x find, the sum of all the y find and the total finds
    int sumY = 0;
    //enter into the classic nested for statements of computer vision
    for (int row = 0; row < video.height; row++) {
      for (int col = 0; col < video.width; col++) {
        //the pixels file into the room long line you use this simple formula to find what row and column the sit in 

        int offset = row * width + col;
        //pull out the same pixel from the current frame 
        int thisColor = video.pixels[offset];

        //pull out the individual colors for both pixels
        float r = red(thisColor);
        float g = green(thisColor);
        float b = blue(thisColor);

        //in a color "space" you find the distance between color the same whay you would in a cartesian space, phythag or dist in processing
        float diff = dist(r, g, b, objectR, objectG, objectB);

        if (diff < threshold) {  //if it is close enough in size, add it to the average
          sumX = sumX + col;
          sumY= sumY + row;
          totalFoundPixels++;
          if (debug) video.pixels[offset] = 0xff000000;//debugging
        }
      }
    }
    if (debug) image(video,0,0);
    if (totalFoundPixels > 0){
      aveX = sumX/totalFoundPixels;
      aveY = sumY/totalFoundPixels;
      ellipse(aveX-10,(aveY-10),20,20);
    }
  }

}

void mousePressed(){

  //if they click, use that picture for the new thing to follow
  int offset = mouseY * width + mouseX;
  //pull out the same pixel from the current frame 
  int thisColor = video.pixels[offset];

  //pull out the individual colors for both pixels
   objectR = red(thisColor);
   objectG = green(thisColor);
   objectB = blue(thisColor);
  println("Chasing new color  " + objectR + " " + objectG + " " + objectB);

}

void keyPressed() {

  //for adjusting things on the fly
  if (key == 's') {
    video.settings();
  } 
  else if (key == '-') {
    threshold--;
    println("Threshold " + threshold);
  } 
  else if (key == '=') {
    threshold++;
    println("Threshold " + threshold);
  }
  else if (key == 'd') {
    background(255);
    debug = !debug;
    println("Debug " + debug);
  }
  else if (key == 't') {
    println("Time Between Frames " + ellapsedTime);
  }

}

Processing Code for Tracking Change

import processing.core.PApplet;

import processing.video.Capture;

public class TrackCenterChange extends PApplet {

  //save your project at "TrackCenterChange" before running
  Capture video;// regular processing libary
  int threshold = 150;  //255 is white, 0 is black
  PImage backgroundImage;//this is used to hold the previous frame
  int aveX, aveY;  //this is what we are trying to find


  public void setup() {
    size(320, 240);
    println(Capture.list());
   // video = new Capture(this, width, height, Capture.list()[6]); //this is a way to specify a specific camera
   video = new Capture(this, width, height); 
    video.settings();
    backgroundImage = new PImage(width,height);
    grabBackground();  //grab the background now so you have a previous frame in the first loop
  }

  public void grabBackground(){
    backgroundImage.copy(video,0,0,video.width,video.height,0,0,video.width,video.height);
    backgroundImage.updatePixels(); 
  }

  public void draw() {
    if (video.available()) {
      video.read();

      int numberOfChanges = 0;  //we are going to find the average location of change pixes so
      int sumX = 0;  //we will need the sum of all the x find, the sum of all the y find and the total finds
      int sumY = 0;
      //enter into the classic nested for statements of computer vision
      for (int row = 0; row < video.height; row++) {
        for (int col = 0; col < video.width; col++) {
          //the pixels file into the room long line you use this simple formula to find what row and column the sit in 

          int offset = row * width + col;
          //pull out the same pixel from the current frame and the previous frame
          int fgColor = video.pixels[offset];
          int bgColor = backgroundImage.pixels[offset];

          //pull out the individual colors for both pixels
          float fgR = red(fgColor);
          float fgG = green(fgColor);
          float fgB = blue(fgColor);

          float bgR = red(bgColor);
          float bgG = green(bgColor);
          float bgB = blue(bgColor);

          //in a color "space" you find the distance between color the same whay you would in a cartesian space, phythag or dist in processing
          float diff = dist(fgR, fgG, fgB, bgR, bgG, bgB);

          if (diff > threshold) {  //if a pixel qualifies as changed from the previous frame add it to the average
            sumX = sumX + col;
            sumY= sumY + row;
            numberOfChanges++;
          }
        }
      }
      image(video,0,0);
      if (numberOfChanges> 10){
        //find the average location of all the changed pixels
        aveX = sumX/numberOfChanges;
        aveY = sumY/numberOfChanges;
        ellipse(aveX-10,(aveY-10),20,20);
      }
      else{
        //if not much change just us the last average
        ellipse(aveX-10,(aveY-10),20,20);
      }
      grabBackground(); //dont forget to make the current frame the background to check against next time
    }
  }


  public void keyPressed() {
    //for adjusting things on the fly
    if (key == 's') {
      video.settings();
    } 
    else if (key == '-') {
      threshold--;
      println("Threshold " + threshold);
    } 
    else if (key == '=') {
      threshold++;
      println("Threshold " + threshold);
    }
  }

  static public void main(String _args[]) {
    PApplet.main(new String[] { 
      "tracking.TrackCenterChange"                     }
    );
  }

}

Flash Code for Tracking Brightest Single Spot

// Camera var camera:Camera;

// My Video

var videoOut:Video;

// My Bitmapdata

var bmpd:BitmapData;

// Bitmap

var bmp:Bitmap;

var timer:Timer;

var winningX:int; var winningY:int;

VideoProcessing();

function VideoProcessing() {

	//trace("Starting");

	// Setup the camera
	camera=Camera.getCamera();

	// Video components
	videoOut = new Video();

	// Set positions
	//videoOut.x=-0;
	//videoOut.y=0;

	// Attach camera to our video
	videoOut.attachCamera(camera);
	//addChild(videoOut);

	// Create the bitmapdata object
	bmpd=new BitmapData(320,240);

	// Create the bitmap image
	bmp=new Bitmap(bmpd);

	// Add it to the stage
	//bmp.x=0;
	//bmp.y=240;
	//addChild(bmp);

	// Create timer
	timer=new Timer(33,0);
	timer.addEventListener(TimerEvent.TIMER, grabFrame);
	timer.start();

	trace("done starting");

}

function grabFrame(e:TimerEvent):void {

	// Save the frame to the bitmapdata object
	bmpd.draw(videoOut);
	var brightest:int=0;

	for (var row:int=0; row<bmpd.height; row=row+1) {//for each row
		for (var col:int=0; col<bmpd.width; col=col+1) {//for each column
			//get the color of this pixels
			var pix:uint=bmpd.getPixel(col,row);
			var blue:int=pix&0xff;
			if (blue>brightest) {
				winningX=col;
				winningY=row;
				brightest=blue;
			}
		}
	}
	//trace(i + " x " + winningX + " y:" + winningY);
	myInstance.x=winningX;
	myInstance.y=winningY;

}

Track Multiple Color Rects

import processing.video.*;

float goalRed = 200;

float goalGreen = 0;

float goalBlue = 0;

int threshold = 40;

int reach = 5;

long elapsedTime;

Capture video;

void setup() {

  size(640, 480);
  video = new Capture(this, width, height);

}

void draw() {

  if (video.available()) {


    video.read();
    long startTime = millis();
    //video.filter(BLUR, 2);

    elapsedTime = millis() - startTime;
    //println(elapsedTime);  
    //uncomment this line or press 't' to see how long the blur takes.

    ArrayList boxes = new ArrayList();
    for (int row = 0; row < video.height; row++) {
      for (int col = 0; col < video.width; col++) {
        int offset = row * width + col;
        int thisPixel = video.pixels[offset];
        float r = red(thisPixel);
        float g = green(thisPixel);
        float b = blue(thisPixel);
        float closeness = dist(r, g, b, goalRed, goalGreen, goalBlue);
        if (closeness < threshold) {
          video.pixels[offset] = 0;
          //be pessimistic
          boolean foundAHome = false;
          //look throught the existing
          for (int i = 0; i < boxes.size(); i++) {
            Rectangle existingBox =  (Rectangle) boxes.get(i);
            //is this spot in an  existing box
            Rectangle inflatedBox = new Rectangle(existingBox); //copy the existing box
            inflatedBox.grow(reach, reach); //widen it's reach
            if (inflatedBox.contains(col, row)) {
              existingBox.add(col,row);
              foundAHome = true; //no need to make a new one
              break; //no need to look through the rest of the boxes
            }
          }
          //if this does not belong to one of the existing boxes make a new one at this place
          if (foundAHome == false) boxes.add(new Rectangle(col, row,0,0));
        }
      }
    }

    //consolidate(boxes,0,0);
    image(video, 0, 0);
    fill(0, 0, 0, 0);
    stroke(255, 0, 0);
    for (int i = 0; i < boxes.size(); i++) {
      Rectangle thisBox =  (Rectangle) boxes.get(i);
      rect(thisBox.x, thisBox.y, thisBox.width, thisBox.height);
    }


  }

}

void mousePressed() {

  int thisPixel = video.pixels[mouseY * width + mouseX];
  goalRed = red(thisPixel);
  goalGreen = green(thisPixel);
  goalBlue = blue(thisPixel);
  println("Goal" + goalRed + " " + goalGreen + " " + goalBlue);

}

void keyPressed() {

  if (key == 's') {
    video.settings();
  } 
  else if (key == '-') {
    threshold--;
    println("Threshold " + threshold);
  } 
  else if (key == '=') {
    threshold++;
    println("Threshold " + threshold);
  } 
  else if (key == '_') {
    reach--;
    println("reach " + reach);
  } 
  else if (key == '+') {
    reach++;
    println("reach " + reach);
  } 
  else if (key == 't') {

    println("Elapsedtime " + elapsedTime);

  }

}

Small Snippet for Getting Rotation Based on Two spots

 ArrayList boxes = getBoxes();  // too most of the code in the above example and put it into a function called getBoxes, or you might use blobs() from openCV lib. 

    println("boxes" + boxes.size());

    if (boxes.size() == 2){
      Rectangle box1 =  (Rectangle) boxes.get(0);
      Rectangle box2 =  (Rectangle) boxes.get(1);
      if (box1.x > box2.x){
        box2 =  (Rectangle) boxes.get(0);
        box1 =  (Rectangle) boxes.get(1);
      }

      myRotation = atan2(box1.y -box2.y, box1.x -box2.x) ;

Identity using different Lengths

import processing.video.*;

int threshold = 100;

int reach = 50;

long elapsedTime;

Capture video;

void setup() {

  size(640, 480);
  video = new Capture(this, width, height);
  PFont f = loadFont("Serif-48.vlw");
  textFont(f, 44);

}

void draw() {

  if (video.available()) {


    video.read();
    //video.filter(BLUR,2);
    long startTime = millis();
    elapsedTime = millis() - startTime;
    //println(elapsedTime);  
    //uncomment this Liner or press 't' to see how long the blur takes.

    ArrayList elements = new ArrayList();
    for (int row = 0; row < video.height; row++) {
      for (int col = 0; col < video.width; col++) {
        int offset = row * width + col;
        int thisPixel = video.pixels[offset];
        float b = brightness(thisPixel);
        if (b < threshold) {
          video.pixels[offset] = 0;
          //be pessimistic
          boolean foundAHome = false;
          //look throught the existing
          for (int i = 0; i < elements.size(); i++) {
            Liner existingElement =  (Liner) elements.get(i);
            if (existingElement.closeTo(col,row,reach)){
              existingElement.addPoint(col,row);
              foundAHome = true; //no need to make a new one
              break; //no need to look through the rest of the boxes
            }
          }
          //if this does not belong to one of the existing boxes make a new one at this place
          if (foundAHome == false) elements.add(new Liner(col, row));
        }
      }
    }

    //consolidate(boxes,0,0);
    if (elements.size() > 1){
         Liner thisElement =  (Liner) elements.get(0);
            Liner thisElement2 =  (Liner) elements.get(1);
            if (thisElement.getSize() > thisElement2.getSize()){
              thisElement.setName("Player1");
              thisElement2.setName("Player2");
            }else{
              thisElement2.setName("Player1");
              thisElement.setName("Player2");
            }

    }

    image(video, 0, 0);
    fill(0, 0, 0, 0);
    stroke(255, 0, 0);
    for (int i = 0; i < elements.size(); i++) {
      Liner thisElement =  (Liner) elements.get(i);
      line(thisElement.leftMost.x, thisElement.leftMost.y, thisElement.rightMost.x, thisElement.rightMost.y);
      fill(0,255,255);
      text(thisElement.getName() , thisElement.leftMost.x, thisElement.leftMost.y);
    }


  }

}

void keyPressed() {

  if (key == 's') {
    video.settings();
  } 
  else if (key == '-') {
    threshold--;
    println("Threshold " + threshold);
  } 
  else if (key == '=') {
    threshold++;
    println("Threshold " + threshold);
  } 
  else if (key == '_') {
    reach--;
    println("reach " + reach);
  } 
  else if (key == '+') {
    reach++;
    println("reach " + reach);
  } 
  else if (key == 't') {

    println("Elapsedtime " + elapsedTime);

  }

} public class Liner{

  Point leftMost;
  Point rightMost;
  String name = "None Yet";

  public Liner(int _x, int _y){
    leftMost = new Point(_x,_y);
    rightMost = new Point(_x,_y);

  }

  public void setName(String _name){
    name = _name;
  }

  public String getName(){
    return name;
  }
  public boolean closeTo(int _x, int _y, int _reach){
    return (_x < rightMost.x + reach  && _x > leftMost.x - reach && 
      (( _y > leftMost.y-reach && _y< rightMost.y+reach) ||( _y > rightMost.y-reach && _y< leftMost.y+reach)));
  }

  public float getSize(){
    return dist(leftMost.x, leftMost.y, rightMost.x, rightMost.y);
  }

  public float getAngle(){


    return atan2(leftMost.y -rightMost.y, leftMost.x -rightMost.x) ;
  }
  public void addPoint(int _x, int _y){
    if (_x < leftMost.x){
      leftMost.x = _x;
      leftMost.y = _y;
    }
    if (_x > rightMost.x){
      rightMost.x = _x;
      rightMost.y = _y;
    }

  }

}

Idenity using Coninutity:

It might be possible to maintain identity if you make the assumption that an item in one frame will be closest to the same item in the next frame. This is a pretty reasonable assumption and even better if you compare not to where the thing was but where it was moving to. This requires that you keep two lists, one for the new finds and a second, more eternal list of what you found before or what you expected to find.

     	                int[] distances = new int[expectedRects.size()*newRects.size()];
		  	int[][] pairings = new int[expectedRects.size()*newRects.size()][3]; //dist,expectedIndex,newIndex
			for (int i = 0; i < expectedRects.size(); i++) {
				Rectangle expectedRect = (Rectangle) expectedRects.get(i);
				float expectedX = expectedRect.x + expectedRect.width / 2;
				float expectedY = expectedRect.y + expectedRect.height /2;

				for (int j = 0; j < newRects.size(); j++) {
					int placeInArray = i*newRects.size() + j;
					Rectangle newRect = (Rectangle) newRects.get(j);
				       int dist = (int) (dist(expectedX, expectedY,(float) (newRect.x + newRect.width / 2), (float)(newRect.y + newRect.height / 2) ));
					distances[placeInArray] = dist;
					pairings[placeInArray] = new int[] {dist,i,j};

				}
			}

			// sort the distances
			Arrays.sort(distances);
			//keep track if these things have been spoken for already 
			boolean[] checkListForExpected = new boolean[expectedRects.size()];
			boolean[] checkListForNew = new boolean[newRects.size()];
			//keep a variable that knows if you you have found enough
			int found = 0;
			//go down the distances in order and find the pairing with the least distance
			for (int i = 0; i < distances.length; i++) {
				int closeOne = distances[i];
				for (int j = 0; j < pairings.length; j++){
					int[] thisPairing = pairings[j];
					int dist = thisPairing[0];
					int expectedIndex = thisPairing[1];
					int newIndex = thisPairing[2];
					//if this pairing has the distance you are after and it's parts are not spoken for
					if (dist == closeOne &&  checkListForNew[newIndex] == false && checkListForExpected[expectedIndex] == false) { 
                                              // not spoken for
						Rectangle expectedRect = (Rectangle) expectedRects.get(expectedIndex);
						Rectangle newRect = (Rectangle) newRects.get(newIndex);
						checkListForExpected[expectedIndex] = true; // mark this found as used
						checkListForNew[newIndex] = true;
						expectedRect.x = newRect.x;
						expectedRect.y = newRect.y;
						expectedRect.width = newRect.width;
						expectedRect.height = newRect.height;
						found++;

					}
					if (found >= newRects.size()) break;
				}
				if (found >= newRects.size()) break;
			}

Libraries

Processing 2.04a + NyARToolkit 1.1.7

import processing.video.*;

// Processing 2.04a + NyARToolkit 1.1.7 //pared down from Amnon Owed http://www.creativeapplications.net/processing/augmented-reality-with-processing-tutorial-processing/

import java.io.*;

import processing.opengl.*;

// for OPENGL rendering

import jp.nyatla.nyar4psg.*;

// the NyARToolkit Processing library

//DON'T FORGET TO PUT THE "PATT.XXX" FILES AND "camera_para.dat" IN A DATA FOLDER IN YOUR SKETCH FOLDER

Capture cam; MultiMarker nya;

void setup() {

  size(640, 480, OPENGL); 
  cam = new Capture(this, 640, 480); 
  cam.start(); 
  frameRate(30);

  // create a new MultiMarker at a specific resolution (arWidth x arHeight), with the default camera calibration and coordinate system


  nya = new MultiMarker(this, width, height, "camera_para.dat", NyAR4PsgConfig.CONFIG_DEFAULT);


  // set the delay after which a lost marker is no longer displayed. by default set to something higher, but here manually set to immediate.


  //nya.setLostDelay(1);

  nya.addARMarker("patt.hiro", 80);  //your have to print out the cooresponding pdf file and put the .patt files in data folder
  nya.addARMarker("patt.kanji", 80);

}

void draw() {

  background(0); // a background call is needed for correct display of the marker results
  cam.read();
  image(cam, 0, 0, width, height); // display the image at the width and height of the sketch window
  nya.detect(cam); // detect markers in the input image at the correct resolution (incorrect resolution will give assertion error)
  if (nya.isExistMarker(0)) {
    setMatrix(nya.getMarkerMatrix(0));  //use this marker to translate and rotate the processing drawing
    translate(0, 0, 25); //offset half the size of the cube.
    fill(255, 0, 0);
    box(50);
  }
  perspective();
  if (nya.isExistMarker(1)) {
    setMatrix(nya.getMarkerMatrix(1));
    translate(0, 0, 25); 
    fill(0, 255, 0);
    box(50);
  }

}

OLD TOOLKIT WITH PROCESSING 1.5

Augmented Reality in Processing
- This is not a proper processing library so you have to put "NYAR2.jar" and "NYARToolkit.jar" (dig through the download) in "code" folder (make it at the same level in your sketch folder as you would data the data folder. Also put "camera_para" and "patt.hiro" in the data folder.
- You also have to prinout the hiro.pdf to be you physical marker
- You can get the pattern files and the associated gifs for print out here
- Note you can get other patterns (not they usually are xxx.patt) and their associated xxx.pdf. It is even possible to use arbitrary patterns that you make as the markers.
- Also see http://www.bryanchung.net/?p=227 which is a proper processing library

import processing.video.*; import processing.core.*; import jp.nyatla.nyar4psg.*; import processing.opengl.PGraphicsOpenGL; import javax.media.opengl.GL;

NyARBoard tag ; NyARBoard tag2 ;

PImage myImage; PImage myImage2;

boolean debug;

Capture video;

void setup(){

  try {//hack due to bug in OPENGL in processing
    quicktime.QTSession.open();
    } catch (quicktime.QTException qte) {
    qte.printStackTrace();
    }
  size(320,240, OPENGL);
  video = new Capture(this, width , height );
  //  video.settings();
   tag2 = new NyARBoard(this, video.width, video.height, "camera_para.dat", "insert your patt file name here", 120);
     tag2.gsThreshold = 120;// (0<n<255) default=110
  tag2.cfThreshold = 0.8;// (0.0<n<1.0) default=0.4
  tag = new NyARBoard(this, video.width, video.height, "camera_para.dat", "patt.hiro", 120);
  tag.gsThreshold = 120;// (0<n<255) default=110
  tag.cfThreshold = 0.8;// (0.0<n<1.0) default=0.4
 myImage = loadImage("insert your image location here");
   myImage2 = loadImage("insert your image location here");;

}

void draw(){

  if( video.available()){
    video.read();
    if (debug) image(video,0,0);
    else background(0);
    if (tag.detect(video)) {

	//int[][] p = tag.pos2d;
       // println(" found tag x " + p[0][0] + " y" + p[0][1] + " x " + p[1][0] + " y" + p[1][1] + " x " + p[2][0] + " y" + p[2][1] + " x " + p[3][0] + " y" + p[3][1]);

      PGraphicsOpenGL pgl = (PGraphicsOpenGL) g;
      GL gl = pgl.gl;

      pgl.beginGL();
      gl.glDisable(GL.GL_DEPTH_TEST);
      gl.glEnable(GL.GL_BLEND);
      // gl.glBlendFunc(GL.GL_SRC_ALPHA,GL.GL_ONE);
      tag.beginTransform(pgl);
      image(  myImage, -  myImage.width / 2 , -  myImage.height/2 ,   myImage.width ,   myImage.height);
      tag.endTransform();
    }
    if (tag2.detect(video)) {

	//int[][] p = tag.pos2d;
       // println(" found tag x " + p[0][0] + " y" + p[0][1] + " x " + p[1][0] + " y" + p[1][1] + " x " + p[2][0] + " y" + p[2][1] + " x " + p[3][0] + " y" + p[3][1]);

      PGraphicsOpenGL pgl = (PGraphicsOpenGL) g;
      GL gl = pgl.gl;

      pgl.beginGL();
      gl.glDisable(GL.GL_DEPTH_TEST);
      gl.glEnable(GL.GL_BLEND);
      // gl.glBlendFunc(GL.GL_SRC_ALPHA,GL.GL_ONE);
      tag2.beginTransform(pgl);
      image(  myImage2, -  myImage2.width / 2 , -  myImage2.height/2 ,   myImage2.width ,   myImage2.height);
      tag2.endTransform();
    }
  }

}

void keyPressed() {

  //for adjusting things on the fly
  if (key == 's') {
    video.settings();
  } 
  else if (key == '-') {
    tag.gsThreshold --;
    println("tag.gsThreshold  " + tag.gsThreshold );
  } 
  else if (key == '=') {
    tag.gsThreshold ++;
    println("tag.gsThreshold  " + tag.gsThreshold );
  }
   else if (key == '_') {
    tag.cfThreshold = tag.cfThreshold -.01;
    println("tag.cfThreshold  " + tag.cfThreshold );
  } 
  else if (key == '+') {
     tag.cfThreshold = tag.cfThreshold +.01;
    println("tag.cfThreshold  " + tag.cfThreshold );
  }
  else if (key == 'd') {
     debug = !debug;
    println("debug  " + debug);
  }

}

Augmented Reality in Flash

Open CV Does Blobs

import hypermedia.video.*; OpenCV opencv;

int w = 320;

int h = 240;

int threshold = 80;

void setup() {

  size( w, h );

  opencv = new OpenCV( this );
  opencv.capture(w,h);

}

void draw() {

  background(0);
  opencv.read();
  //opencv.flip( OpenCV.FLIP_HORIZONTAL );
   opencv.absDiff();
  image( opencv.image(OpenCV.GRAY), 0, 0); // absolute difference image
  opencv.threshold(threshold);
  opencv.invert();

  // working with blobs
  Blob[] blobs = opencv.blobs( 1, 200, 4, true );
  noFill();
  fill(255,0,0);
  for( int i=0; i<blobs.length; i++ ) {
    Rectangle bounding_rect	= blobs[i].rectangle;
    println(blobs[i].centroid);
    rect( bounding_rect.x, bounding_rect.y, bounding_rect.width, bounding_rect.height );
  }

}

void keyPressed() {

  if ( key==' ' ) {
    opencv.remember();
  }
  else if (key == '-') {
    threshold --;
    println("Threshold" + threshold);
  }
  else if (key == '=') {
    threshold ++;
    println("Threshold" + threshold);
  }

}

public void stop() {

  opencv.stop();
  super.stop();

}

Open CV Good for Face Tracking

import hypermedia.video.*; import java.awt.Rectangle;

OpenCV opencv;

// contrast/brightness values

int contrast_value = 0;

int brightness_value = 0;

Rectangle bestRect = new Rectangle(0,0,0,0);

void setup() {

    size( 320, 240 );

    opencv = new OpenCV( this );
    opencv.capture( width, height );                   // open video stream
    opencv.cascade( OpenCV.CASCADE_FRONTALFACE_ALT );  // load detection description, here-> front face detection : "haarcascade_frontalface_alt.xml"


    // print usage
    println( "Drag mouse on X-axis inside this sketch window to change contrast" );
    println( "Drag mouse on Y-axis inside this sketch window to change brightness" );

}

public void stop() {

    opencv.stop();
    super.stop();

}

void draw() {

    // grab a new frame
    // and convert to gray
    opencv.read();
    //opencv.convert( GRAY );
    opencv.contrast( contrast_value );
    opencv.brightness( brightness_value );

    // proceed detection
    Rectangle[] faces = opencv.detect( 1.2, 2, OpenCV.HAAR_DO_CANNY_PRUNING, 40, 40 );

    // display the image
    image( opencv.image(), 0, 0 );

  //  if (faces.length== 2){ //switch faces
  //    PImage firstFace = bigImage.get(faces[0].x,faces[0].y,faces[0].width,faces[0].height);
  //    PImage secondFace = bigImage.get(faces[1].x,faces[1].y,faces[1].width,faces[1].height);
  //    image(firstFace,faces[1].x,faces[1].y,faces[1].width,faces[1].height);
  //    image(secondFace,faces[0].x,faces[0].y,faces[0].width,faces[0].height);
 //   }



    // draw face area(s)
    noFill();
    stroke(255,0,0);
    for( int i=0; i<faces.length; i++ ) {
      if (i == 0) bestRect = new Rectangle(faces[i]);
        rect( faces[i].x, faces[i].y, faces[i].width, faces[i].height ); 
    }
     stroke(0,255,0);
    rect(bestRect.x,bestRect.y,bestRect.width,bestRect.height);

}

/**

 * Changes contrast/brigthness values
 */

void mouseDragged() {

    contrast_value   = (int) map( mouseX, 0, width, -128, 128 );
    brightness_value = (int) map( mouseY, 0, width, -128, 128 );

}

Skin Rects

import processing.video.*;

public class Skin extends PApplet {

  float skinRedLower = .35f;

  float skinRedUpper = .55f;

  float skinGreenLower = .26f;

  float skinGreenUpper = .35f;

   int reach = 3;
  Capture video;

  int w = 320;

  int h = 240;

  boolean debug = true;

  long elapsedTime = 0;

  void setup(){
    size(w,h);  
    video = new Capture(this,w,h,30);	
  }


  public void draw(){  //called everytime there is a new frame

    long startTime = System.currentTimeMillis();

    if (video.available()){
      video.read(); //get the incoming frame as a picture
      ArrayList rects = findRectangles();

      //println(elapsedTime);  
      consolidate(rects,0,0);
      cleanUp(rects,100);
           if (debug) image(video,0,0);
      fill(0, 0, 0, 0);
      stroke(255, 0, 0);
      for (int i = 0; i < rects .size(); i++) {
        Rectangle thisBox =  (Rectangle) rects .get(i);
        rect(thisBox.x, thisBox.y, thisBox.width, thisBox.height);
      }

      elapsedTime = System.currentTimeMillis() - startTime;

    }
  }

  boolean test(int _thisPixel){
    float r = red(_thisPixel);
    float g = green(_thisPixel);
    float total = r + g + blue(_thisPixel);

    float percentRed = r/total;
    float percentGreen = g/total;
    return (percentRed < skinRedUpper && percentRed > skinRedLower  && percentGreen < skinGreenUpper && percentGreen > skinGreenLower);
  }



  void keyPressed() {

    //for adjusting things on the fly
    if (key == 's') {
      video.settings();
    } 
    else if (key == '-') {
      skinRedUpper = skinRedUpper - .01f;

      skinRedLower = skinRedLower - .01f;

    } 
    else if (key == '=') {
      skinRedUpper = skinRedUpper + .01f;

      skinRedLower = skinRedLower + .01f;
    }
    else if (key == '_') {
      skinGreenUpper = skinGreenUpper - .01f;

      skinGreenLower = skinGreenLower - .01f;
    } 
    else if (key == '+') {
      skinGreenUpper = skinGreenUpper + .01f;

      skinGreenLower = skinGreenLower + .01f;

    }
    else if (key == 'r') {
    reach--;
    println("reach " + reach);
  } 
  else if (key == 'R') {
    reach++;
    println("reach " + reach);
  } 
  else if (key == 't') {

    println("Elapsedtime " + elapsedTime);

  }
    else if (key == 'd') {
      debug = !debug;
      println("debug  " + debug);
    }
    println("RU:" + skinRedUpper + " RL:" + skinRedLower +"GU:" + skinGreenUpper + " GL:" + skinGreenLower );


  }
  void cleanUp(ArrayList _rects, int _sizeThreshold){
   for (int j = _rects.size() - 1; j > -1; j--) { 

		Rectangle newRect = (Rectangle) _rects.get(j);
		if (newRect.getHeight()*newRect.getWidth() < _sizeThreshold) _rects.remove(j); //if the area to small, loose it

    }
  } 


      public void consolidate(ArrayList _shapes, int _consolidateReachX, int _consolidateReachY) { 

		//check every combination of shapes for overlap 
		//make the repeat loop backwards so you delete off the bottom of the stack
		for (int i = _shapes.size() - 1; i > -1; i--) {
			//only check the ones up 
			Rectangle shape1 = (Rectangle) _shapes.get(i);
			Rectangle inflatedShape1 = new Rectangle(shape1); //copy the existing box
			inflatedShape1.grow(_consolidateReachX, _consolidateReachY); //widen it's reach

			for (int j = i - 1; j > -1; j--) {
				Rectangle shape2 = (Rectangle) _shapes.get(j);
				if (inflatedShape1.intersects(shape2) ){
					shape1.add(shape2);
					//System.out.println("Remove" + j);
					_shapes.remove(j);
					break;
				}
			}
		}

    } 
  ArrayList findRectangles() {
    ArrayList boxes = new ArrayList();

    for (int row = 0; row < video.height; row++) {
      for (int col = 0; col < video.width; col++) {
        int offset = row * width + col;
        int thisPixel = video.pixels[offset];
        if (test(thisPixel)) {
          video.pixels[offset] = 0xffff00;
          //be pessimistic
          boolean foundAHome = false;
          //look throught the existing
          for (int i = 0; i < boxes.size(); i++) {
            Rectangle existingBox =  (Rectangle) boxes.get(i);

            //is this spot in an  existing box
            Rectangle inflatedBox = new Rectangle(existingBox); //copy the existing box
            inflatedBox.grow(reach, reach); //widen it's reach
            if (inflatedBox.contains(col, row)) {
              existingBox.add(col,row);
              foundAHome = true; //no need to make a new one
              break; //no need to look through the rest of the boxes
            }
          }
          //if this does not belong to one of the existing boxes make a new one at this place
          if (foundAHome == false) boxes.add(new Rectangle(col, row,0,0));
        }
      }
    }
    return boxes;
  }

}

Eyetracking

You can copy and paste but you have to rename your processing project EyeTrack. Also you have to add an image called "myPicture.jpg" to the project. This will be the picture they look at. You should look at the keypresses to see how to adjust things, most notably "d" for toggling debug mode and "p" an "P" for ajusting the threshold for the pupil and "g" and "G" for the threshold for the glint.

/*You can copy and paste but you have to rename your processing project EyeTrack.

 Also you have to add an image called "myPicture.jpg" to the project. This will be 
 the picture they look at. You should look at the keypresses to see how to adjust 
 things, most notably "d" for toggling debug mode and "p" an "P" for ajusting the 
 threshold for the pupil and "g" and "G" for the threshold for the glint.
 */

import java.awt.Point; import java.awt.Rectangle; import java.util.ArrayList; import processing.core.PApplet; import processing.core.PImage; import processing.video.Capture; public class Eyetrack extends PApplet {

  Capture video;// regular processing libary

  int pupilThreshold = 100;

  int glintThreshold = 240;

  int edge = 50;

  int reach = 5;

  int minPupilArea = 75;

  long elapsedTime;

  float calibrationSlope;

  int calibrationIntercept;

  Rectangle pupil;

  Rectangle glint;

  CoordinatesTranslator calibrator;

  boolean debug = true;

  PImage testImage;

  PGraphics debugScreen;

  boolean leaveTrails;

  static public void main(String _args[]) {
    PApplet.main(new String[] { 
      "Eyetrack"
    }
    );
  }

  public void setup() {
    size(640, 480);
    println(Capture.list());
    video = new Capture(this, width, height);
    debugScreen= createGraphics(width, height, P3D);
    video.settings();
    testImage = loadImage("yourimage");
    size(testImage.width, testImage.height);
    calibrator = new CoordinatesTranslator();

    image(testImage, 0, 0);
  }

  public void draw() {
    if (video.available()) {

      video.read();
      long startTime = millis();
      elapsedTime = millis() - startTime;
      if (debug) {
        //background(0);
        debugScreen.beginDraw();
        debugScreen.image(video, 0, 0);
      }
      else {
        if (leaveTrails == false) image(testImage, 0, 0);
      }
      pupil = findPupil();
      glint = null;
      if (pupil != null) glint = findGlint(pupil);
      if (glint != null && pupil != null) { // if we got both
        // find out where the pupil is relative to the glint
        int rawX = pupil.x+ pupil.width/2 - glint.x + glint.width/2;
        int rawY =glint.y + glint.height/2- pupil.y + pupil.height/2  ;
        if (calibrator.isCurrentlyCalibrating()) { // check if we are in calibration mode
          Point placeToLook = calibrator.getScenePoint(); // get where they are supposed to look at
         stroke(255, 0, 0);
          ellipse(placeToLook.x - 4, placeToLook.y - 4, 8, 8);// draw it so they look
          calibrator.doCalibrationRoutine(true, rawX, rawY);
          println("cablibrating");
        } 
        else {
          int[] adjustedPoint = calibrator.translate(rawX, rawY); //use the calibrator to find out where the x,y from camera corolate to on image
          //println("Adjusted x" + adjustedPoint[0]  + " adjusted y" + adjustedPoint[1]);
          ellipse(adjustedPoint[0] - 2, adjustedPoint[1] - 2, 4, 4);
        }
      }
      if (debug) {
        debugScreen.endDraw();
        debugScreen.updatePixels();
        image(debugScreen, 0, 0);
      }
    }
  }

  public Rectangle findPupil() {
    // /FIND THE PUPIL
    ArrayList boxes = new ArrayList();
    for (int row = edge; row < video.height - edge * 2; row++) {
      for (int col = edge; col < video.width - edge * 2; col++) {
        int offset = row * video.width + col;
        int thisPixel =  video.pixels[offset];
        //look for dark things
        if (brightness(thisPixel) < pupilThreshold) {
          debugScreen.pixels[offset] = 0;
          // be pessimistic
          boolean foundAHome = false;
          // look throught the existing
          for (int i = 0; i < boxes.size(); i++) {
            Rectangle existingBox = (Rectangle) boxes.get(i);
            // is this spot in an existing box
            Rectangle inflatedBox = new Rectangle(existingBox); // copy the existing box
            inflatedBox.grow(reach, reach); // widen it's reach
            if (inflatedBox.contains(col, row)) {
              existingBox.add(col, row);
              foundAHome = true; // no need to make a new one
              break; // no need to look through the rest of the boxes
            }
          }
          // if this does not belong to one of the existing boxes make a new one at this place
          if (foundAHome == false) boxes.add(new Rectangle(col, row, 0, 0));
        }
      }
    }

    consolidate(boxes, 0, 0);

    // OF EVERYTHING YOU FIND TAKE THE ONE CLOSEST TO THE CENTER
    Rectangle pupil = findClosestMostBigOne(boxes, video.width / 2, video.height / 2, minPupilArea);
    if (debug) {
      // show the the edges of the search
      debugScreen.fill(0, 0, 0, 0);
      debugScreen.stroke(0, 255, 255);
      debugScreen.rect(edge, edge, video.width - 2 * edge, video.height - 2 * edge);
      // show all the pupil candidates
      debugScreen.stroke(255, 255, 0);
      for (int i = 0; i < boxes.size(); i++) {
        Rectangle thisBox = (Rectangle) boxes.get(i);
        debugScreen.rect(thisBox.x, thisBox.y, thisBox.width, thisBox.height);
      }
      // show the winning pupil candidate in red
      if (pupil != null) {
        stroke(255, 0, 0);
        debugScreen.rect(pupil.x, pupil.y, pupil.width, pupil.height);
      }
    }
    return pupil;
  }

  public Rectangle findGlint(Rectangle _pupil) {
    // ADJUST THE BOUNDS OF YOUR SEARCH TO BE AROUND AND UNER THE PUPIL
    int glintEdgeL = Math.max(edge, _pupil.x - _pupil.width * 2);
    int glintEdgeR = Math.min(video.width - edge, _pupil.x +  _pupil.width * 2);
    int glintTop = _pupil.y;
    int glintBottom = Math.min(video.height - edge, _pupil.y +  _pupil.height * 2);

    ArrayList glintsCandidates = new ArrayList();
    for (int row = glintTop; row < glintBottom; row++) {
      for (int col = glintEdgeL; col < glintEdgeR; col++) {
        int offset = row * video.width + col;
        int thisPixel = video.pixels[offset];
        //look for bright things
        if (brightness(thisPixel) > glintThreshold) {
          if (debug) debugScreen.pixels[offset] = 0xffff00;
          // be pessimistic
          boolean foundAHome = false;
          // look throught the existing
          for (int i = 0; i < glintsCandidates.size(); i++) {
            Rectangle existingBox = (Rectangle) glintsCandidates.get(i);
            // is this spot in an existing box
            Rectangle inflatedBox = new Rectangle(existingBox); // copy the existing box
            inflatedBox.grow(reach, reach); // widen it's reach
            if (inflatedBox.contains(col, row)) {
              existingBox.add(col, row);
              foundAHome = true; // no need to make a new one
              break; // no need to look through the rest of the boxes
            }
          }
          // if this does not belong to one of the existing boxes make a new one at this place
          if (foundAHome == false) glintsCandidates.add(new Rectangle(col, row, 0, 0));
        }
      }
    }
    // FIND THE GLINT THAT IS CLOSEST TO THE PUPIL
    Rectangle glint = findClosestMostBigOne(glintsCandidates, _pupil.x + _pupil.width, _pupil.y + _pupil.height / 2, 0);
    debugScreen.stroke(0, 0, 255);

    if (debug) {// show all the candidate
      // show the edges of the search for the glint
      debugScreen.stroke(0, 255, 0);
      debugScreen.rect(glintEdgeL, glintTop, glintEdgeR - glintEdgeL, glintBottom - glintTop);
      for (int i = 0; i < glintsCandidates.size(); i++) {
        Rectangle thisBox = (Rectangle) glintsCandidates.get(i);
        debugScreen.rect(thisBox.x, thisBox.y, thisBox.width, thisBox.height);
      } // show the winner in red
      if (glint != null) {
        debugScreen.stroke(255, 0, 0);
        debugScreen.rect(glint.x, glint.y, glint.width, glint.height);
      }
    }

    return glint;
  }

  public void consolidate(ArrayList _shapes, int _consolidateReachX, int _consolidateReachY) {
    // check every combination of shapes for overlap
    // make the repeat loop backwards so you delete off the bottom of the stack
    for (int i = _shapes.size() - 1; i > -1; i--) {
      // only check the ones up
      Rectangle shape1 = (Rectangle) _shapes.get(i);
      Rectangle inflatedShape1 = new Rectangle(shape1); // copy the existing box
      inflatedShape1.grow(_consolidateReachX, _consolidateReachY); // widen it's reach

      for (int j = i - 1; j > -1; j--) {
        Rectangle shape2 = (Rectangle) _shapes.get(j);
        if (inflatedShape1.intersects(shape2)) {
          shape1.add(shape2);
          // System.out.println("Remove" + j);
          _shapes.remove(j);
          break;
        }
      }
    }
  }

  public Rectangle findClosestMostBigOne(ArrayList _allRects, int _x, int _y, int _minArea) {
    if (_allRects.size() == 0) return null;
    int winner = 0;
    float closest = 1000;

    for (int i = 0; i < _allRects.size(); i++) {
      Rectangle thisRect = (Rectangle)  _allRects.get(i);
      if (thisRect.width * thisRect.height < _minArea) continue;
      float thisDist = dist(_x, _y, thisRect.x + thisRect.width / 2, thisRect.y + thisRect.height / 2);
      if (thisDist < closest) {
        closest = thisDist;
        winner = i;
      }
    }
    return (Rectangle) _allRects.get(winner);
  }

  public void keyPressed() {
    if (key == 's') {
      video.settings();
    } 
    else if (key == 'c') {
      calibrator.startCalibrationRoutine(3, width, 3, height, 3);
      debug = false;
    } 
    else if (key == 'l') {
      leaveTrails = !leaveTrails;
      if (leaveTrails == false) image(testImage, 0, 0);
    } 
    else if (key == 'd') {

      debug = !debug;
      if (debug == false) image(testImage, 0, 0);
      println("debug " + debug);
    } 
    else if (key == 'p') {
      pupilThreshold--;
      println("PuplilThreshold " + pupilThreshold);
    } 
    else if (key == 'P') {
      pupilThreshold++;
      println("PuplilThreshold " + pupilThreshold);
    } 
    else if (key == 'r') {
      reach--;
      println("reach " + reach);
    } 
    else if (key == 'R') {
      reach++;
      println("reach " + reach);
    } 

    else if (key == 'A') {
      minPupilArea++;
      println("minArea " + minPupilArea);
    } 
    else if (key == 'a') {
      minPupilArea--;
      println("minArea " + minPupilArea);
    } 
    else if (key == 'G') {
      glintThreshold++;
      println("glintThreshold " + glintThreshold);
    } 
    else if (key == 'g') {
      glintThreshold--;
      println("glintThreshold " + glintThreshold);
    } 
    else if (key == 'E') {
      edge++;
      println("edge " + edge);
    } 
    else if (key == 'e') {
      edge --; // = Math.max(0, edge--);
      println("edge " + edge);
    } 
    else if (key == 't') {

      println("Elapsedtime " + elapsedTime);
    }
  }
  public class CoordinatesTranslator {
    /**

     	 * 

     	 */



    boolean calibrationDotBlinkState;



    float xSlope = 1.0f;



    float ySlope = 1.0f;



    float xIntercept = 0f;



    float yIntercept = 0f;



    public ArrayList calibrationPoints = new ArrayList();



    int sceneX = 0;



    int sceneY = 0;



    ArrayList recordedXPoints = new ArrayList();



    ArrayList recordedYPoints = new ArrayList();



    int calibrationOps = 0;



    boolean calibrate = false;



    int numberOfSamples;



    long milliSecondsPerPoint = 10000;



    Point[] allScenePoints;



    long lastPositionStart;



    int whichPoint = 0;



    public boolean isCurrentlyCalibrating() {

      return calibrate;
    }



    public int[] translate(int _x, int _y) {

      int[] out = { 
        (int) (_x * xSlope + xIntercept), (int) (_y * ySlope + yIntercept)
      };

      return out;
    }



    public void startCalibrationRoutine(int _numberAcross, int _pixelsAcross, int _numberDown, int _pixelsDown, int _secondsPerPoint) {

      clearCalibrations();

      milliSecondsPerPoint = _secondsPerPoint * 1000;

      int xInc = _pixelsAcross / (_numberAcross + 1);

      int yInc = _pixelsDown / (_numberDown + 1);

      allScenePoints = new Point[_numberDown * _numberAcross];

      for (int r = 0; r < _numberDown; r++) {

        for (int c = 0; c < _numberAcross; c++) {

          allScenePoints[r * _numberAcross + c] = new Point(c * xInc + xInc, r * yInc + yInc);

          System.out.println("cal point" + allScenePoints[r * _numberAcross + c]);
        }
      }

      // lastPositionStart = 0; //long time ago

      whichPoint = 0;

      lastPositionStart = System.currentTimeMillis();

      Point currPoint = allScenePoints[whichPoint];

      startCalibratingNewPoint(currPoint.x, currPoint.y, -1);



      calibrate = true;
    }



    public int[] doCalibrationRoutine(boolean _valid, int _x, int _y) {

      if (calibrate) {

        //check on the progression of the target dots



        long now = System.currentTimeMillis();

        if ((now - lastPositionStart) > milliSecondsPerPoint) {

          System.out.println("New Calibration" + whichPoint);

          if (whichPoint != -1) {

            finishCalibrationForNewPoint();
          }

          whichPoint++;



          if (whichPoint >= allScenePoints.length) {

            calibrate = false;

            return null;
          }



          lastPositionStart = now;

          Point currPoint = allScenePoints[whichPoint];

          startCalibratingNewPoint(currPoint.x, currPoint.y, -1);
        }
      }

      if (_valid) {

        calibrationDotBlinkState = !calibrationDotBlinkState;

        addToCalibrationForThisPoint(_x, _y);
      }

      int[] r = { 
        sceneX, sceneY
      };

      return r;
    }



    public Point getScenePoint() {

      if (calibrate) {

        return new Point(sceneX, sceneY);
      } 
      else {

        return null;
      }
    }



    public void startCalibratingNewPoint(int _sceneX, int _sceneY, int _numberOfSamples) {

      numberOfSamples = _numberOfSamples;

      calibrationOps = 0;

      sceneX = _sceneX;

      sceneY = _sceneY;

      System.out.println("startNewCalibrationPoint");

      recordedXPoints = new ArrayList();

      recordedYPoints = new ArrayList();

      calibrate = true;
    }



    private void addToCalibrationForThisPoint(int x, int y) {

      calibrationOps++;

      recordedXPoints.add(new Integer(x));

      recordedYPoints.add(new Integer(y));

      if ((numberOfSamples != -1) && (calibrationOps >= numberOfSamples))

        finishCalibrationForNewPoint();
    }





    public void finishCalibrationForNewPoint() {

      System.out.println("finishNewCalibrationPoint");

      calibrate = false;

      // for(int a = 0; a < recordedXPoints.

      if (recordedXPoints.size() > calibrationOps / 2) {

        Object[] rxp = recordedXPoints.toArray();

        Arrays.sort(rxp);

        Object[] ryp = recordedYPoints.toArray();

        Arrays.sort(ryp);

        int eyeX = ((Integer) (rxp[rxp.length / 2])).intValue();

        int eyeY = ((Integer) (ryp[ryp.length / 2])).intValue();

        System.out.println("calibs " + recordedXPoints.toString() + " " + eyeX + " scene " + sceneX);

        System.out.println("calibs " + recordedYPoints.toString() + " " + eyeY + " scene " + sceneY);

        int[] newPoints = { 
          eyeX, eyeY, sceneX, sceneY
        };

        calibrationPoints.add(newPoints);

        linearRegressionCalibration();
      } 
      else {

        System.out.println("calibraion aborted, no calibration points");
      }
    }



    public void clearCalibrations() {

      calibrate = false;

      xSlope = 1.0f;

      ySlope = 1.0f;

      xIntercept = 0f;

      yIntercept = 0f;

      calibrationPoints = new ArrayList();

      System.out.println("clear calibrations");
    }



    public void linearRegressionCalibration() {

      System.out.println("Linear Regression");

      float Exy = 0.0f; // sum of products = x1y1 + x2y2 + . . . + xnyn

      float Ex = 0.0f; // sum of x-values = x1 + x2 + . . . + xn

      float Ey = 0.0f; // sum of y-values = y1 + y2 + . . . + yn

      float Ex2 = 0.0f; // sum of squares of x-values = x12 + x22+ . . . +

      // xn2

      for (int i = 0; i < calibrationPoints.size(); i++) {

        int[] target = (int[]) calibrationPoints.get(i);

        System.out.println("samples " + i + " " + target[0] + " " + target[1] + " " + target[2] + " " + target[3]);

        Exy = Exy + target[0] * target[2];

        Ex = Ex + target[0];

        Ey = Ey + target[2];

        Ex2 = Ex2 + target[0] * target[0];
      }

      // System.out.println("samples " + " " + Exy + " " + Ex + " " + Ey + " " + Ex2);

      float n = (float) calibrationPoints.size();

      System.out.println("n" + n);

      float denom = (float) (n * (Ex2) - (Ex * Ex));

      // if (denom == 0.0)

      // denom = 1.0f;

      xSlope = (n * Exy - Ex * Ey) / (n * Ex2 - Ex * Ex);

      xIntercept = (Ey - xSlope * Ex) / n;

      Exy = 0;

      Ex = 0;

      Ey = 0;

      Ex2 = 0;

      for (int i = 0; i < calibrationPoints.size(); i++) {

        int[] target = (int[]) calibrationPoints.get(i);

        Exy = Exy + target[1] * target[3];

        Ex = Ex + target[1];

        Ey = Ey + target[3];

        Ex2 = Ex2 + target[1] * target[1];
      }

      n = calibrationPoints.size();

      ySlope = (n * Exy - Ex * Ey) / (n * Ex2 - Ex * Ex);

      yIntercept = (Ey - ySlope * Ex) / n;

      // variablesToPrefs(prefs);

      System.out.println("xslope " + xSlope + " intercept" + xIntercept);

      System.out.println("yslope " + ySlope + " intercept" + yIntercept);

      // http://people.hofstra.edu/faculty/Stefan_Waner/RealWorld/calctopic1/regression.html
    }



    public boolean getCalibrationDotBlinkState() {

      return calibrationDotBlinkState;
    }



    public void setCalibrationDotBlinkState(boolean calibrationDotBlinkState) {

      this.calibrationDotBlinkState = calibrationDotBlinkState;
    }
  }

}

Network Camera

import processing.core.PApplet;

import java.awt.Dimension; import java.awt.Image; import java.awt.image.BufferedImage; import java.awt.image.PixelGrabber; import java.io.BufferedInputStream; import java.io.DataInputStream; import java.io.IOException; import java.io.InputStream; import java.lang.reflect.Method; import java.net.HttpURLConnection; import java.net.URL;

import processing.core.PApplet; import processing.core.PImage;

import com.sun.image.codec.jpeg.JPEGCodec; import com.sun.image.codec.jpeg.JPEGImageDecoder;

public class CaptureAxis extends PApplet {

	//.28 Phys Comp (406)
	//.29 inactive - can go in ex-res when needed
	//.188 ("Is Our Machines Learning")
	//.189 On your desk...

	//Capture video; 
	//this is what you use for ordinary video cameras
	//requires pointing at the video.jar library from processing

	CaptureAxisCamera video;//This acts the same the regular but connects to axis net cam

	static public void main(String _args[]) {
		PApplet.main(new String[] { "CaptureAxis" });
	}

	public void setup() {

		size(640,480);
		video = new CaptureAxisCamera(this, "128.122.151.200",width,height,false);

		//video = new Capture(this, 640,480);
		//this is what you use for ordinary video cameras
	}

	public void draw() {
		if (video.available()) {
			video.read();
			image(video, 0, 0);
		}
	}

/**

 * 
 * @author David E. Mireles, Ph.D. (adapted for processing by Dan O'Sullivan)
 */

public class CaptureAxisCamera extends PImage implements Runnable {

	public boolean useMJPGStream = false;

	public String ip = "";
	public String jpgURL = "http://128.122.151.200axis-cgi/jpg/image.cgi?resolution=352x240";

	public String mjpgURL  = "http://128.122.151.189/axis-cgi/mjpg/video.cgi?resolution=352x240";

	DataInputStream dis;

    Image image;

    BufferedImage bimage;

	public Dimension imageSize = null;

	public boolean connected = false;

	private boolean initCompleted = false;

	HttpURLConnection huc = null;

	PApplet parent;

	boolean crop;

	boolean available;

	Method captureEventMethod;

	/** Creates a new instance of AxisCamera */
	public CaptureAxisCamera(PApplet _parent, String _ip,int _w, int _h, boolean _useMJPGStream) {
		ip = _ip;
		parent = _parent;
		useMJPGStream = _useMJPGStream;
		jpgURL = "http://"+ ip + "/axis-cgi/jpg/image.cgi?resolution="+ String.valueOf(_w)+ "x" +String.valueOf(_h);

		//jpgURL = "";
		mjpgURL  = "http://"+ ip +"/axis-cgi/mjpg/video.cgi?resolution"+ String.valueOf(_w)+ "x" +String.valueOf(_h);

		// initialize my PImage self
		super.init(_w, _h, RGB);




		try {
			captureEventMethod = parent.getClass().getMethod("captureEvent", new Class[] { CaptureAxisCamera.class });
		} catch (Exception e) {
			// no such method, or an error.. which is fine, just ignore
		}



		Thread myThread = new Thread(this);
		myThread.start();

		parent.registerDispose(this);
	}

	/**
	 * True if a frame is ready to be read.
	 * 
	 * <PRE> // put this somewhere inside draw if (capture.available()) capture.read();
	 * 
	 * </PRE>
	 * 
	 * Alternatively, you can use captureEvent(Capture c) to notify you whenever available() is set to true. In which case, things might look like this:
	 * 
	 * <PRE>
	 * 
	 * public void captureEvent(Capture c) { c.read(); // do something exciting now that c has been updated }
	 * 
	 * </PRE>
	 */
	public boolean available() {
		return available;
	}

	public void read() {
		// try {
		// synchronized (capture) {
		if (image != null){
		loadPixels();
		synchronized (pixels) {
			// System.out.println("read1");
			if (crop) {
				// System.out.println("read2a");
				// f#$)(#$ing quicktime / jni is so g-d slow, calling copyToArray
				// for the invidual rows is literally 100x slower. instead, first
				// copy the entire buffer to a separate array (i didn't need that
				// memory anyway), and do an arraycopy for each row.
				/*
				 * if (data == null) { data = new int[dataWidth * dataHeight]; } raw.copyToArray(0, data, 0, dataWidth * dataHeight); int sourceOffset = cropX + cropY * dataWidth; int destOffset = 0; for (int y = 0; y < cropH; y++) { System.arraycopy(data, sourceOffset, pixels, destOffset, cropW); sourceOffset += dataWidth; destOffset += width; }
				 */
			} else { // no crop, just copy directly
				// System.out.println("read2b");
				// theData = (byte[]) imageBuffer.getData();

				PixelGrabber pg = new PixelGrabber(image,0,0,width,height,pixels,0,width);

				   try {
					      pg.grabPixels();
					    } catch (InterruptedException e) { }
				// raw.copyToArray(0, pixels, 0, width * height);
			// }
			// System.out.println("read3");
					    }
			available = false;
			// mark this image as modified so that PGraphicsJava2D and
			// PGraphicsOpenGL will properly re-blit and draw this guy
			updatePixels();
			// System.out.println("read4");
		}
		}
	}
	public void connect() {
		try {
			URL u = new URL(useMJPGStream ? mjpgURL : jpgURL);
			huc = (HttpURLConnection) u.openConnection();
			// System.out.println(huc.getContentType());
			InputStream is = huc.getInputStream();
			connected = true;
			BufferedInputStream bis = new BufferedInputStream(is);
			dis = new DataInputStream(bis);
			if (!initCompleted) initDisplay();
		} catch (IOException e) { // incase no connection exists wait and try again, instead of printing the error
			try {
				huc.disconnect();
				Thread.sleep(60);
			} catch (InterruptedException ie) {
				huc.disconnect();
				connect();
			}
			connect();
		} catch (Exception e) {
			;
		}
	}

	public void initDisplay() { // setup the display
		if (useMJPGStream)
			readMJPGStream();
		else {
			readJPG();
			disconnect();
		}
		initCompleted = true;
	}

	public void disconnect() {
		try {
			if (connected) {
				dis.close();
				connected = false;
			}
		} catch (Exception e) {
			;
		}
	}


	public void readStream() { // the basic method to continuously read the stream
		try {
			if (useMJPGStream) {
				while (true) {
					readMJPGStream();

				}
			} else {
				while (true) {
					connect();
					readJPG();

					disconnect();

				}
			}

		} catch (Exception e) {
			;
		}
	}

	public void readMJPGStream() { // preprocess the mjpg stream to remove the mjpg encapsulation
		readLine(3, dis); // discard the first 3 lines
		readJPG();
		readLine(2, dis); // discard the last two lines
	}
	public BufferedImage getImage(){
		available = false;
		return bimage;
	}
	public void readJPG() { // read the embedded jpeg image
		try {
			JPEGImageDecoder decoder = JPEGCodec.createJPEGDecoder(dis);
			bimage = decoder.decodeAsBufferedImage();
			image = bimage;
			available = true;
			if (captureEventMethod != null) {
				try {
					captureEventMethod.invoke(parent, new Object[] { this });
				} catch (Exception e) {
					System.err.println("Disabling captureEvent()  because of an error.");
					e.printStackTrace();
					captureEventMethod = null;
				}
			}

		} catch (Exception e) {
			e.printStackTrace();
			disconnect();
		}
	}
	/**
	 * Called by PApplet to shut down video so that QuickTime can be used later by another applet.
	 */
	public void dispose() {
		disconnect();

	}
	public void readLine(int n, DataInputStream dis) { // used to strip out the header lines
		for (int i = 0; i < n; i++) {
			readLine(dis);
		}
	}

	public void readLine(DataInputStream dis) {
		try {
			boolean end = false;
			String lineEnd = "\n"; // assumes that the end of the line is marked with this
			byte[] lineEndBytes = lineEnd.getBytes();
			byte[] byteBuf = new byte[lineEndBytes.length];

			while (!end) {
				dis.read(byteBuf, 0, lineEndBytes.length);
				String t = new String(byteBuf);
				// System.out.print(t); //uncomment if you want to see what the lines actually look like
				if (t.equals(lineEnd)) end = true;
			}
		} catch (Exception e) {
			e.printStackTrace();
		}

	}

	public void run() {
		connect();
		readStream();
	}

}

<<<<<<<

Hello Processing for Android (Bouncing Ball)

int xpos; int ypos; int xdir =1 ; int ydir = 1;

void setup() {

  fill(255, 0, 0);

}

void draw() {

  ellipse(xpos, ypos, 20, 20);
  if (xpos > width  || xpos < 0) xdir = - xdir;
  if (ypos > height || ypos < 0) ydir = - ydir;
  xpos = xpos + xdir;
  ypos = ypos + ydir;

} =======

int xpos; int ypos; int xdir =1 ; int ydir = 1;

void setup() {

  fill(255, 0, 0);

}

void draw() {

  ellipse(xpos, ypos, 20, 20);
  if (xpos > width  || xpos < 0) xdir = - xdir;
  if (ypos > height || ypos < 0) ydir = - ydir;
  xpos = xpos + xdir;
  ypos = ypos + ydir;

} >>>>>>>

Android Upload Picture

// set Sketch Permissions : INTERNET // Set Sketch Permissions : CAMERA

import java.io.ByteArrayOutputStream; import java.io.FileNotFoundException; import java.io.IOException; import java.io.OutputStream; import java.net.DatagramPacket; import java.net.DatagramSocket; import java.net.InetAddress; import java.net.SocketException; import java.net.UnknownHostException; import android.app.Activity; import android.graphics.ImageFormat; import android.graphics.Rect; import android.graphics.YuvImage; import android.hardware.Camera; import android.hardware.Camera.Size; import android.os.Bundle; import android.app.Activity; import android.content.ContentValues; import android.content.Context; import android.content.res.Configuration; import android.hardware.Camera; import android.net.DhcpInfo; import android.net.Uri; import android.net.wifi.WifiManager; import android.os.Bundle; import android.provider.MediaStore.Images.Media; import android.util.Log; import android.view.SurfaceHolder; import android.view.SurfaceView; import android.view.View; import android.view.View.OnClickListener; import android.widget.Toast;

import android.content.Context;

import android.hardware.Camera;

import android.hardware.Camera.PreviewCallback;

import android.hardware.Camera.PictureCallback;

import android.view.SurfaceHolder;

import android.view.SurfaceHolder.Callback;

import android.view.SurfaceView;

import android.view.Surface;

import android.graphics.PixelFormat;

import java.io.DataInputStream;

import java.io.DataOutputStream;

import java.io.IOException;

import java.net.HttpURLConnection;

import java.net.MalformedURLException;

import java.net.URL;

import java.net.URLEncoder;

import android.util.Log; int server_port ; DatagramSocket s ; InetAddress address ; int border = 0; int videoW = 176; int videoH = 144; int packetSize = 1000; CameraSurfaceView gCamSurfView; // This is the physical image drawn on the screen representing the camera: PImage gBuffer; byte[] imageBytes; void setup() {

  size(screenWidth, screenHeight);
  server_port = 12345;
  try {
    s = new DatagramSocket();
    address = InetAddress.getByName("128.122.151.100");
  } 
  catch (Exception e) {
    e.printStackTrace();
  }

} void draw() {

  // nuttin'... onPreviewFrame below handles all the drawing.

}

void mousePressed() {

  // doing other things here, and then:
  gCamSurfView.snapOne();

} //----------------------------------------------------------------------------------------- //----------------------------------------------------------------------------------------- // Override the parent (super) Activity class: // States onCreate(), onStart(), and onStop() aren't called by the sketch. Processing is entered // at the 'onResume()' state, and exits at the 'onPause()' state, so just override them: void onResume() {

  super.onResume();
  println("onResume()!");
  // Sete orientation here, before Processing really starts, or it can get angry:
  orientation(LANDSCAPE);
  // Create our 'CameraSurfaceView' objects, that works the magic:
  gCamSurfView = new CameraSurfaceView(this.getApplicationContext());

} class CameraSurfaceView extends SurfaceView implements SurfaceHolder.Callback, Camera.PreviewCallback {

  // Object that accesses the camera, and updates our image data
  // Using ideas pulled from 'Android Wireless Application Development', page 340

  SurfaceHolder mHolder;
  Camera camera = null;

  Camera.Size prevSize;

  // SurfaceView Constructor:  : ---------------------------------------------------
  CameraSurfaceView(Context context) {
    super(context);
    mHolder = getSurfaceHolder();
    //mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    mHolder.addCallback(this);
  }

  void snapOne() {
    int quality = 75;  //1-100
    YuvImage yuvimage = new YuvImage(imageBytes, ImageFormat.NV21, camera.getParameters().getPreviewSize().width, camera.getParameters().getPreviewSize().height, null);
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    yuvimage.compressToJpeg(new Rect(border, border, camera.getParameters().getPreviewSize().width-2*border, camera.getParameters().getPreviewSize().height-2*border), quality, baos);
    byte[] output = baos.toByteArray();
    String filename = "yourName+Data";
    new Uploadeer(this, "http://itp.nyu.edu/~dbo3/up.php", output, filename + ".jpg");
  }

  // SurfaceHolder.Callback stuff: ------------------------------------------------------
  public void surfaceCreated (SurfaceHolder holder) {
    camera = Camera.open();

    try {
      camera.setPreviewDisplay(holder);
      Camera.Parameters parameters = camera.getParameters();
      parameters.setPreviewSize(videoW, videoH); // data set: w:533, h:320
      Size size = parameters.getPreviewSize(); 
      camera.setPreviewCallback(this);
      camera.setParameters(parameters);
    } 
    catch (IOException exception) {
      camera.release();
    }
  }  

  public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
    camera.startPreview();
  }

  public void surfaceDestroyed (SurfaceHolder holder) {
    camera.stopPreview();
    camera.release();
  }

  //  Camera.PreviewCallback stuff: ------------------------------------------------------
  public   void onPreviewFrame(byte[] data, Camera cam) {
    imageBytes = data;
    ///COMPRESS IMAGE

    /*
    //Make sure it will fit in the packet, change the border to make it fit
     int msg_length= output.length;
     if (msg_length > packetSize-20) border = Math.min(videoH/2+4, border +1);
     else if (msg_length < packetSize-50) border = Math.max(1, border-1);
     //println("SIZe", "B:"+ border + " L:"+ msg_length);
     //SEND IMAGE
     if (msg_length <= packetSize) {
     DatagramPacket p = new DatagramPacket(output, msg_length, address, server_port);
     try {
     s.send(p);
     } 
     catch (IOException e) {
     println("Sending problem " + System.currentTimeMillis());
     }
     }
     */
  }

}

class Uploadeer extends Thread {

  String urlString;

  byte[] payload;

  String fileNameOnServer;

  CameraSurfaceView parent;

  public Uploadeer(CameraSurfaceView _p, String _urlString, byte[] _payload, String _fileNameOnServer) {
    parent = _p;
    urlString = _urlString;
    payload = _payload;
    fileNameOnServer = _fileNameOnServer;
    start();
  }

  public void run() {
    HttpURLConnection conn = null;
    DataOutputStream dos = null;
    DataInputStream inStream = null;

    String lineEnd = "\r\n";
    String twoHyphens = "--";
    String boundary = "-----------------------------29772313742745";

    try {
      // ------------------ CLIENT REQUEST

      URL url = new URL(urlString);
      // Open a HTTP connection to the URL
      conn = (HttpURLConnection) url.openConnection();

      // Allow Inputs
      conn.setDoInput(true);
      // Allow Outputs
      conn.setDoOutput(true);
      // Don't use a cached copy.
      conn.setUseCaches(false);
      // Use a post method.
      conn.setRequestMethod("POST");
      conn.setRequestProperty("Connection", "Keep-Alive");
      // conn.setRequestProperty("Cookie", "JSESSIONID="+PlayList.getSessionId());
      conn.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);

      dos = new DataOutputStream(conn.getOutputStream());

      dos.writeBytes(twoHyphens + boundary + lineEnd);
      // dos.writeBytes("Content-Disposition: form-data; name=\"fileNameOnServer\""+lineEnd+URLEncoder.encode(fileNameOnServer,"UTF-8") + lineEnd);
      dos.writeBytes(("Content-Disposition: form-data; name=\"data_file\"; filename=\"" + lineEnd + URLEncoder.encode(fileNameOnServer, "UTF-8") + "\"\r\n"));
      dos.writeBytes("Content-Type: " + "image/jpg" + " \r\n");
      dos.writeBytes(lineEnd);

      // create a buffer of maximum size
      dos.write(payload);

      dos.writeBytes(lineEnd);
      dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);


      dos.flush();
      dos.close();
    } 
    catch (MalformedURLException ex) {
      println( "error: " + ex.getMessage());
    }

    catch (Exception ioe) {
      println("error: " + ioe.getMessage());
    }

    // ------------------ read the SERVER RESPONSE

    try {
      inStream = new DataInputStream(conn.getInputStream());
      String str;

      while ( (str = inStream.readLine ()) != null) {
        println("Server Response" +str );
      }

      inStream.close();
    } 
    catch (Exception ioex) {
      println("error: " + ioex.getMessage());
    }
  }

}

OLD ANDROID UPLOAD PROGRAM

/**

 CameraPixelData
 Eric Pavey - 2010-11-15


 set Sketch Permissions : INTERNET
 Set Sketch Permissions : CAMERA
 Add to AndroidManifest.xml:
 uses-feature android:name="android.hardware.camera"
 uses-feature android:name="android.hardware.camera.autofocus"
 */

import android.content.Context;

import android.hardware.Camera;

import android.hardware.Camera.PreviewCallback;

import android.hardware.Camera.PictureCallback;

import android.view.SurfaceHolder;

import android.view.SurfaceHolder.Callback;

import android.view.SurfaceView;

import android.view.Surface;

import android.graphics.PixelFormat;

import java.io.DataInputStream;

import java.io.DataOutputStream;

import java.io.IOException;

import java.net.HttpURLConnection;

import java.net.MalformedURLException;

import java.net.URL;

import java.net.URLEncoder;

import android.util.Log;

// Setup camera globals:

CameraSurfaceView gCamSurfView;

// This is the physical image drawn on the screen representing the camera:

PImage gBuffer;

void setup() {

  size(screenWidth, screenHeight);

}

void draw() {

  // nuttin'... onPreviewFrame below handles all the drawing.

}

void mousePressed() {

  // doing other things here, and then:
  gCamSurfView.snapOne();

}

//----------------------------------------------------------------------------------------- //----------------------------------------------------------------------------------------- // Override the parent (super) Activity class: // States onCreate(), onStart(), and onStop() aren't called by the sketch. Processing is entered // at the 'onResume()' state, and exits at the 'onPause()' state, so just override them:

void onResume() {

  super.onResume();
  println("onResume()!");
  // Sete orientation here, before Processing really starts, or it can get angry:
  orientation(LANDSCAPE);

  // Create our 'CameraSurfaceView' objects, that works the magic:
  gCamSurfView = new CameraSurfaceView(this.getApplicationContext());

}

//----------------------------------------------------------------------------------------- //-----------------------------------------------------------------------------------------

class CameraSurfaceView extends SurfaceView implements SurfaceHolder.Callback, Camera.PreviewCallback, Camera.PictureCallback {

  // Object that accesses the camera, and updates our image data
  // Using ideas pulled from 'Android Wireless Application Development', page 340

  SurfaceHolder mHolder;
  Camera cam = null;
  Camera.Size prevSize;

  // SurfaceView Constructor:  : ---------------------------------------------------
  CameraSurfaceView(Context context) {
    super(context);
    // Processing PApplets come with their own SurfaceView object which can be accessed
    // directly via its object name, 'surfaceView', or via the below function:
    // mHolder = surfaceView.getHolder();
    mHolder = getSurfaceHolder();
    // Add this object as a callback:
    mHolder.addCallback(this);
  }

void finishedUpload(){

  if(cam != null)
    cam.startPreview();
    println("Finished Upload");

}

  void snapOne() {
    println("SnapOne");
    if (cam != null)
    cam.takePicture(null, null, null, this);

  }

  // SurfaceHolder.Callback stuff: ------------------------------------------------------
  void surfaceCreated (SurfaceHolder holder) {
    // When the SurfaceHolder is created, create our camera, and register our
    // camera's preview callback, which will fire on each frame of preview:
    cam = Camera.open();

    cam.setOneShotPreviewCallback(this);
    cam.setPreviewCallback(this);
    Camera.Parameters parameters= cam.getParameters();

    parameters.setPictureFormat(PixelFormat.JPEG);
    // Find our preview size, and init our global PImage:
    prevSize = parameters.getPreviewSize();
    gBuffer = createImage(prevSize.width, prevSize.height, RGB);
    cam.setParameters(parameters);
  }  
  public void onPictureTaken(byte[] data, Camera camera) {
    println("Took picture " + data.length);
    //cam.stopPreview();
    new Uploadeer(this,"http://itp.nyu.edu/~dbo3/up.php", data, "danoAndroidTest.jpg");

  }


  void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
    // Start our camera previewing:
    if (cam != null)
    cam.startPreview();
  }

  void surfaceDestroyed (SurfaceHolder holder) {
    // Give the cam back to the phone:
    cam.stopPreview();
    cam.release();
    cam = null;
  }

  //  Camera.PreviewCallback stuff: ------------------------------------------------------
  void onPreviewFrame(byte[] data, Camera cam) {
    // This is called every frame of the preview.  Update our global PImage.
    gBuffer.loadPixels();
    // Decode our camera byte data into RGB data:
    decodeYUV420SP(gBuffer.pixels, data, prevSize.width, prevSize.height);
    gBuffer.updatePixels();
    // Draw to screen:

    image(gBuffer, 0, 0);
  }



  //  Byte decoder : ---------------------------------------------------------------------
  void decodeYUV420SP(int[] rgb, byte[] yuv420sp, int width, int height) {
    // Pulled directly from:
    // http://ketai.googlecode.com/svn/trunk/ketai/src/edu/uic/ketai/inputService/KetaiCamera.java
    final int frameSize = width * height;

    for (int j = 0, yp = 0; j < height; j++) {       
      int uvp = frameSize + (j >> 1) * width, u = 0, v = 0;
      for (int i = 0; i < width; i++, yp++) {
        int y = (0xff & ((int) yuv420sp[yp])) - 16;
        if (y < 0)
          y = 0;
        if ((i & 1) == 0) {
          v = (0xff & yuv420sp[uvp++]) - 128;
          u = (0xff & yuv420sp[uvp++]) - 128;
        }

        int y1192 = 1192 * y;
        int r = (y1192 + 1634 * v);
        int g = (y1192 - 833 * v - 400 * u);
        int b = (y1192 + 2066 * u);

        if (r < 0)
          r = 0;
        else if (r > 262143)
          r = 262143;
        if (g < 0)
          g = 0;
        else if (g > 262143)
          g = 262143;
        if (b < 0)
          b = 0;
        else if (b > 262143)
          b = 262143;

        rgb[yp] = 0xff000000 | ((r << 6) & 0xff0000) | ((g >> 2) & 0xff00) | ((b >> 10) & 0xff);
      }
    }
  }

}

class Uploadeer extends Thread {

  String urlString;

  byte[] payload;

  String fileNameOnServer;

  CameraSurfaceView parent;

  public Uploadeer(CameraSurfaceView _p,String _urlString, byte[] _payload, String _fileNameOnServer) {
    parent = _p;
    urlString = _urlString;
    payload = _payload;
    fileNameOnServer = _fileNameOnServer;
    start();
  }

  public void run() {
    HttpURLConnection conn = null;
    DataOutputStream dos = null;
    DataInputStream inStream = null;

    String lineEnd = "\r\n";
    String twoHyphens = "--";
    String boundary = "-----------------------------29772313742745";

    try {
      // ------------------ CLIENT REQUEST

      URL url = new URL(urlString);
      // Open a HTTP connection to the URL
      conn = (HttpURLConnection) url.openConnection();

      // Allow Inputs
      conn.setDoInput(true);
      // Allow Outputs
      conn.setDoOutput(true);
      // Don't use a cached copy.
      conn.setUseCaches(false);
      // Use a post method.
      conn.setRequestMethod("POST");
      conn.setRequestProperty("Connection", "Keep-Alive");
      // conn.setRequestProperty("Cookie", "JSESSIONID="+PlayList.getSessionId());
      conn.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);

      dos = new DataOutputStream(conn.getOutputStream());

      dos.writeBytes(twoHyphens + boundary + lineEnd);
      // dos.writeBytes("Content-Disposition: form-data; name=\"fileNameOnServer\""+lineEnd+URLEncoder.encode(fileNameOnServer,"UTF-8") + lineEnd);
      dos.writeBytes(("Content-Disposition: form-data; name=\"data_file\"; filename=\"" + lineEnd + URLEncoder.encode(fileNameOnServer, "UTF-8") + "\"\r\n"));
      dos.writeBytes("Content-Type: " + "image/jpg" + " \r\n");
      dos.writeBytes(lineEnd);

      // create a buffer of maximum size
      dos.write(payload);

      dos.writeBytes(lineEnd);
      dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);


      dos.flush();
      dos.close();
    } 
    catch (MalformedURLException ex) {
      println( "error: " + ex.getMessage());
    }

    catch (Exception ioe) {
      println("error: " + ioe.getMessage());
    }

    // ------------------ read the SERVER RESPONSE

    try {
      inStream = new DataInputStream(conn.getInputStream());
      String str;

      while ( (str = inStream.readLine ()) != null) {
        println("Server Response" +str );
      }
      parent.finishedUpload();
      inStream.close();
    } 
    catch (Exception ioex) {
      println("error: " + ioex.getMessage());
    }
  }

}

Upload an Image Using Mobile Processing

import processing.video.*;

import java.io.DataOutputStream; import java.io.InputStream;

import javax.microedition.io.Connector; import javax.microedition.io.HttpConnection;

public class CaptureIt1 extends PMIDlet {

  Capture myCam;
  PImage myImage = null;
  int xpos = 0;
  int ypos = 0;


  void setup(){
    myCam = new Capture(this);
    myCam.show(0,0);  //this takes over the screen showing camera preview
    noLoop();  //this disables the draw loop

  }

  void draw(){
    background(255);
    if (myImage != null){
      image(myImage,xpos,ypos);
    }
  }

  void keyPressed(){
    if (key == '5'){
      byte[] imageData = myCam.read("image/png",160,120); //read in the picture
      //  new PostFile(imageData, "test.jpg", "http://itp.nyu.edu/~dbo3/up.php");
      myCam.hide(); //tell Capture to stop previewing
      myImage = loadImage(imageData); //make an image from the array of bytes
      loop();  //this reenables the draw loop
    }
    else if (key == '4'){
      xpos = xpos -1;
    }
    else if (key == '6'){
      xpos = xpos + 1;
    } 
    else if (key == '9'){
      //resume preview
      myCam.show(0,0);  //this takes over the screen showing camera preview
      noLoop();  //this disables the draw loop
    }
  }

  void destroy() {
    //// close the camera object on sketch exit
    myCam.close();
  }

}

Kinect Track Closest Thing

import org.openkinect.processing.Kinect;

import processing.core.PApplet;

public class AveragePoint extends PApplet {

    // Dan O'Sullivan based on 
	// Daniel Shiffman
	// Kinect Point Cloud example
	// http://www.shiffman.net
	// https://github.com/shiffman/libfreenect/tree/master/wrappers/java/processing

	// Kinect Library object
	Kinect kinect;

	// Size of kinect image
	int w = 640;

	int h = 480;

	int threshold = 550;

	public void setup() {
		size(w, h);
		kinect = new Kinect(this);
		kinect.start();
		kinect.enableDepth(true);

		// We don't need the grayscale image in this example
		// so this makes it more efficient
		kinect.processDepthImage(false);
		ellipseMode(CENTER);

	}

	public void draw() {

		background(0);
		fill(255);
		textMode(SCREEN);
		text("Kinect FR: " + (int) kinect.getDepthFPS() + "\nProcessing FR: " + (int) frameRate, 10, 16);

		// Get the raw depth as array of integers
		int[] depth = kinect.getRawDepth();

		int allX = 0;
		int allY = 0;
		int all = 0;

		for (int x = 0; x < w; x ++) {
			for (int y = 0; y < h; y ++) {
				int offset = x + y * w;
				int rawDepth = depth[offset];
				if (depth == null) return;
				if (rawDepth < threshold) {
					allX += x;
					allY += y;
					all++;
				}
			}

		}
		if (all != 0) ellipse(width - allX / all, allY / all, 20, 20);

	}

	public void keyPressed() {
		if (key == '-') {
			threshold--;
		} else if (key == '=') {
			threshold++;
		}
		println("Threshold:" + threshold);
	}

	public void stop() {
		kinect.quit();
		super.stop();
	}

}

Kinect Remove Background

import java.awt.Rectangle;

import org.openkinect.processing.Kinect;

import processing.core.PApplet; import processing.core.PImage;

public class RemoveBackground extends PApplet {

    // Dan O'Sullivan based on 
	// Daniel Shiffman
	// Kinect Point Cloud example
	// http://www.shiffman.net
	// https://github.com/shiffman/libfreenect/tree/master/wrappers/java/processing

	// Kinect Library object
	Kinect kinect;

	// Size of kinect image
	int w = 640;

	int h = 480;

	int frontThreshold = 500;

	int backThreshold = 800;

	int depthXPicOffset = 30;
	int depthYPicOffset = 30;

	public void setup() {

		size(w, h);
		kinect = new Kinect(this);
		kinect.start();
		kinect.enableDepth(true);
		kinect.enableRGB(true);

		// We don't need the grayscale image in this example
		// so this makes it more efficient
		kinect.processDepthImage(false);

	}

	public void draw() {
		background(255, 0, 0);
		loadPixels();

		fill(255);
		PImage myImage = kinect.getVideoImage();
		// Get the raw depth as array of integers
		int[] depth = kinect.getRawDepth();
		// We're just going to calculate and draw every 4th pixel (equivalent of 160x120)
		int skip = 1;

		for (int x = 0; x < w; x += skip) {
			for (int y = 0; y < h; y += skip) {
				int offset = x + y * w;
				int rawDepth = depth[offset];
				if (rawDepth > frontThreshold && rawDepth < backThreshold) {
					int imageOffset = x - depthXPicOffset + (y + depthYPicOffset) * w;
					if (imageOffset < myImage.pixels.length) (pixels[offset] )  = myImage.pixels[imageOffset] ;
				}
			}
		}
		updatePixels();

	}

	public void keyPressed() {
		if (key == '-') {
			frontThreshold--;
		} else if (key == '=') {
			frontThreshold++;
		}
		if (key == '_') {
			backThreshold--;
		} else if (key == '+') {
			backThreshold++;
		}
		if (key == 'x') {
			depthXPicOffset--;
		} else if (key == 'X') {
			depthXPicOffset++;
		}else if (key == 'y') {
			depthYPicOffset--;
		} else if (key == 'y') {
			depthYPicOffset++;
		}
		println("Threshold between:" + frontThreshold + " and " + backThreshold + " depthPicOffset" + depthXPicOffset + " " + depthXPicOffset);
	}

	public void stop() {
		kinect.quit();
		super.stop();
	}

}

Track Multiple Rectangles with Kinect

import java.awt.Rectangle; import java.util.ArrayList;

import org.openkinect.processing.Kinect;

import processing.core.PApplet; import processing.core.PVector;

public class MultipleRects extends PApplet {

	// Dan O'Sullivan based on
	// Daniel Shiffman
	// Kinect Point Cloud example
	// http://www.shiffman.net
	// https://github.com/shiffman/libfreenect/tree/master/wrappers/java/processing

	// Kinect Library object
	Kinect kinect;

	// Size of kinect image
	int w = 640;

	int h = 480;

	int threshold = 550;

	public void setup() {
		size(w, h);
		kinect = new Kinect(this);
		kinect.start();
		kinect.enableDepth(true);

		// We don't need the grayscale image in this example
		// so this makes it more efficient
		kinect.processDepthImage(false);

	}

	public void draw() {

		background(0);
		fill(255);
		textMode(SCREEN);
		text("Kinect FR: " + (int) kinect.getDepthFPS() + "\nProcessing FR: " + (int) frameRate, 10, 16);

		// Get the raw depth as array of integers

		int[] depth = kinect.getRawDepth();

		int skip = 1;
		int reach = 5;

		ArrayList myRects = new ArrayList();
		for (int x = 0; x < w; x += skip) {
			for (int y = 0; y < h; y += skip) {
				int offset = x + y * w;

				int rawDepth = depth[offset];

				if (rawDepth < threshold) {

					boolean foundAHome = false;
					for (int i = 0; i < myRects.size(); i++) {
						Rectangle thisRect = (Rectangle) myRects.get(i);
						Rectangle bigger = new Rectangle(thisRect);
						bigger.grow(reach, reach);
						if (bigger.contains(x, y)) {
							thisRect.add(x, y);
							foundAHome = true;
							break;
						}

					}
					if (foundAHome == false) {
						myRects.add(new Rectangle(x, y, 1, 1));
					}
				}
			}

		}
		fill(0, 0, 0, 0);
		stroke(255, 0, 0);
		for (int i = 0; i < myRects.size(); i++) {
			Rectangle thisBox = (Rectangle) myRects.get(i);
			if (thisBox.width * thisBox.height < 400) continue;
			rect(thisBox.x, thisBox.y, thisBox.width, thisBox.height);
		}

	}

	public void keyPressed() {
		if (key == '-') {
			threshold--;
		} else if (key == '=') {
			threshold++;
		}
		println("Threshold:" + threshold);
	}

	public void stop() {
		kinect.quit();
		super.stop();
	}

}

import java.awt.image.BufferedImage; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; import java.io.DataInputStream; import java.io.DataOutputStream; import java.io.IOException; import java.io.UTFDataFormatException;

import processing.core.PApplet; import processing.core.PImage; import processing.net.Client; import processing.net.Server; import processing.video.Capture;

import com.sun.image.codec.jpeg.ImageFormatException; import com.sun.image.codec.jpeg.JPEGCodec; import com.sun.image.codec.jpeg.JPEGEncodeParam; import com.sun.image.codec.jpeg.JPEGImageDecoder; import com.sun.image.codec.jpeg.JPEGImageEncoder;

public class TeleView2 extends PApplet {

	Server myServer;

	int endOfMessageChar = 10;

	Client myClient;

	int socketPort = 10002;

	String ip = "localhost";

	Capture video;

	boolean serverRole = true;

	static String[] args;

	static public void main(String _args[]) {
		args = _args;
		PApplet.main(new String[] { "upload.TeleView2" });
	}

	public void setup() {
		size(320, 240);
		getParams();
		video = new Capture(this, 320, 240);
		if (serverRole) {
			myServer = new Server(this, socketPort);
			println("Starting a server at port " + socketPort);
		} else {
			myClient = new Client(this, ip, socketPort);
			println("Connecting as client at " + socketPort + " on " + ip);
			new TeleListener(myClient);
			new Sender(myClient, video);
		}

		frameRate(20);

	}

	public void draw() {
		// listen to client, should be able to do this with ClientEvent but this seems not to work with Clients created by the server
		if (video.available()) {
			video.read();

		}

	}

	 public void mousePressed(){
	 if (myClient != null && video != null)
	 new Sender(myClient,video);
	 }
	public void incoming(PImage _img) {
		if (_img != null) image(_img, 0, 0);
		new Sender(myClient, video);
	}

	public void serverEvent(Server someServer, Client _newClient) {
		myClient = _newClient;
		new TeleListener(myClient);
		println("We have a new client: " + _newClient.ip());
	}

	public byte[] imageToByteArray(PImage _image) {

		// bit if a hack to convert PImage to a Buffered Image, draw it on the buffered image
		// ((BufferedImage) image).getRGB(x, y, w, h, img.pixels, offset, img.width);

		BufferedImage bi = new BufferedImage(_image.width, _image.height, BufferedImage.TYPE_INT_RGB);

		bi.setRGB(0, 0, _image.width, _image.height, _image.pixels, 0, _image.width);
		// compress and turn into a byte array
		// okay just to copy and paste this method

		ByteArrayOutputStream baOut = new ByteArrayOutputStream();
		JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(baOut);
		JPEGEncodeParam param = encoder.getDefaultJPEGEncodeParam(bi);
		param.setQuality(0.7f, false);
		encoder.setJPEGEncodeParam(param);
		try {
			encoder.encode(bi);
			baOut.flush();
		} catch (ImageFormatException e) {
			System.out.println("could not  form image" + e);
		} catch (IOException e) {
			System.out.println("could not encode image" + e);
		}

		return baOut.toByteArray();
	}

	public PImage byteArrayToPImage(byte[] _pic) {
		// decompress into a buffferedImage
		// okay just to copy and paste this method
		ByteArrayInputStream in = new ByteArrayInputStream((byte[]) _pic);
		JPEGImageDecoder decoder = JPEGCodec.createJPEGDecoder(in);
		BufferedImage snapShot = null;
		try {
			snapShot = decoder.decodeAsBufferedImage();
		} catch (ImageFormatException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (IOException e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		} catch (Exception e) {
			// TODO Auto-generated catch block
			e.printStackTrace();
		}
		if (snapShot == null) return null;
		return new PImage(snapShot);
	}

	void getParams() {
		if (args != null) { // running as an application
			if (args.length > 0) socketPort = Integer.parseInt(args[0]);
			if (args.length > 1) {
				ip = args[1];
				serverRole = false;
			}
		} else { // running as an applet

			String isItThere = getParameter("socketPort");
			if (isItThere != null) socketPort = Integer.parseInt(isItThere);
			isItThere = getParameter("ip");
			if (isItThere != null) ip = isItThere;

		}
	}

	public class TeleListener extends Thread {
		DataInputStream dis;

		TeleListener(Client _c) {
			dis = new DataInputStream(_c.input);
			start();
		}

		public void run() {
			while (true) {
				int size;
				try {
					try {

						String sizeString = dis.readUTF();
						if (sizeString == null) break;

						size = Integer.parseInt(sizeString.trim());
					} catch(UTFDataFormatException e){
						size = dis.available();
					}catch (RuntimeException e) {

						size = dis.available();
					}
					// make a buffer that size to accept the incoming bytes
					byte[] incoming = new byte[size];
					dis.readFully(incoming);

					PImage myImage = byteArrayToPImage(incoming);

					incoming(myImage);
				} catch (NumberFormatException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
					break;
				} catch (IOException e) {
					// TODO Auto-generated catch block
					e.printStackTrace();
					break;
				}

			}
		}
	}

	public class Sender extends Thread {
		DataOutputStream dos;

		PImage img;

		Sender(Client _c, PImage _image) {
			dos = new DataOutputStream(_c.output);
			img = (PImage) _image;
			start();
		}

		public void run() {
			byte[] imageInBytes = imageToByteArray(img);

			// tell them how many bytes to expect
			String outsize = String.valueOf(imageInBytes.length) + "\n";
			// println("Sending" + serverRole + " " + size);

			try {
				dos.writeUTF(outsize);
				dos.flush();
				dos.write(imageInBytes);
				dos.flush();
			} catch (IOException e) {
				// TODO Auto-generated catch block
				e.printStackTrace();
			}

		}

	}

}

import java.awt.*; import java.io.* import java.net.*

import java.awt.image.BufferedImage; import processing.core.PApplet; import processing.core.PImage; import processing.video.Capture;

public class UploadPicture extends PApplet {

  Capture video;//regular processing libary

  static public void main(String _args[]) {
    PApplet.main(new String[] { 
      "UploadPicture"
    }
    );
  }



  public void setup() {
    size(640, 480);
    video = new Capture(this, 640, 480);
  }



  public void draw() {

    if (video.available()) {
      video.read();
      image(video, 0, 0);
    }
  }



  public void keyPressed() {
    if (key == 's') {
      video.settings();
    }
    else if (key == 'g') {
      println("Grab it");
      byte[] a = imageToByteArray(video);
      new Uploadeer("http://itp.nyu.edu/~dbo3/up.php", a, "processingTest.jpg");
    }
  }

  public byte[] imageToByteArray(PImage _image) {

    BufferedImage bi = new BufferedImage(_image.width, _image.height,BufferedImage.TYPE_INT_RGB);
    bi.setRGB(0, 0, _image.width, _image.height, _image.pixels, 0, _image.width);
    //compress and turn into a byte array
    //okay just to copy and paste this method

    ByteArrayOutputStream baOut = new ByteArrayOutputStream();
    JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(baOut);
    JPEGEncodeParam param = encoder.getDefaultJPEGEncodeParam(bi);
    param.setQuality(0.9f, false);
    encoder.setJPEGEncodeParam(param);
    try {
      encoder.encode(bi);
    } 
    catch (ImageFormatException e) {
      System.out.println("could not  form image" + e);
    } 
    catch (IOException e) {
      System.out.println("could not encode image" + e);
    }

    return baOut.toByteArray();
  }



  class Uploadeer extends Thread {
    String urlString;
    byte[] payload;
    String fileNameOnServer;
    public Uploadeer(String _urlString, byte[] _payload, String _fileNameOnServer) {
      urlString = _urlString;
      payload = _payload;
      fileNameOnServer = _fileNameOnServer;
      start();
    }

    public void run() {
      HttpURLConnection conn = null;
      DataOutputStream dos = null;
      DataInputStream inStream = null;
      String lineEnd = "\r\n";
      String twoHyphens = "--";
      String boundary = "-----------------------------29772313742745";
      try {
        // ------------------ CLIENT REQUEST

        URL url = new URL(urlString);
        // Open a HTTP connection to the URL
        conn = (HttpURLConnection) url.openConnection();
        // Allow Inputs
        conn.setDoInput(true);
        // Allow Outputs
        conn.setDoOutput(true);
        // Don't use a cached copy.
        conn.setUseCaches(false);
        // Use a post method.
        conn.setRequestMethod("POST");
        conn.setRequestProperty("Connection", "Keep-Alive");
        // conn.setRequestProperty("Cookie", "JSESSIONID="+PlayList.getSessionId());
        conn.setRequestProperty("Content-Type", "multipart/form-data;boundary=" + boundary);
        dos = new DataOutputStream(conn.getOutputStream());
        dos.writeBytes(twoHyphens + boundary + lineEnd);
        // dos.writeBytes("Content-Disposition: form-data; name=\"fileNameOnServer\""+lineEnd+URLEncoder.encode(fileNameOnServer,"UTF-8") + lineEnd);
        dos.writeBytes(("Content-Disposition: form-data; name=\"data_file\"; filename=\"" + lineEnd + URLEncoder.encode(fileNameOnServer, "UTF-8") + "\"\r\n"));
        dos.writeBytes("Content-Type: " + "image/jpg" + " \r\n");
        dos.writeBytes(lineEnd);
        // create a buffer of maximum size
        dos.write(payload);
        dos.writeBytes(lineEnd);
        dos.writeBytes(twoHyphens + boundary + twoHyphens + lineEnd);
        dos.flush();
        dos.close();
      } 
      catch (MalformedURLException ex) {
        println( "error: " + ex.getMessage());
      }

      catch (Exception ioe) {
        println("error: " + ioe.getMessage());
      }

      // ------------------ read the SERVER RESPONSE

      try {
        inStream = new DataInputStream(conn.getInputStream());
        String str;

        while ( (str = inStream.readLine ()) != null) {
          println("Server Response" );
        }
        inStream.close();
      } 
      catch (Exception ioex) {
        println("error: " + ioex.getMessage());
      }
    }
  }

}

PHP for Receiving Images on the Server

<? //print_r($_POST); //print_r($_FILES); //This is a function used later on for testing extenstions //skip it for now function endsWith( $str, $sub ) {

   return ( substr( $str, strlen( $str ) - strlen( $sub ) ) === $sub );

} //get ready unset($filename);

//get the files in from the post if(!isset($_FILES) && isset($HTTP_POST_FILES)) $_FILES = $HTTP_POST_FILES; //if no file was sent if(!isset($_FILES['data_file'])) $error["data_file"] = "File named data_file was not found.";

$filename = urldecode($_FILES['data_file']['name'] ); //this will contain the extra information that you sent over //if they don't have a path name use the default of an "up" //folder in the current directory. This requires you to have a //world writable "up" directory in the same directory as up.php //maybe not so cool to be able to put it other places on the machine //$pos = strpos($filename,"/home/"); //if ($pos === false) {

			$filename = "./up/" . $filename;

//} //if they did not send over any filename if(empty($filename)) $error["filename"] = "The name of the file was not found.";

//if there were no previous problems if(empty($error)) {

	//security measure, only good file extensions get in
	if (endsWith($filename, ".png") || endsWith($filename,".jpg") || endsWith($filename,".log")){
		//this is it, move and name the file within the server
		$result = @move_uploaded_file($_FILES['data_file']['tmp_name'], $filename );
		if(empty($result))
			$error["result"] = "There was an error moving the uploaded file. size" . filesize($result );
	}else{
		$error["extension"] = "Bad File Extension.";
	}

}

if(is_array($error)) {

	//send back the litany of errors
	while(list($key, $val) = each($error))
		{
		echo $val;
		echo "<br>\n";
		}

}else{

	//send back a sunny message
	$size = filesize($tmp );
	echo "OK: $size bytes FILENAME: $filename  ";

}

Kinect Simple Hand Tracking

import SimpleOpenNI.*;

SimpleOpenNI context; boolean handsTrackFlag = false; PVector handVec = new PVector();

void setup() {

  size(640, 480);  // strange, get drawing error in the cameraFrustum if i use P3D, in opengl there is no problem
  //size(1024,768,OPENGL);
  context = new SimpleOpenNI(this);
  context.setMirror(false);
  context.enableDepth();
  //unfortunately have to track guestures to get hands
  context.enableGesture();
  context.enableHands();
  context.addGesture("RaiseHand");
  fill(255, 0, 0);

}

void draw() {

  // update the cam
  context.update();
  //paint the image
  image(context.depthImage(), 0, 0, width, height);
  if (handsTrackFlag) 
  {
    PVector myPositionScreenCoords  = new PVector(); //storage device
    //convert the weird kinect coordinates to screen coordinates.
    context.convertRealWorldToProjective(handVec, myPositionScreenCoords);
    ellipse(myPositionScreenCoords.x, myPositionScreenCoords.y, 20, 20);
  }

}

// ----------------------------------------------------------------- // hand events

void onCreateHands(int handId, PVector pos, float time) {

  println("onCreateHands - handId: " + handId + ", pos: " + pos + ", time:" + time);
  handsTrackFlag = true;
  handVec = pos;

}

void onUpdateHands(int handId, PVector pos, float time) {

  //println("onUpdateHandsCb - handId: " + handId + ", pos: " + pos + ", time:" + time);
  //store the location of the hand in a vector object
  handVec = pos;

}

void onDestroyHands(int handId, float time) {

  println("onDestroyHandsCb - handId: " + handId + ", time:" + time);
  handsTrackFlag = false;
  //go back to looking for the guesture that gave you hand.
  context.addGesture("RaiseHand");

}

// ----------------------------------------------------------------- // gesture events

void onRecognizeGesture(String strGesture, PVector idPosition, PVector endPosition) {

  println("onRecognizeGesture - strGesture: " + strGesture + ", idPosition: " + idPosition + ", endPosition:" + endPosition);
  //stop looking for the gesture
  context.removeGesture(strGesture);
  //use the location of this guesture tell you where to start tracking the hand
  context.startTrackingHands(endPosition);

}

void onProgressGesture(String strGesture, PVector position, float progress) {

  //println("onProgressGesture - strGesture: " + strGesture + ", position: " + position + ", progress:" + progress);

}

// ----------------------------------------------------------------- // Keyboard event

void keyPressed() {

  switch(key)
  {
  case ' ':
    context.setMirror(!context.mirror());
    break;
  }

}

Kinect Background Removal

import processing.opengl.*; import SimpleOpenNI.*;

SimpleOpenNI kinect;

//based on Greg's Book Making things see.

boolean tracking = false; int userID; int[] userMap; // declare our images PImage backgroundImage; PImage resultImage;

void setup() {

  size(1280, 480, OPENGL);
  // load the background image CHANGE htp to http
  backgroundImage = loadImage("htp://www.eheart.com/SHASTA/wallpaper/alpenglow-640x480.jpg");
  kinect = new SimpleOpenNI(this);
  kinect.enableDepth();
  // enable color image from the Kinect
  kinect.enableRGB();
  kinect.enableUser(SimpleOpenNI.SKEL_PROFILE_NONE);
  // turn on depth-color alignment
  kinect.alternativeViewPointDepthToImage();
  //create a buffer image to work with instead of using sketch pixels
  resultImage = new PImage(640, 480, RGB);

}

void draw() {

  kinect.update();
  // get the Kinect color image
  PImage rgbImage = kinect.rgbImage();

  image(rgbImage, 640, 0);
  if (tracking) {
    //ask kinect for bitmap of user pixels
    userMap = kinect.getUsersPixels(SimpleOpenNI.USERS_ALL);
    for (int i =0; i < userMap.length; i++) {
      // if the pixel is part of the user
      if (userMap[i] != 0) {
        // set the pixel to the color pixel
        resultImage.pixels[i] = rgbImage.pixels[i];
      }
      else {
        //set it to the background
        resultImage.pixels[i] = backgroundImage.pixels[i];
      }
    }
    //update the pixel from the inner array to image
    resultImage.updatePixels();
    image(resultImage, 0, 0);
  }

}

void onNewUser(int uID) {

  userID = uID;
  tracking = true;
  println("tracking");

}

Kinect Calibration and Joint Distance

import SimpleOpenNI.*; SimpleOpenNI kinect; String youAre = "";

void setup() {

  kinect = new SimpleOpenNI(this);
  kinect.enableDepth();
  kinect.enableUser(SimpleOpenNI.SKEL_PROFILE_ALL);

  size(640, 480);
  fill(255, 0, 0);
  stroke(0, 0, 255);
  strokeWeight(3);
  smooth();
  PFont myFont = createFont("FFScala", 32);
  textFont(myFont);

}

void draw() {

  kinect.update();
  image(kinect.depthImage(), 0, 0);
  text(youAre, 100, 100);
  IntVector userList = new IntVector();
  kinect.getUsers(userList);

  if (userList.size() > 0) {
    int userId = userList.get(0);

    if ( kinect.isTrackingSkeleton(userId)) {
      drawSkeleton(userId);
    }

    float elbowDistance = getJointDistance(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW, SimpleOpenNI.SKEL_LEFT_ELBOW);
    //float hdist = PVector.dist(leftHand, rightHand);
    println(elbowDistance );
    if (elbowDistance > 450) {
      youAre = "Chillin'";
    }
    else {
      youAre = "Guarded";
    }
  }
  //c

}

void drawSkeleton(int userId) {

  stroke(0);
  strokeWeight(5);

  kinect.drawLimb(userId, SimpleOpenNI.SKEL_HEAD, SimpleOpenNI.SKEL_NECK);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_LEFT_SHOULDER);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_LEFT_ELBOW);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_ELBOW, SimpleOpenNI.SKEL_LEFT_HAND);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_NECK, SimpleOpenNI.SKEL_RIGHT_SHOULDER);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_RIGHT_ELBOW);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW, SimpleOpenNI.SKEL_RIGHT_HAND);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER, SimpleOpenNI.SKEL_TORSO);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER, SimpleOpenNI.SKEL_TORSO);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_LEFT_HIP);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_HIP, SimpleOpenNI.SKEL_LEFT_KNEE);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_LEFT_KNEE, SimpleOpenNI.SKEL_LEFT_FOOT);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_TORSO, SimpleOpenNI.SKEL_RIGHT_HIP);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_HIP, SimpleOpenNI.SKEL_RIGHT_KNEE);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_KNEE, SimpleOpenNI.SKEL_RIGHT_FOOT);
  kinect.drawLimb(userId, SimpleOpenNI.SKEL_RIGHT_HIP, SimpleOpenNI.SKEL_LEFT_HIP);

  noStroke();

  fill(255, 0, 0);
  drawJoint(userId, SimpleOpenNI.SKEL_HEAD);
  drawJoint(userId, SimpleOpenNI.SKEL_NECK);
  drawJoint(userId, SimpleOpenNI.SKEL_LEFT_SHOULDER);
  drawJoint(userId, SimpleOpenNI.SKEL_LEFT_ELBOW);
  drawJoint(userId, SimpleOpenNI.SKEL_NECK);
  drawJoint(userId, SimpleOpenNI.SKEL_RIGHT_SHOULDER);
  drawJoint(userId, SimpleOpenNI.SKEL_RIGHT_ELBOW);
  drawJoint(userId, SimpleOpenNI.SKEL_TORSO);
  drawJoint(userId, SimpleOpenNI.SKEL_LEFT_HIP);
  drawJoint(userId, SimpleOpenNI.SKEL_LEFT_KNEE);
  drawJoint(userId, SimpleOpenNI.SKEL_RIGHT_HIP);
  drawJoint(userId, SimpleOpenNI.SKEL_LEFT_FOOT);
  drawJoint(userId, SimpleOpenNI.SKEL_RIGHT_KNEE);
  drawJoint(userId, SimpleOpenNI.SKEL_LEFT_HIP);
  drawJoint(userId, SimpleOpenNI.SKEL_RIGHT_FOOT);
  drawJoint(userId, SimpleOpenNI.SKEL_RIGHT_HAND);
  drawJoint(userId, SimpleOpenNI.SKEL_LEFT_HAND);

}

float getJointDistance(int userId, int jointID1, int jointID2) {

  PVector joint1 = new PVector();
  PVector joint2 = new PVector();
  kinect.getJointPositionSkeleton(userId, jointID1, joint1);
  kinect.getJointPositionSkeleton(userId, jointID2, joint2);
  return PVector.dist(joint1, joint2);

} void drawJoint(int userId, int jointID) {

  PVector joint = new PVector();
  float confidence = kinect.getJointPositionSkeleton(userId, jointID, joint);
  if (confidence < 0.5) {
    return;
  }
  PVector convertedJoint = new PVector();
  kinect.convertRealWorldToProjective(joint, convertedJoint);
  ellipse(convertedJoint.x, convertedJoint.y, 5, 5);

}

// user-tracking callbacks! void onNewUser(int userId) {

  println("start pose detection");
  kinect.startPoseDetection("Psi", userId);

}

void onEndCalibration(int userId, boolean successful) {

  if (successful) {
    println(" User calibrated !!!");
    kinect.startTrackingSkeleton(userId);
  }
  else {
    println(" Failed to calibrate user !!!");
    kinect.startPoseDetection("Psi", userId);
  }

}

void onStartPose(String pose, int userId) {

  println("Started pose for user");
  kinect.stopPoseDetection(userId);
  kinect.requestCalibrationSkeleton(userId, true);

}

///GREG's Code //http://urbanhonking.com/ideasfordozens/2012/01/14/machine-pareidolia-hello-little-fella-meets-facetracker/

import oscP5.*;

OscP5 oscP5;

PVector posePosition = new PVector();

boolean found;

float eyeLeftHeight;

float eyeRightHeight;

float mouthHeight;

float mouthWidth;

float nostrilHeight;

float leftEyebrowHeight;

float rightEyebrowHeight;

float poseOrientationX;

float poseOrientationY;

float poseOrientationZ;

float poseScale;

PImage myImg;

void setup() {

  size(640, 480);
  //frameRate(30);
  oscP5 = new OscP5(this, 8338);
  oscP5.plug(this, "mouthWidthReceived", "/gesture/mouth/width");
  oscP5.plug(this, "mouthHeightReceived", "/gesture/mouth/height");
  oscP5.plug(this, "eyebrowLeftReceived", "/gesture/eyebrow/left");
  oscP5.plug(this, "eyebrowRightReceived", "/gesture/eyebrow/right");
  oscP5.plug(this, "eyeLeftReceived", "/gesture/eye/left");
  oscP5.plug(this, "eyeRightReceived", "/gesture/eye/right");
  oscP5.plug(this, "jawReceived", "/gesture/jaw");
  oscP5.plug(this, "nostrilsReceived", "/gesture/nostrils");
  oscP5.plug(this, "found", "/found");
  oscP5.plug(this, "poseOrientation", "/pose/orientation");
  oscP5.plug(this, "posePosition", "/pose/position");
  oscP5.plug(this, "poseScale", "/pose/scale");
  myImg = loadImage("INSERT YOUR IMAGE HERE.jpg");
  imageMode(CENTER) ;

}

void draw() {

  background(255);
  stroke(0);

    translate(posePosition.x, posePosition.y);
       scale(poseScale*.4);
      image(myImg,0,-25);
       scale(poseScale);
    noFill();

    ellipse(-20, eyeLeftHeight * -9, 20, 7);
    ellipse(20, eyeRightHeight * -9, 20, 7);
    ellipse(0, 20, mouthWidth* 3, mouthHeight * 3);
    ellipse(-5, -2, nostrilHeight,nostrilHeight);
    ellipse(5, -2 , nostrilHeight, nostrilHeight);
    rectMode(CENTER);
    fill(0);
    rect(-20, leftEyebrowHeight * -5, 20, 3);
    rect(20, rightEyebrowHeight * -5, 20, 3);

}

public void mouthWidthReceived(float w) {

  println("mouth Width: " + w);
  mouthWidth = w;

}

public void mouthHeightReceived(float h) {

  println("mouth height: " + h);
  mouthHeight = h;

}

public void eyebrowLeftReceived(float h) {

  println("eyebrow left: " + h);
  leftEyebrowHeight = h;

}

public void eyebrowRightReceived(float h) {

  println("eyebrow right: " + h);
  rightEyebrowHeight = h;

}

public void eyeLeftReceived(float h) {

  println("eye left: " + h);
  eyeLeftHeight = h;

}

public void eyeRightReceived(float h) {

  println("eye right: " + h);
  eyeRightHeight = h;

}

public void jawReceived(float h) {

  println("jaw: " + h);

}

public void nostrilsReceived(float h) {

  println("nostrils: " + h);
  nostrilHeight = h;

}

public void found(int i) {

  println("found: " + i); // 1 == found, 0 == not found
  found = i == 1;

}

public void posePosition(float x, float y) {

  println("pose position\tX: " + x + " Y: " + y );
  posePosition = new PVector(x, y);

}

public void poseScale(float s) {

  println("scale: " + s);
  poseScale = s;

}

public void poseOrientation(float x, float y, float z) {

  println("pose orientation\tX: " + x + " Y: " + y + " Z: " + z);
  poseOrientationX = x;
  poseOrientationY = y;
  poseOrientationZ = z;

}

void oscEvent(OscMessage theOscMessage) {

  if (theOscMessage.isPlugged()==false) {
    println("UNPLUGGED: " + theOscMessage);
  }

}

Listen for single UDP packet video in Processing

import com.sun.image.codec.jpeg.JPEGCodec; import com.sun.image.codec.jpeg.JPEGImageDecoder; import java.net.DatagramPacket; import java.net.DatagramSocket; import java.net.InetAddress;

int packetSize = 1000; UDPClientThread udpComm; PImage incoming;

void setup() {

  size(640, 480);
  try {
    InetAddress thisIp =
      InetAddress.getLocalHost();
    System.out.println("IP:"+thisIp.getHostAddress());

    InetAddress serverIP = InetAddress.getByName("localhost");
    int serverPort = 12345;
    udpComm = new UDPClientThread(serverIP, serverPort, packetSize);
    udpComm.start();// calls the run method to get this thread rolling
  }
  catch (UnknownHostException e) {
    e.printStackTrace();
  }

}

//callback public void gotInput(String _name, byte[] _input) {

  ByteArrayInputStream in = new ByteArrayInputStream((byte[]) _input);
  incoming= null;
  try {
    JPEGImageDecoder decoder = JPEGCodec.createJPEGDecoder(in);
    incoming = new PImage(decoder.decodeAsBufferedImage());
  } 
  catch (IOException e) {
    System.out.println("I 0 Problem");
  }

}

void draw() {

  background(255); 
  if (incoming != null) image(incoming, 40, 40);

}

///////LISTEN///////////////

public class UDPClientThread extends Thread {

  DatagramSocket socket = null;
  int serverPort;
  InetAddress serverIP;
  byte[] inbuf  ;

  public UDPClientThread( InetAddress _serverIP, int _serverPort, int packetSize)

  {

    serverIP = _serverIP;
    serverPort = _serverPort;
    inbuf = new byte[packetSize];
    System.out.println("Create new socket" + serverIP + " " + serverPort);
    try {
      socket = new DatagramSocket(serverPort);	//use any available local port
    }
    catch (IOException e) {
      System.out.println("Error: " + e);
    }
  }

  public void run()

  {

    DatagramPacket other = null;
    System.out.println("Started Listening" );
    while (true)
    {
      if (socket != null)
      {
        try {
          Thread.sleep(10);
        } 
        catch(InterruptedException e) {
        }
        //System.out.println("Waiting for more" );
        try {
          other = new DatagramPacket(inbuf, inbuf.length);
          socket.receive(other);//hangs and waits
          String uniqueName = other.getSocketAddress() + ":" +  other.getPort();
          gotInput(uniqueName, inbuf);
          //System.out.println("got");
        }
        catch (IOException e) {
          System.out.println("error listening" + e);
        }
      }
    }
  }

}

Send single video packet via UDP from Android phone

import android.app.Activity; import android.content.ContentValues; import android.content.Context; import android.content.res.Configuration; import android.hardware.Camera; import android.net.DhcpInfo; import android.net.Uri; import android.net.wifi.WifiManager; import android.os.Bundle; import android.provider.MediaStore.Images.Media; import android.util.Log; import android.view.SurfaceHolder; import android.view.SurfaceView; import android.view.View; import android.view.View.OnClickListener; import android.widget.Toast;

int server_port ; DatagramSocket s ; InetAddress address ; int border = 10; int videoW = 176; int videoH = 144; int packetSize = 1000;

CameraSurfaceView gCamSurfView;

// This is the physical image drawn on the screen representing the camera:

PImage gBuffer;

void setup() {

  size(screenWidth, screenHeight);
  server_port = 12345;
  try {
    s = new DatagramSocket();
    address = InetAddress.getByName("128.122.151.100");
  } 
  catch (Exception e) {
    e.printStackTrace();
  }

}

void draw() {

  // nuttin'... onPreviewFrame below handles all the drawing.

}

void onResume() {

  super.onResume();
  println("onResume()!");
  // Sete orientation here, before Processing really starts, or it can get angry:
  orientation(LANDSCAPE);
  // Create our 'CameraSurfaceView' objects, that works the magic:
  gCamSurfView = new CameraSurfaceView(this.getApplicationContext());

}

class CameraSurfaceView extends SurfaceView implements SurfaceHolder.Callback, Camera.PreviewCallback {

  // Object that accesses the camera, and updates our image data
  // Using ideas pulled from 'Android Wireless Application Development', page 340

  SurfaceHolder mHolder;
  Camera camera = null;

  Camera.Size prevSize;

  // SurfaceView Constructor:  : ---------------------------------------------------
  CameraSurfaceView(Context context) {
    super(context);
    mHolder = getSurfaceHolder();
    //mHolder.setType(SurfaceHolder.SURFACE_TYPE_PUSH_BUFFERS);
    mHolder.addCallback(this);
  }

  // SurfaceHolder.Callback stuff: ------------------------------------------------------
  public void surfaceCreated (SurfaceHolder holder) {
    camera = Camera.open();

    try {
      camera.setPreviewDisplay(holder);
      Camera.Parameters parameters = camera.getParameters();
      parameters.setPreviewSize(videoW, videoH); // data set: w:533, h:320
      Size size = parameters.getPreviewSize(); 
      camera.setPreviewCallback(this);
      camera.setParameters(parameters);
    } 
    catch (IOException exception) {
      camera.release();
    }
  }  

  public void surfaceChanged(SurfaceHolder holder, int format, int w, int h) {
    camera.startPreview();
  }

  public void surfaceDestroyed (SurfaceHolder holder) {
    camera.stopPreview();
    camera.release();
  }

  //  Camera.PreviewCallback stuff: ------------------------------------------------------
  public   void onPreviewFrame(byte[] data, Camera cam) {

    ///COMPRESS IMAGE
    YuvImage yuvimage = new YuvImage(data, ImageFormat.NV21, cam.getParameters().getPreviewSize().width, cam.getParameters().getPreviewSize().height, null);
    ByteArrayOutputStream baos = new ByteArrayOutputStream();
    yuvimage.compressToJpeg(new Rect(border, border, cam.getParameters().getPreviewSize().width-2*border, cam.getParameters().getPreviewSize().height-2*border), 50, baos);
    byte[] output = baos.toByteArray();

    //Make sure it will fit in the packet, change the border to make it fit
    int msg_length= output.length;
    if (msg_length > packetSize-20) border = Math.min(videoH/2+4, border +1);
    else if (msg_length < packetSize-50) border = Math.max(1, border-1);
    //println("SIZe", "B:"+ border + " L:"+ msg_length);
    //SEND IMAGE
    if (msg_length <= packetSize) {
      DatagramPacket p = new DatagramPacket(output, msg_length, address, server_port);
      try {
        s.send(p);
      } 
      catch (IOException e) {
        println("Sending problem " + System.currentTimeMillis());
      }
    }

  }

}