Introduction To Computational Media (ICM) : Week 9
Loose Ends
Video Tracking
One interesting thing that can be done with video capture is to use it as a means to track obects through space.
Here is an example (courtesy of DanO):
import processing.video.*;
Capture video;
float targetRed = 255.0; //set some numbers for the target you are chasing
float targetGreen = 0.0;
float targetBlue = 0.0;
void setup()
{
video = new Capture(this, 320, 240, 15); //initiate the video, resolution and frame rate
size(320, 240); //give you Processing window a size
}
void captureEvent(Capture camera)
{
camera.read();
}
void draw()
{
float worldRecord = 1000.0; //intialize the worldrecord
int xFound = 0; // initialize the location of the red tracking ball
int yFound = 0;
for(int j=0; j < video.height; j=j+1) { //for each row
for(int i=0; i < video.width; i=i+1) { //for each column
//get the color of this pixels
//find pixel in linear array using formula: pos = row*rowWidth+column
color pix = video.pixels[j*video.width+i];
//find the difference
float diff = abs(targetRed - red(pix)) + abs(targetGreen - green(pix)) + abs(targetBlue - blue(pix));
if (diff < worldRecord){ // if this is closest to our target color
worldRecord = diff;
yFound = j; //mark the spot for drawing it later
xFound = i;
}
}
}
image(video,0,0); //draw the video, this might be optional
//after all the pixels have been tested, draw the winner
fill(255,0,0);
ellipse(xFound, yFound, 10, 10);
}
void mousePressed(){
//allow the target color to be changed
color pix = video.pixels[mouseY*video.width+mouseX];
targetRed = red(pix); //get the color of the pixel they clicked on
targetGreen = green(pix);
targetBlue = blue(pix);
}
More examples
Pulling back the curtain
Processing is really Java
In fact the code you write in Processing, goes through a minor conversion to straight Java before it is compiled.
Here is a very simple Processing example:
int x = 0;
int y = 0;
int xDirection = 1;
int yDirection = 1;
void setup()
{
size(500,500);
}
void draw()
{
background(255);
if (x > width || x < 0)
{
xDirection = xDirection * -1;
}
x += xDirection;
if (y > height || y < 0)
{
yDirection = yDirection * -1;
}
y += yDirection;
ellipseMode(CENTER);
ellipse(x,y,50,50);
}
To illustrate what this looks like after Processing converts it to straight Java, we can Export and take a look at the .java file that is created.
Here is the .java file for the above code.
Not much is different!
Processing added a bunch of import statements automatically which tells the Java Compiler and the Java Virtual Machine what is needed in the way of external libraries and from the Java API.
More interesting is that it wrapped all of our code inside a class:
public class SimpleEllipse extends PApplet
{
// Our code...
}
Low and behold, everything is in a class. This is because, Java is an Object Orientated language. Almost everything in Java is a class and correspondingly your Processing code gets instantiated as an Object.
Let's deconstruct the line: public class SimpleEllipse extends PApplet {
The name of the class, SimpleEllipse is what I saved the example as. The rest of the items on that line are as follows:
public: This tells Java who has access to this class in code. public means that other Java programs can encorporate and use this class and make objects of it.
extends PApplet: This tells Java that our class, SimpleEllipse is a child of PApplet. PApplet is the class that allows us to use all of the Processing methods, variables and constructs that we are used to, such as size(), mousePressed(), mouseX and so on. More about this later...
Java Mode
We can actually write our code in Processing in straight Java by simply including the same type of class definition that Processing does for us. This allows us to use the full Java API by simply using the imports that are required for each class that we want to use.
We don't need to include the ones that Processing does for us:
import processing.core.*;
import java.applet.*;
import java.awt.*;
import java.awt.image.*;
import java.awt.event.*;
import java.io.*;
import java.net.*;
import java.text.*;
import java.util.*;
import java.util.zip.*;
Of course, the J2SE 1.4 Javadocs are our best friend when working like this.
All of the possible classes we can use (without resorting to external libraries) are listed in the frame to the left.
Clicking on the name of a class, such as ArrayList will bring up the documentation for that particular class. It will show you on the very top what the import statement will need to include. (For ArrayList we don't need to import anything as Processing already does it for us.) The other items that it shows us are what it is a child of (extends), the constructors and the methods that are available with it.
Using Java Classes
ArrayList happens to be a nice one for us to learn. It basically allows us to create Arrays with the ability to grow and shrink.
Here is an example (using our trusty Ball class from earlier in the year):
import java.util.*; // We don't actually need to do this..
public class ArrayListExample extends PApplet
{
//declaring a global variable of type ArrayList
ArrayList a;
void setup()
{
size(400,300);
colorMode(RGB,255,255,255,100);
// Creating an instance of ArrayList and storing it in our global variable
a = new ArrayList();
}
void draw()
{
background(0);
if (mousePressed)
{
// Create a new ball
Ball newBall = new Ball(width,height,10,mouseX,mouseY);
// Add the ball to the ArrayList
a.add(newBall);
}
// Iterate through the ArraList and get each Ball
for (int i = 0; i < a.size(); i++)
{
// when something comes out of the arraylist via "get" we have to
// remind ourselves what type it is. In this case, it's an instance of "Ball"
Ball b = (Ball) a.get(i);
b.compute();
b.display();
}
}
/*
We are including our Ball class here instead of in a different tab as when we enter "Java mode" we loose the default imports and can't use normal methods like ellipse() and ellipseMode(). By including it inside the main class we get those back.
*/
class Ball
{
// our Class variables.
int wWidth;
int wHeight;
int bSize;
int x, y;
int xDirection, yDirection;
// Constructor. This gets called when we create a "new Ball"
Ball(int windowWidth, int windowHeight, int ballSize)
{
wWidth = windowWidth;
wHeight = windowHeight;
bSize = ballSize;
x = 1;
y = 1;
xDirection = 1;
yDirection = 1;
}
// This is an Overloaded constructor
Ball(int windowWidth, int windowHeight, int ballSize, int xPosition, int yPosition)
{
this(windowWidth, windowHeight, ballSize);
x = xPosition;
y = yPosition;
}
// Our method to determine current position of the ball.
void compute()
{
if (x < wWidth && x > 0)
{
// Move along x axis
x += xDirection;
}
else
{
// Change direction, from positive to negative and vice versa
xDirection = xDirection * -1;
x += xDirection;
}
if (y < wHeight && y > 0)
{
y += yDirection;
}
else
{
yDirection = yDirection * -1;
y += yDirection;
}
}
// This actually displays our ball, it gets called in the main draw function
void display()
{
compute(); // first we run the computation
ellipseMode(CENTER);
ellipse(x,y,bSize,bSize);
}
} // Close our Ball class
} // Close our PApplet derived class
You will notice that this program starts to slow down as we keep adding Ball's to our ArrayList. We can rectify this by removing Balls from the ArrayList as well by using the ArrayList remove() method.
if (mousePressed)
{
// Create a new ball
Ball newBall = new Ball(width,height,10,mouseX,mouseY);
// Add the ball to the ArrayList
a.add(newBall);
// If we have 100 balls or more, remove the first Ball
if (a.size() > 100)
{
a.remove(0);
}
}
Here it is running
Another useful Java class is Rectangle. It provides us with some more functionality than the rect() method that is built into Processing because it is an class/object.
Here are is the Javadoc:
http://java.sun.com/j2se/1.4.2/docs/api/java/awt/Rectangle.html. You should notice, that it has a nice contains method built in.
Here is an example
Other examples that might interest you:
DanO's Google Parsing with ArrayList example
DanO's TextField GUI example
DanO's Rectangle Shoot'em example
Handling Errors
By now you have all probably run into code that might give an error when running it but might not at other times. The loadStrings method for instance will throw an error if a network connection is unavailable or if the server being called is down.
Fortunately, Java has a means to deal with this and allows you to make your program still function even if there is an error.
It is called, try, catch. You simply enclose the error prone code in a try block and if an error is thrown your catch block will be called. In the catch block you need to include the type of Exception that is thrown. The easiest thing to do is to catch Exception itself.
void setup()
{
String[] htmlSource;
try
{
htmlSource = loadStrings("http://blablah blah.com");
}
catch (Exception e)
{
htmlSource = new String[1];
htmlSource[0] = "Couldn't Load Page";
}
println(htmlSource);
}
Not quite there
Even though we are writing straight Java in Processing, we are still using the Processing libraries which are nice. If we had to do all of this from scratch it would look like the Java code on this page:
https://itp.nyu.edu/~sve204/cgi-bin/wiki/wiki.pl?EmulatingProcessing
Compiling Java by hand isn't terribly difficult but you do need to have a JDK installed and your environment setup correctly.
Here is how to compile the Java code in the example above:
Create a text file and copy in the code.
Save it with the name of the class followed by ".java"
Open up a terminal or command line program, change directories to the directory containing the .java file and run the following commands (Mac or Windows):
cd /to/directory/containing/file/
javac EmulateProcessing.java
java EmulateProcessing
"javac" compiles the .java file, "java" runs it.
Books and Tutorials
Java is a big (and flexible) language and there are some very good tutorials and books available.
Here is a list of some that I like http://stage.itp.nyu.edu/~sve204/cgi-bin/wiki/wiki.pl?JavaTutorials
DanO recommends, Head First Java.