Introduction To Computational Media (ICM) : Week 6
External Libraries
Processing offers us the ability to use external libraries (created in Java) that are crafted in a special manner.
A couple of interesting and useful ones are:
Video: http://www.processing.org/reference/libraries/video/index.html video capture and playback.
Serial: http://www.processing.org/reference/libraries/serial/index.html reading and writing data through the serial port.
Net: http://www.processing.org/reference/libraries/net/index.html reading and writing data over the network.
OpenGL: http://www.processing.org/reference/libraries/opengl/index.html highly optimized graphics API.
Sonia: http://sonia.pitaru.com/ advanced audio capabilities.
More at: http://www.processing.org/reference/libraries/index.html
To utilize an external library, you first need to install that library in Processing.
The library should be placed in the Processing Application's "libraries" folder.
To use the library in your Processing application, you need to create an "import" statement which specifies the library name including any "packages" that library is in. (The easy way to do this is through Processing's "Import Library" command under the "Sketch" menu.
The import statement for the video library:
import processing.video.*;
The * means import all of the Classes defined within that Library.
Gimme Eyes
Video Capture
Processing has an external library for capturing video from live cameras (firewire, composite or s-video and DV, basically anythiing that QuickTime supports).
(If you are working on a pc you may need this additional driver for your camera: http://www.vdig.com/WinVDIG/. You may also need to download QuickTime http://www.apple.com/quicktime/download/win.html and if you have a choice, do the "Custom" install and select "QuickTime for Java" support.)
Here is an example of displaying the video from a camera:
// Import the Library
import processing.video.*;
// Create an Object of type Capture
Capture myCapture;
void setup()
{
size(320, 240);
// List the available video capture devices
println(Capture.list());
// Initialize the capture object, give it "this", the width you would like, the height and the framerate.
//myCapture = new Capture(this, width, height, 30);
// Another version of the capture object initialization, this time specifying the capture device
myCapture = new Capture(this, width, height, "IIDC FireWire Video", 30);
}
// A function that get's called when a new frame is available
void captureEvent(Capture myCapture)
{
// Read the captured frame
myCapture.read();
}
void draw()
{
// Draw the captured frame as an image
image(myCapture, 0, 0);
}
As you can see, capturing video is taken care of by the "Capture" class/object and drawing it to screen is done the same way images are drawn to screen.
Filtering/Tinting Images/Video
Processing has a couple of built in functions for filtering and tinting images/video. Here is an example (click the mouse to see the different effects on the video).
// Import the Library
import processing.video.*;
// Create an Object of type Capture
Capture myCapture;
int clickCounter = 0;
int currentMode = 0;
int numModes = 8;
void setup()
{
size(320, 240);
// List the available video capture devices
println(Capture.list());
// Initialize the capture object, give it "this", the width you would like, the height and the framerate.
//myCapture = new Capture(this, width, height, 30);
// Another version of the capture object initialization, this time specifying the capture device
myCapture = new Capture(this, width, height, "IIDC FireWire Video", 1);
}
void draw()
{
if (myCapture.available())
{
myCapture.read();
// Check the currentMode variable and do the appropriate effect
if (currentMode == 1)
{
println("threshold");
myCapture.filter(THRESHOLD); // Add the threshold filter to the image
}
else if (currentMode == 2)
{
println("blue");
myCapture.filter(BLUR); // Add the blur filter to the image
}
else if (currentMode == 3)
{
println("invert");
myCapture.filter(INVERT); // Add the invert filter to the image
}
else if (currentMode == 4)
{
println("posterize");
myCapture.filter(POSTERIZE,3); // Add the posterize filter to the image
}
else if (currentMode == 5)
{
println("gray");
myCapture.filter(GRAY);
}
else if (currentMode == 6)
{
println("opaque");
myCapture.filter(OPAQUE);
}
myCapture.updatePixels();
}
if (currentMode == 7)
{
color c_blue = color(0,0,255);
tint(c_blue); // Call the tint filter (won't see until the next time through)
}
else
{
noTint();
}
// Draw the captured frame as an image
image(myCapture, 0, 0);
}
void mousePressed()
{
clickCounter++;
currentMode = clickCounter % numModes;
println(currentMode);
}
Breakfast Food
Serial Input and Output
By now, you should have learned a bit about Serial from Arduino to the PC in Physical Computing. If not, you will shortly ;-).
Tom Igoe has put together some great pages dedicated to Serial:
Serial to the Desktop
More on Serial Communication: interpreting the bytes
DanO also has several examples:
ICMSerial
Here is the very bare-bones serial reading code from Learning Processing.
We will be using slightly different code than what Learning Processing and DanO provide but the concepts are the same.
// Original Example by Dan O'Sullivan - Heavy mods by Shawn V.
import processing.serial.*; // Import the serial library
Serial port; // The serial port
int bgColor; // Background color
int shape;
int shapeColor; // Fill color
int xpos, ypos; // position of the shape
void setup()
{
size(256, 256);
noStroke(); // No border on the next thing drawn
// Print a list of the serial ports, for debugging purposes to find out what your ports are called:
println(Serial.list());
port = new Serial(this, Serial.list()[2], 9600); //you can pull the name out of the list
//port = new Serial(this, "COM11", 9600); //or you can just specify it
println("Serial Should Be Ready!");
bgColor = 0;
shapeColor = 255;
xpos = 0;
ypos = 0;
}
void draw()
{
background(bgColor);
fill(shapeColor,0,0);
// Draw the shape, either an ellipse or a rectangle
if (shape == 0)
{
ellipse(xpos, ypos, 20, 20);
}
else
{
rect(xpos, ypos, 20, 20);
}
}
void serialEvent(Serial port)
{
/* Read Byte Example // byte is a value between 0 and 255, it is the most basic way things are transmitted over serial
while(port.available() > 0) // Gives us number of bytes available
{
byte input = port.read(); // Read a single byte
println(input); // Here we would do something with that byte
}
port.write(65); // Tell the board we are ready for the next thing
*/
/* Read Char Example
while(port.available() > 0)
{
char input = port.readChar(); // Reads a single byte interpretted as a character (ASCII)
println(input);
}
port.write('A');
*/
/* Read String Example */
String input = port.readStringUntil(13); // Read from serial as a string until the byte you are looking for
// Requires that you return (Ascii 13) at the end of your transmission
// Returns null if it doesn't find the byte
if (input != null)
{
//println("Raw Input: " + input);
String[] parts = input.split(","); // If you put commas (Ascii 44) between things in your transmission
// Gives us an array, parts that we can use to drive our variables
xpos = int(parts[0]); // The first thing sent is our xpos\
// Map it's value from 0 to 400
// http://www.processing.org/reference/map_.html
xpos = map(xpos,400,625,0,400);
ypos = int(parts[1]); // The second thing sent is our ypos
ypos = scaleIt(height,0,400,625,ypos);
bgColor = int(parts[2]); // The third thing sent is our bgColor
bgColor = scaleIt(255,0,500,515,bgColor);
shapeColor = int(parts[3]); // The fourth thing sent is our shape color
shapeColor = scaleIt(255,0,300,700,shapeColor);
shape = int(parts[4]); // The fifth thing sent is our shape
// Let's see what we got
//println("Parsed and Scaled x:" + xpos + " y:" + ypos + " bgColor:" + bgColor + " shapeColor:" + shapeColor + " shape:" + shape);
port.write(65); // Send a capital A (ascii 65) or any other arbitrary character in case the microcontroller is waiting to hear from you
}
}
The code on our Arduino (See Arduino (Text Style))
Serial Reference:
Serial Libraries - Check this for possible gotchas getting serial to work on your machine.
Learning Processing: Chapter 19: Example 19-9: Handshaking