• View
• Print
• Recent Changes
• Login

Intro to Computational Media, Week 6

Modified from Danny Rozin's ICM notes

Libraries

• Libraries are helper files that include additional functionality that the creators did not include in the core functionality
• In Processing libraries are objects. (classes)
• In Processing we have two kinds of libraries
  • Built in libraries
  • Contributed libraries

Some Built in Libraries

• Video - capture , playback, record digital video.
• Serial - Send and receive serial communication.
• OpenGL - Use Open GL acceleration of graphics.
• Network - Client /Server communication over IP.
• PDF - Export your sketch in pdf format.
• XML - Import XML from web.

To use a built -in library, You must first import it into your sketch by using :

	Sketch -> Import Library-> Library name

• This will add a line of code in the top of your sketch:

                import processing.libraryName.*;

As libraries are classes, we need to have a variable of that type to hold :

		Serial ourSerial;

• Libraries are classes, we will have to call new before using, and sometimes pass arguments(see reference for each library)

 		ourSerial = new Serial(this, 9600);

• The rest of the functionality will be addressed with dot notation:

		x = ourSerial.read();

Contributed libraries are written by 3rd party people and are not included in the Processing installation.

You can still find them in the Libraries section of processing.org

Some useful libraries include : ESS and Sonia for sound, proXML for networking, JMyron for video.

To add a library to your sketch:

• Download the library from the processing.org
• unZip the files.
• Place the files into the “libraries” folder inside your “processing” folder.
• Use Sketch -> Import Library-> Library name
• Use like the built-in libraries.

Serial Communication

Serial communication is the most common way computers communicate with external devices

Serial communication is sometimes used inside other protocols such as USB (Universal Serial Bus), Fire Wire etc.

• Most new computers do not come with a serial port, so you need to get a USB-Serial adaptor
• The arduino has a built in serial to USB adaptor

Asynchronous serial communication is the lowest level of communication, the data is turned into 1 and 0 and sent over a wire to the other device as pulses of electricity.

In order for asynchronous communications to work between two devices, they must agree on a few things :

• Port, if there are a few ports, which one to use. These are typically named COM1, COM2 on a PC, and named something like /dev/tty/someDevice on a Mac or Linux.
• Speed, The rate in which the pulses will be sent, these are measured in bps (bits per second) and usually default to 9600, other common rates are 38,400, 115,200, this is sometime called baud rate.

Finding your serial Port

Even before you initialize your serial object with new, you can use the class to get a list of available ports on your computer:

	ourPorts=	Serial.list(); 

• Note that we are using the name of the class, not our object. This is the first time we see this type of access.
• This will return an array of strings with the names of the ports.

To open the port and initialize the object you call new

	ourSerial = new Serial(this, port, speed);

• this. a special word that tells the library who we (our sketch) are. This will be used if the library has to call us back (callBack)
• port. usualy COM1 or whatever came back from Serial.list();
• Speed. 9600 or whatever you set on the arduino side.
  ourSerial = new Serial(this, Serial.list()[0] , 9600);

Writing

Just like in files, writing is easier...

		Serial.write(data);

• data can be an int, string, array. Really very flexible.
  string stuffToSend = “hello”; Serial ourSerial = new Serial(this, Serial.list()[0]); ourSerial.write(stuffToSend );

Reading

• Anything that comes into the port is stored inside the buffer
• The way the buffer is handled if FIFO (First In First Out)
• Serial is just 1 and 0 so it is up to us to decide if to evaluate the stream as *ints (numbers) or string (characters) . If you can send numbers that are 0-255 *then using ints is the best, if it has to be big numbers, strings divided by commas are usually used

Many options for reading the incoming stream:

• read() returns a number 0-255 of the first byte in the buffer. and takes it out of the buffer.
• readString() returns the whole buffer as a string and empties the buffer;
• readChar() - returns a single char, the first byte in the buffer. and takes it out of the buffer.

A few more, look up in reference

Before you read the buffer it is good to see if there is anything waiting there for you:

• available() returns the number of bytes waiting

		while (ourSerial.available() > 0) {    
			int inByte = ourSerial.read();
			println(inByte); 
		}

If you don’t want to constantly call available() to see if there is something waiting for you, you can use the serialEvent() callBack function.

void serialEvent (Serial P){} will be called whenever enough bytes are waiting in the buffer

	Serial ourSerial = new Serial(this, Serial.list()[0]);
	ourSerial .buffer (3);

	void serialEvent(Serial p) {   
		String inString = (ourSerial.readString());
		println(inString );
	}

VIDEO

Most computers come with cameras or with ports that allow easy connection to a camera (FireWire, USB).

The video library uses QuickTime on all platforms. On Mac QuickTime is native- on PC you need to install winVDIG from http://www.shiffman.net/vdig/WinVDIG_105.exe

WinVDIG is notoriously difficult to work with. Because Processing uses quicktime for its movie Library, PC users often have trouble with this library.

• Try the JMyron video library if the processing library is too irritating.

To open the camera and initialize the object you call new:

	ourCapture = new Capture(this, width, height, FPS);

• this. a special word that tells the library who we (our sketch) are. This will be used if the library has to call us back (callBack)
• width , height - size of the video frame, try to use a 4/3 ratio to avoid stretching . 640 /480 , 320/240, 160/120 are popular sizes
• FPS how many frames per second (30 max for most video cameras).
  ourCapture = new Capture(this, 640,480 , 30);

The capture object is actually a PImage object, so you can display it and access it like any image. But you have to make sure it is refreshed with new pixel values to keep the video moving. To do that you use read():

		ourCapture = new Capture(this, 320, 240, 30);
		void draw(){
			ourCapture.read();
			image(ourCapture,0,0)
	}

In order not to waste too much time reading frames when the camera does not have a new frame ready yet, you can use available()

		ourCapture = new Capture(this, 320, 240, 30);
		void draw(){
			if (ourCapture.available()){
				ourCapture.read();
				image(ourCapture,0,0);
			}
		}

A different approach to knowing when a frame is ready is using the callback captureEvent()

		ourCapture = new Capture(this, 320, 240, 30); //in setup

	void draw(){
			image(ourCapture,0,0);
	}

	void captureEvent(Capture c) {
	ourCapture.read();  
	}