Introduction To Computational Media (ICM) : Week 9

Text and Networks

Strings and Yarn

Strings are a sequence of characters. Strings are a data type in Processing/Java but rather than being a primitive datatype like char, int or boolean; they are a Object (notice the big O) of type String (Class).

Here is how you declare a String:

        // The shortcut
        String myString = "Hi, My Name is Shawn";

Another way you might do it:

        // The long way
        String myString = new String("Hi, My Name is Shawn");

Processing includes some documentation on the String class but the real documentation can be found on Sun's Java site's JavaDocs: JavaTM 2 Platform, Standard Edition, v 1.4.2 API Specification

Of primary interest in the documentation for the String class are the functions/methods described in the Method Summary. These are all of the methods that the programmers of Java thought to include as part of the String class and therefore are the methods that you can perform on any String object itself.

Here is an example using indexOf(String str):

        String aString = "Nothing much to say, but here is some text that I need to show this example in class.  Hi Everybody!!!!";                  
        // Search through aString for the String "class" and save it's index in the variable found                  
        int found = aString.indexOf("class");
        println(found);

There are quite a few useful methods that you should be aware of. Here is an abreviated list: charAt(index), equals(Object anObject), length(), substring(index1,index2), toLowerCase(), toUpperCase(). All of these are also detailed in the Processing reference.

Control Characters

Strings can contain control characters such as:

newline: "\n"

tab: "\t"
as well as escape characters for things like double quotes (""). The escape character is the backslash ("\") so to put a double quote in a string you would write it as follows:

        String myTestString = "I am quoting Shakespeare or Hamlet\n\twhen I say \"To be or not to be, that is the question.\"";
        println(myTestString);

which outputs:

        I am quoting Shakespeare or Hamlet
	       when I say "To be or not to be, that is the question."

Splitting

Processing has another built-in function that operate on Strings: split(). While splitting can be accomplished through the Java String class methods it is generally easier to do through Processing.

Example (shamelessly stolen from Dan Shiffman):

        String spaceswords = "The quick brown fox jumps over the lazy dog.";
        String list1[] = split(spaceswords);
        println(list1[0]);
        println(list1[1]);
        
        String commaswords = "The,quick,brown,fox,jumps,over,the,lazy,dog.";
        String list2[] = split(commaswords, ',');
        for (int i = 0; i < list2.length; i++) {
          println(list2[i] + " " + i);
        }
        
        //calculate sum of a list of numbers in a string
        String numbers = "8,67,5,309";
        int list[] = int(split(numbers, ',')); // Converting the String array to an int array
        int sum = 0;
        for (int i = 0; i < list.length; i++) {
          sum = sum + list[i];
        }
        println(sum);

The split() funtion either splits a String on the space character as shown in the first example or takes in a char on which to split as in the second example (using a comma ','). The third example shows splitting on a comma and subsequently converting the array of strings to an array of integers. As shown in the above examples the split function returns an array of Strings.

http:// and Web Access

Processing has some very interesting Network and File related utilities

loadStrings() can take in the name of a File as a parameter or a URL. It returns an array of Strings containing the contents of that file or URL. (Note: This will be the raw source of a document or URL, containing all of the HTML and other data contained within.)

Here is a quick example that loads the contents of this page:

     
            String lines[] = loadStrings("http://itp.nyu.edu/~sve204/icm/week9.html"); 
            println("there are " + lines.length + " lines"); 
            for (int i=0; i < lines.length; i++) { 
              println(lines[i]); 
            }

Running this, you will see all of the HTML source for this page output in the Processing console but of course you could do something more fancy like: Dan Shiffman's loadStrings() Example

Query String

This is all well and good but what about pulling Dynamic data off of a website?

Here is a URL: http://stage.itp.nyu.edu/ICM/shiffman/week9/

Here is another URL: http://itp.nyu.edu/~dbo3/cgi-bin/ClassWiki.cgi?ICM_Net

Notice the difference? The second one has a question mark followed by some text at the end. This is called a query string. URL's that have query strings are actually executing an application on the server side that takes in variables and data through it.

The standard format of a query string is as follows

?first_variable=first_value&second_variable=second_value&third_variable=third_value and so on..

An HTML form often is used to pass the user supplied information to a program on the server but in many cases all that the form is doing is placing the filled in values on the query string for processing.

Here is an example form that does this:
Form GET method
Notice that after you fill out the form and click submit that the variables are place on the Query String of the URL.

Using this same method, we can do Google searchs or in essence submit data to any webpage (that allows form input through a "GET") by adding a query string to the URL.

Enter Processing:

Since we can load data off of the network using the loadStrings() method with a URL, we can also pass data over the network using a URL with a query string.

Here is the URL for Yahoo's RSS weather feed for my zip code: http://xml.weather.yahoo.com/forecastrss?p=11201. Notice at the end of the url our magic "?" followed by "p=11201", this is our query string, change the zip code to any other and you will get the weather for that area instead.

Scraping

I choose an RSS feed of Yahoo's weather data for good reason. HTML is messy, difficult to work with and has been corrupted by designers to have purely visual design elements included. I have one word for this YUCK!.

Fortunately, XML and RSS (a type of XML) have entered the scene as of late. Instead of being mucked up with elements concerning visual design, these formats are meant for machine readability instead of human readability.

Here is the source from the above URL.

You might notice that everything is tagged with what it actually is. For instance, there is a tag as follows:

            <yweather:condition text="Sunny/Wind" code="34" temp="63" date="Thu, 03 Nov 2005 12:51 pm EST" />

Perfect! Now all we have to do is be able to parse this in Processing and we can make a project that reacts to the current temperature anywhere on earth ;-)

Thanks to Dan Shiffman (once again) here is an example

Here is another way to do it that uses the ProXML library for Processing.

Security Restrictions

One last note.. Unfortunately (for us), Processing (Java) has serious restrictions on network access. One of these restrictions is that you can not call a function such as loadStrings() on a URL that isn't on the same server that the applet is served from.

Fortunately (for us), there is a way around this using a Proxy

On the ITP server, we have installed a script written in PHP which redirects any URL requests through it to aviod this security restriction.

To use the script in your Processing applet that uses loadStrings, change the url that you put in loadStrings from: http://www.example.com/some/path to http://itp.nyu.edu/icm/proxy/proxy.php?url=http://www.example.com/some/path

To check out the script or to customize it for your own use, checkout my wiki page that documents it.