Apparently, I have learned absolutely nothing all semester, because what seemed like a very straightforward project proved to be completely beyond my abilities.

The overarching goal is to generate data for the visualization I’m making for Lisa Strausfeld and Christian Marc Schmidt’s Mainstreaming Information class. Click the image above to see some slides (PDF, 128 KB) explaining the gist of the project, provisionally called Vocabulap (vocabulary + overlap; not a handsome coinage). My specific goals for the A2Z midterm were as follows (with subsequent comments in all caps):

For A2Z midterm
===============
Prep
—-
* Remove all blank lines
DONE
* Remove all extra spaces
DONE
* Break all lines – DONE
* Rename all to number consecutively: A01, A02, . . . A10 (for old books); B01, B02, . . . B10 (for new books)
DONE

Compare major sets
——————
* Extract the text from between the body tags in each file. Dump it out as a new file with the extension body.txt in the folder ../body.
THIS IS HARDER THAN IT LOOKS (FOR ME, AT LEAST). EASIER TO JUST CUT THEM OFF BY HAND.
* Concatenate all the files in each set.
DID THIS FROM THE COMMAND LINE, USING CAT
* Make a list of unique words in each concatenated set, with the number of times the word appears.
CAN GET THE UNIQUE WORDS, BUT NOT THE COUNT.
* Strip out all words beginning with numerals.
DONE BY HAND
* Create the following lists:
– Words shared by both major sets, with frequency counts
– Words unique to set 1, with frequency counts
– words unique to set 2, with frequency counts
I APPARENTLY CANNOT DO ANY OF THIS.

Find unique words in each book
——————————
For each book:
* Concatenate all the files in that major set *except* the file for that book.
* Make a list of the unique words, with frequency counts, in
– the current book
– the set of all books except the current one
* Make three lists:
– Words shared by all books in the major set, with frequency counts
– Words that appear only in the current book, with frequency counts
– Words that appear only outside the current book, with frequency counts

Return lines surrounding specific words
—————————————
For each word in a given list:
* Get the line numbers on which it appears.
For each appearance,
* Print the line above
* Print the line with the word, replacing it with itself wrapped in span tags to apply color
* Print the line below

The most essential piece of code that I could not get working is the comparison doodad. It almost worked for, like, five seconds, but it was generating a huge file of every unique word times however many words were in the document, or something like that. When I tried to fix it, it completely stopped working. The offending code is as follows:

/*  1. Takes in a file name from the command line.
    2. Makes a string array out of the hard-coded comparison file.
    3. Imports the contents of the file whose name was passed in.
    4. For each line of the input file (i.e., each word), changes it to
       lowercase and checks to see if it's contained in the comparison file.
    5. If it's not in the comparison file, checks to see if it's in a hashset of
       unique words.
    6. If the word's not in the hashset, add it.
    7. Print the contents of the hashset.
*/

import java.util.ArrayList;
import java.util.HashSet;
import com.decontextualize.a2z.TextFilter;

public class CompareUnique extends TextFilter
{
    public static void main(String[] args)
    {
        new CompareUnique().run();
    } // end main

    private String filename = "body/unique/allB_uci.txt";
    private HashSet uniqueWords = new HashSet();
    private HashSet lowercaseWords = new HashSet();

    // make a String array out of the contents of the comparison file
    String[] checkAgainst = new TextFilter().collectLines(fromFile(filename));

  public void eachLine(String word)
  {
    String wordLower = word.toLowerCase();
    for (int i = 0; i < checkAgainst.length; i++)
    {
        if (checkAgainst[i] != null && checkAgainst[i].contains(wordLower))
		{} // end if
		else if (checkAgainst != null)
		{
            if (lowercaseWords != null && lowercaseWords.contains(wordLower))
            { } // end if
            else if (lowercaseWords != null)
            {
                uniqueWords.add(wordLower);
                lowercaseWords.add(wordLower);
            } // end else
 		} // end else
    } // end for
  } // end eachLine

  public void end()
  {
    for (String reallyunique: uniqueWords) {
      println(reallyunique);
    } // end for
  } // end end

} // end class

I know, it seems very simple, but you have no idea how long it took me to get this far.

So, basically, for the midterm I’ve got bupkis—just a big pile of text files, and a list of unique words for each.

Above is the PowerPoint slideshow that Diego made for our in-class presentation. There is also copious supporting material at the following locations:

• Photos, with lots of chatty comments
  
• Videos, interspersed with kittenage
• Arduino code
• Processing code
  • colors_highlight_new_swatch – this is the color palette module. It reads a text file containing all the colors in the appropriate Krylon spraypaint line and outputs them as a grid of colored squares. The active swatch and rolled-over ones are highlighted with colored borders. It’s glitchy, as you can see if you look in the upper left corner, but basically it works.
  • spraycans – I would like to make my own set of brushes, based on Diego’s, that are all the same style but in different sizes. These would then be mapped to the number of fingers being held up, with more fingers triggering a bigger spray cone. This would make it easier for users to get predictable, slightly repeatable results. Toward that end, I made a spraycan object whose spray cone, label number (for the size), and can color (for the paint) are variable. Epileptics should probably not view this sketch.
  • Main application – The “final” in “final_application_4″ refers to the this being our final project for the semester, not to the code itself actually being final. For, verily, it is not. This is the wrapper into which all the other modules will eventually be rolled. It’s a whole lot of nested if statements and functions. This program doesn’t run unless the glove is plugged in; I should fix that, to make it easier to check code while programming.

This is the second smidgen of the code for our final project. It pulls RGB values and color names from a tab-delimited text file (which is, itself, based on the actual Krylon color options) and outputs this grid of swatches. The swatches don’t do anything yet—just drawing them took me, like, two days, thank you very much, and that was with some very helpful help from Shawn. Partly this is because I apparently can’t keep in my head for more than thirty seconds how arrays and objects work, and partly it’s because I just. can’t. focus. And partly it’s because I apparently have no idea what the fuck I’m doing.

I’m beginning to really like Diego’s Plan B, as proposed over the weekend:

Fake our own deaths.

I keep telling myself I have to not panic and just write the code one piece at a time. Then, later, I can get somebody to help me put it all together, which is something I know I have no clue how to do. But just accomplishing what seem like they should be simple things is stumping me. Most recently, I was trying to get the swatch name to be displayed when a square is rolled over. This is not by any means an important feature—in fact, it doesn’t really serve any purpose. But I should be able to do it, and I can’t figure out why it doesn’t work. I keep getting errors like “The function showName() does not exist” or “Cannot find anything named ’swatches.’” Both of these things do, of course, exist, and I can’t see any reason why they wouldn’t be found.

I know it’s pretty rare for computers to just fuck up and much more common for me to do so, so I’m sure the answer is that I’ve done something really dumb and sloppy somewhere. But it could easily take me two hours to find that error, and every new goddamn line of code seems to cause some new mysterious issue. I can feel myself growing older just thinking about it. And there are a lot more lines of code to go. Here’s the stuff I was hoping to get done, oh, by, say, last Thursday:

And that’s not including the task of sewing us a new glove, for which I bought fabric on Monday, as well as a book about how to use my virgin four-year-old Juki serger. I know that’ll be fun once I get started, but I just can’t get started.

Maybe I need a drink.

I decided to start my final ICM project with the drawing interface, because it’s a discrete program in itself, and because I could actually sort of picture what it might look like. So I made a list of components—stroke palette, color picker, eraser, eyedropper, transparency picker, etc.—and decided to build them one at a time. At the same time, I joined forces on the Physcomp final with Diego, for his glove project that turned out to be a drawing tool, so I knew the interface might need to change, in order to work with that hardware.

It seemed like it had been weeks since I’d written sketch in Processing, so I started by rereading the chapters on objects and arrays from the book. Then I made the supersimple mouse-tracking line-drawer that we did at the beginning of the semester. And then I added a stroke-width selection palette to work with that. The first version is above. How it works:

The line turns on and off with a mouse click, so when the sketch opens, if you wave your mouse around, you get no line. Then you click once to turn the drawing tool—let’s call it a pencil—on, and you draw, draw, draw. You can click to turn off the pencil at any time, select a different stroke width, and click to start drawing again, from a new mouse position. The stroke that’s currently selected in the palette appears highlighted in green, with the line in black; all the inactive stroke sizes are gray.

What’s wrong with this interface:

1. At program launch, it’s supposed to default to the thinnest line, in black. I forgot to set that up, so instead, if you turn on the pencil without selecting a stroke width first, you get the last line style that was drawn to make the palette, which is the 16-point version in gray. Oops.
2. There’s no indicator showing whether the pencil is on or off. So a couple of times when I thought it was off, I moved my mouse and found that it was on.
3. There’s no undo. If I could undo the previous stroke, problem 2 would be surmountable.
4. There’s no eraser. This would also help with problem 2.
5. The thicker lines are not smooth. I guess this is because I’ve got the stroke ends set to square, because it made the palette look neater; either round ends need to be turned back on for drawing, or I need to interpolate the pixels between the line segments.
6. The focus highlighting on the fattest stroke in the palette isn’t wide enough.
7. You shouldn’t be able to draw behind or over the palette.

Questions:

• Clicking to turn the pencil on and off is a bit confusing, perhaps. Maybe it should be click-and-drag, instead?
• This stroke selector is probably too difficult to operate when you’re using a glove-mounted LED that’s being tracked by a camera, instead of drawing directly with a mouse. Maybe it should be a single slider, instead, for the PComp version. You basically need to be able to use this interface with mittens on.

So, obviously, I need to solve the various problems, of which #3 is probably the tricksiest. To make an undo command, I think I’ll probably save the drawing state every time the pencil is turned off. And “undo” should use the standard keyboard shortcut, of course, as well as having a button.

Once those issues are addressed, I think color is the next thing to deal with. The color picker may be so large as to have to be retractable, in which case saving the drawing state becomes even more essential.

If you have trouble imagining how one might make expressive images using tools like Processing, take a look at the beautiful pixel-style art of Craig @ Superbrothers.

Of course, some of the loveliness of these illustrations comes from the fact that they’re not made of flat, pure-hued pixels—the colored rectangles are grainy, organic, and warm. But the greater part of their success comes, I think, from the concision of the shapes. So much information is conveyed by so few squares.

If you can channel that kind of economy, you can probably draw with anything.

Via illoz.

I’m working with Dimitri(o)s, Diego, and Jason on the video project du jour, and this week we had to draw storyboards for our piece. The guys wrote a script on Thursday, while I had a prior engagement. Today we split the script up into four chunks, and each of us drew the panels for one section. My first four panels are above. The whole storyboard is in this PDF that Diego made. As a bonus, doing this assignment also got me off the hook for missing a few days of DrawMo!

The gist of the story is that this guy (ML, aka Male Lead) discovers that his headphones allow him to hear other people’s thoughts—but only negative ones.

Next week, we’re somehow going to try to shoot this thing, in the subway. Fortunately, it looks like videotaping in the subway, even with a tripod, is not illegal, as long as you don’t block access or passage. Basically, as long as we don’t act like those film crew assholes who’re always redirecting me around my own fucking office building, we should be fine. Cops often have interesting misconceptions about the laws they’re supposedly enforcing, though, so I’ll try to remember to print out a copy of the rules before we go.

P.S. We used a cleaner version of the storyboard form, which I made because the one Spencer supplied filled me with sorrow.

Get it now! Detailed, full-color documentation of the famed MC Squared midterm project!

MC_Squared(fin).pdf (14.68 MB; sorry, it contains a couple of embedded videos)

We gave our presentation today, the thing mostly worked, and it wasn’t too embarrassing. And, unlike some people in the class, my group actually got two or three hours of precious, golden sleep the night—well, morning—before. (We closed down the floor at about 3:30 a.m., but a few of our classmates relocated to the library or some such place to keep working. Everybody seemed pretty crispy by 9:30 this morning.)

First I thought I’d use one of these nifty sensors I got from SparkFun,

but then I realized I have no idea how you’re supposed to hook them up. Stick the pins straight into the breadboard? Solder wires on? How long should the wires be? So instead I used the stupid knob again, plus one of the IRs I bought for our midterm project.

The thrill of the knob has totally worn off. Then I saw Jorge soldering wires to an ultrasonic range finder just like the one I have, and I thought maybe it was a good time to try out my own. Ha! Thus began one of my more frustrating soldering bouts so far.

It must have taken me forty-five minutes to solder three freaking wires onto this cookie . . . and then it took me another hour to realize that the reason it wasn’t working was that I’d soldered the yellow wire to the wrong hole. And then I couldn’t get it unsoldered to save my life, so I just attached a fourth wire.

Finally I got them all hooked up:

And then, there was serial output:

• “Now you get a range of garbage characters.”
• “List all the available serial ports.”
• // print out the values you got:
for (int sensorNum = 0; sensorNum < sensors.length; sensorNum++) {
print("Sensor " + sensorNum + ": " + sensors[sensorNum] + "\t");
}
// add a linefeed after all the sensor values are printed:
println();

Et cetera.

After all that hair-pulling, the ultrasonic sensor was giving me really erratic readings (then again, so was the push-button switch: its value didn't change when I pushed the button, but it did when I touched the button). So I switched to two IR sensors, since I had so many lying around.

Then I got to the part about the handshake. Handshaking? Was not happening for me. I think it may have had something to do with this:

The Processing application was looking for the word "hello," but the Arduino didn't seem to be able to say the word without stuttering horribly. I tried for more than an hour, I think, to get them to talk to each other, but finally I had to just give up.

Part 2 in the saga that began last week.

After spending about an hour playing with the Minim library in Processing, I went into the lab to see if I could get it to work with actual input from our IR sensors. This was basically a repeat of this week’s homework, which I’d done, for once, before the morning it was due, so the wiring part was uncharacteristically easy. I need to get some header pins, though; the stranded wire on the IR sensors is a pain to plug into a breadboard.

So our project—which I realize I didn’t explain last week—is going to be a cubeoid musical (or, at least, noisy) instrument with an infrared sensor set into each side, mounted corner-up (as demonstrated by Diego) on a camera tripod. One or more players can then use their hands or other body parts or utensils or pets to trigger different sounds from each side. We were thinking that for Phase One, i.e., this week, we’d have the sounds be synthesized tones, and that for Phase Two, the final version, we’d make it play various different loops.

It turned out, however, that it’s far easier—for me, at least—to get Minim to play loops than to synthesize sounds. And there are a lot of free sound clips out there in the world. I got mine from CanadianMusicArtists.com. This pre-prototype, therefore, has only two sensors, both of which trigger audio loops. It also has the beginnings of a lame-ass bouncing ball animation, but it doesn’t do what I want it to do, mostly because the signal’s changing too rapidly. Graphics were a tentative feature for Phase Two, so I’m not going to fuss with that part any more this week.

Here’s some crappy video of Filippo (left) and Diego (right) making the sensors generate noise. You can barely hear it, unfortunately—listen for the annoying rapid clicking sound, which I think is the hi-hat sound.

The beauty part? This doubles as my ICM homework.

Here’s the Arduino code:

/* Reads data from two analog sensors (IR sensors, in this specific example)
and outputs the values in a format that can be easily parsed in Processing.
*/

int ledPin = 7;
int irSensor0 = 0;
int irSensor1 = 1;
int sensorValue = 0;

void setup()
{
// Flash the LED three times to announce the start of program.
pinMode( 7, OUTPUT );
digitalWrite( 7, LOW );
delay( 300 );
digitalWrite( 7, HIGH );
delay( 300 );
digitalWrite( 7, LOW );
delay( 300 );
digitalWrite( 7, HIGH );
delay( 300 );
digitalWrite( 7, LOW );
delay( 300 );
digitalWrite( 7, HIGH );
delay( 300 );
digitalWrite( 7, LOW );

// Start serial port at 9600 bps:
Serial.begin( 9600 );
}

void loop()
{
if (Serial.available() > 0)
{
// Read the first (0) sensor:
sensorValue = analogRead( irSensor1 );

// print the results:
Serial.print( sensorValue, DEC );
Serial.print( "\t" );

// read the second (1) sensor:
sensorValue = analogRead( irSensor0 );
// print the results:
Serial.println( sensorValue, DEC );

// Follow the last sensor value with a println() so that
// each set of four readings prints on a line by itself:
Serial.println( sensorValue, DEC );
// delay ( 100 );
}
}

And here’s the Processing code, where most of the excitement takes place.

I was trying to figure out the math to make part of the graph show up as brown—i.e., earth—and then scatter flowers on top, but something wasn’t working out and I was running late, so I gave up.

Anyway, here are the progress shots:

Setup:

Pot hooked up:

Blinking LED:

I also shot a fascinating movie of the program loading on the Arduino and starting up—you know, flickering yellow light, then blinking LED; hot stuff:

Final code on Arduino:

int potPin = 0;
int potValue = 0;
int ledPin = 2;

void setup()
{
// flash LED three times to announce start of program
pinMode( 2, OUTPUT );
digitalWrite( 2, LOW );
delay( 500 );
digitalWrite( 2, HIGH );
delay( 500 );
digitalWrite( 2, LOW );
delay( 500 );
digitalWrite( 2, HIGH );
delay( 500 );
digitalWrite( 2, LOW );
delay( 500 );
digitalWrite( 2, HIGH );
delay( 500 );
digitalWrite( 2, LOW );
delay( 500 );
digitalWrite( 2, HIGH );

// start serial port at 9600 bps:
Serial.begin( 9600 );
}

void loop()
{
// read analog input, divide by 4 to fit it in the range 0-255:
potValue = analogRead( potPin );
potValue = potValue / 4;
Serial.print( potValue, BYTE );
// pause for 10 milliseconds:
delay( 10 );
}


Final Processing applet