« September 2006 | Main | November 2006 »

October 29, 2006

Color Code (PComp Midterm)

Below are pics and code for our midterm project (in collaboration with Daniel Soltis, Scott Varland, and Jeff Sable): a improved version of the game Mastermind. See previous entries for our observation of the original Mastermind, and my initial notes and code for more information.

Though there is much further we feel we could go with this project (and might, for the final), we feel that it is already a success. Testing (and our own experience) showed that the game was engaging, satisfying, attractive, and technically accurate...all of which were our goals for the project. We strove to eliminate the frustration we experienced and observed with the original game, while adding compelling visual elements.

Logistically, the final game works by a processing program randomly selecting a four-color code for the player to try to break, using 6 possible colors. The physical interface, constructed from a cigar box, uses photocells and an arduino to detect the color of the cube fitted in each slot based on opacity when the user pushes the enter button. On the computer interface, the four colors are displayed, along with a semi-transparent, pixelated four-part "exquisite corpse" graphic. A column to the right tells the player the number of cubes with the correct color in the correct position, the number with the wrong color but correct position, and the total number with at least one value correct. For each correct color in the incorrect position, a portion of the "exquisite corpse" graphic is brought to full transparency, but remains pixelated. For each correct color in the correct position, a portion of the "exquisite corpse" graphic is revealed, unpixelated at full transparency. The graphics used for the exquisite corpse are randomly selected from a pool of images for each location (e.g., head, torso, feet). As described in more detail in the instructions shown below, the player has 10 chances to break the code or lose the game.

Challenges we overcame along the way included:

  • as we could not find manufactured translucent colored cubes, we had to find a way to get accurate and reliable photocell readings from the cubes. We initially tried glass paint, but the results were unreliable—the solution that finally worked, after many tries, was to print saturation-calibrated labels on transparent adhesive-backed sheets, and attach them by hand.
  • the computer interface was refined several times to make the code feedback more intuitive to the player. User testing had found that some of the original feedback was confusing, so we tried several methods before settling on the solution described above.

Classic Mastermind

Initial Prototype

Final Physical Interface

Sceen Interface

Full Layout

Instructions

Daniel Testing the Game

Processing Code

import processing.serial.*;

Serial port;                         // The serial port
int[] serialInArray = new int[4];    // Where we'll put what we receive
int serialCount = 0;                 // A count of how many bytes we receive

ExCorpse[] corpseImage = new ExCorpse[4]; //where we'll keep the images
int maxFiles = 6; //this should be the total number of different images (for a single type such as head) we have in the data folder

int IMGWIDTH = 500; //some constants for layout purposes
int IMGHEIGHT = 160;
int HISTORYWIDTH = 220;
int HINTWIDTH = 200;
int GUTTER = 10;

PFont font; //font for title and messages
PFont font2; //for for hints and clues

color redBlock = color(208, 36, 36); //these are the actual colors
color greenBlock = color(4, 203, 4);
color yellowBlock = color(235, 224, 31);
color blueBlock = color(60, 83, 220);

int[] answerCode = new int[4]; //where we'll store the secret code
int[] guessCode = new int[4]; //where we'll store the guesses
int guessNumber = 0; //keeps track of how many guesses

int RESET = 0; //constants to indicate whether a click resets the game or updates with the next guess
int UPDATE = 1; 
int actionPhase = UPDATE; //we start out in update mode

String actionMessage = "NEXT GUESS"; //this displays the bottom left message

void setup(){
  //  println(Serial.list());
  port = new Serial(this, Serial.list()[0], 9600);
  font = loadFont("Handwriting-Dakota-48.vlw");
  font2 = loadFont("ArialMT-14.vlw");

  size(960, 720);
  background(172, 196, 154);

  drawBackground();
  initializeImages();

  //Select the code answer on setup
  for (int i = 0; i<4; i++){
    answerCode[i] = floor(random(6));
  }
  println(answerCode);
}

void draw(){ //nothing! ha ha ha ha ha ha ha!
}

void serialEvent(Serial port) { //whenever we get data from the arduino, which is sent whenever the user presses the button

  if (actionPhase == UPDATE){ //if we're in updating mode
    // Add the latest byte from the serial port to array:

    serialInArray[serialCount] = port.read();
    serialCount++;

    //INTEPRETATION OF ARDUINO INPUT HAPPENS HERE
    if (serialCount > 3 ) { //once we get all four numbers for the colors
      for (int i=0; i<4; i++) //for each number in our guess array, interpret the number we received into a color
      {
        if (serialInArray[i] >= 0 && serialInArray[i] < 10){
          guessCode[i] = 0; //black
        }
        else if (serialInArray[i] >= 210 && serialInArray[i] < 255){
          guessCode[i] = 1; //clear
        }
        else if (serialInArray[i] >= 52 && serialInArray[i] < 100){
          guessCode[i] = 2; //red
        }
        else if (serialInArray[i] >= 100 && serialInArray[i] < 155){
          guessCode[i] = 3; //green
        }
        else if (serialInArray[i] >= 10 && serialInArray[i] < 52){
          guessCode[i] = 4; //blue
        }
        else if (serialInArray[i] >= 155 && serialInArray[i] < 210){
          guessCode[i] = 5; //yellow
        }
      }
      //println(guessCode);
      //println(serialInArray);
      //println("comparing!");
      if (guessNumber < 9){ //if we've made fewer than the maximum number of guesses
        delay(10); //i think i was trying to get rid of that flicker; i'm scared to remove it now without debugging that
        if (guessNumber >= 0){ 
          CompareUpdate(); //compare the guess to the answer and update results
        } 
        guessNumber = guessNumber + 1; //tally up the number of guesses
      }
      else{ //on the last guess
        CompareUpdate(); //compare and update the results
        guessNumber = -1; //reset to a new game with all guesses available
        actionPhase = RESET; //set so the next click will start a new game
      }
      serialCount = 0; //reset so we make a new array on the next button press
    }
    //NOTE: I think there are actually 3 phases: reset, begin new game, and update. this is why
    //in the resets, the guess number gets set to negative numbers sometimes.  it was on-the-fly
    //debugging to make everything synch up, but it's not very good code, and it makes it hard
    //to have anything happen after the end of one game and before the beginning of another

  }
  else if (actionPhase == RESET){ //if we're in reset mode, choose a new code and new images
    serialInArray[serialCount] = port.read();
    serialCount++;

    if (serialCount > 4 ) { //save the number and all colors into an array, just for housekeeping, but don't do anything with them
      serialCount = 0;
    }

    drawBackground();
    initializeImages();
    for (int i = 0; i<4; i++){ //select a new answer code of 4 numbers between 0 and 5
      answerCode[i] = floor(random(6));
    }
   // println(answerCode);  //UNCOMMENT ME FOR DEBUGGING PURPOSES, SO YOU CAN WIN FASTER
    actionPhase = UPDATE; //ready to guess!
  }
}

//FUNCTION TO DO ALL THE LOGIC OF THE PUZZLE AND PRESENT RESULTS
void CompareUpdate(){
  //some variables that need to die at the end of this function
  int guessMatch = 0; //running tally of how many gueses match a color
  int answerMatch = 0; //running tally of how many answers match a color
  int matchTally = 0; //how many colors match between guess and answer
  int perfectMatch = 0; //how many colors and positions match
  delay(100); //a delay to try and get rid of the flicker? not sure; leave it for now

  //THIS BIT COLORS IN THE HISTORY BOXES WITH YOUR GUESS
  for (int i = 0; i<4; i++){ //this bit compares the inputed data to display what color that data represents
    fill(interpetColor(guessCode[i]));
    rect((29 + IMGWIDTH) + i*52, 90 + guessNumber*55, 45, 45); //and then draws them in the appropriate row of the history block
  }

  //THIS BIT COMPARES THE GUESS TO THE SECRET ANSWER
  for (int i = 0; i<6; i++){ //for each of the six possible colors
    for (int j=0; j<4; j++){ //for each of the possible four choices
      if (guessCode[j] == i){ //if the guess is that color
        guessMatch = guessMatch + 1; //tally that up
      } 
      if (answerCode[j] == i){ //if the answer is that number
        answerMatch = answerMatch + 1;  //tally that up
      }  
    }
    answerMatch = min(guessMatch, answerMatch); //find how many of the same color are in the answer and guess
    matchTally = matchTally + answerMatch; //keep track of the total of all colors
    answerMatch = 0; //reset for a new color
    guessMatch = 0; //reset for a new color
  }

  //THIS BIT UPDATES THE STATE OF THE IMAGES
  for (int k=0; k= 9){ //if the player is at the end of their guesses

    fill(172, 196, 154); //cover up the old message at the top left
    rect(100, 0, 300, 65); 
    for (int i = 0; i<4; i++){ //display the correct answers as little blocks above the lose message
      fill(interpetColor(answerCode[i]));
      rect(120 + (i * 15), 15, 10, 10);
    }   
    textFont(font, 24);
    fill(0);
    actionMessage = "SORRY! PLAY AGAIN."; //let the player know they lost
    text(actionMessage, 110, 50);
    actionPhase = RESET; //change the phase so that the next click is a reset
    perfectMatch = 0;
    actionMessage = "NEXT GUESS"; 
  }
  //END OF FUNCTION
}

//THIS CLASS GOVERNS THE PIXELLATION AND DRAWING OF THE EXQUISITE CORPSE IMAGES

class ExCorpse{ 
  PImage exquisiteImage;
  int IMGWIDTH = 500;
  int IMGHEIGHT = 160;

  ExCorpse(String _image){ // this declaration of the object loads the entered image
    exquisiteImage = loadImage(_image); 
  }

  void drawTransPixel(int _x, int _y){ //draw the image pixellated and trasparent
    image(exquisiteImage, _x, _y, IMGWIDTH, IMGHEIGHT); //draw the image in the correct location
    loadPixels(); //load the pixels of the screen
    for(int row=_y; row<_y + IMGHEIGHT; row = row + 40) { //for each row of the pixels
      for(int col=_x; col<_x + IMGWIDTH; col = col + 50) { //for each column of the pixels
        color pix = pixels[(col+25) + (row+20)*width]; //get the color of the image at the center of each 40X50 pixel
        pix = color(red(pix), green(pix), blue(pix), 80); //make that color transparent
        noStroke();
        fill(255);//set the background to white for the pixel
        rect(col, row, 50, 40); //draw the pixel
        fill(pix);//fill the pixel with the transparent color
        rect(col, row, 50, 40); //draw the pixel
      }
    }
  }

  void drawPixel(int _x,int _y){ //draw the image pixellated at full opacity
    image(exquisiteImage, _x, _y, IMGWIDTH, IMGHEIGHT); //draw the image in the correct location
    loadPixels(); //load the pixels of the display screen
    for(int row=_y; row<_y + IMGHEIGHT; row = row+20) { //for each row of the pixels
      for(int col=_x; col<_x + IMGWIDTH; col = col + 20) { //for each column of the pixels
        color pix = pixels[(col+10) + (row+10)*width]; //get the color of the image in the center of each 20X20 pixel
        noStroke();
        fill(pix); //fill the pixel with the color
        rect(col, row, 20, 20); //draw the pixel
      }
    }
  }

  void drawFull(int _x,int _y){ //draw the image at full resolution
    image(exquisiteImage, _x, _y, IMGWIDTH, IMGHEIGHT); //draw the image
  }
}
//END OF CLASS

//FUNCTION TO FORMAT SCREEN
void drawBackground(){//this sets up the formatting of the screen

  background(172, 196, 154); //BACKGROUND COLOR
  strokeWeight(1);
  stroke(0);
  noFill();
  rect(GUTTER-1, GUTTER + 60, IMGWIDTH+1, 4*IMGHEIGHT+1); //RECTANGLE THAT CONTAINS THEIMAGES
  fill(231, 231, 197);
  rect(2*GUTTER + IMGWIDTH, GUTTER, HISTORYWIDTH, 4*IMGHEIGHT); //RECTANGLE THAT CONTAINS THE COLORED BLOCKS
  //println(width - (4*GUTTER + IMGWIDTH + HINTWIDTH));
  fill(231, 231, 197);
  rect(3*GUTTER + IMGWIDTH + HISTORYWIDTH, GUTTER, HINTWIDTH, 4*IMGHEIGHT); //RECTANGLE THAT CONTAINS THE HINTS
  fill(102, 102, 90);
  rect(3*GUTTER + IMGWIDTH + HISTORYWIDTH, GUTTER, HINTWIDTH, 72); //top rectangle that will contain the headers for the hints

  for (int j = 0; j<10; j++){  //these are the blocks that will be filled with color
    for (int i = 0; i<4; i++){
      fill(102, 102, 90, 50);
      rect((28 + IMGWIDTH) + i*52, 90 + j*55, 46, 46);
    }
  }
  noStroke();
  fill(255, 255, 255, 100);
  rect(width - (HINTWIDTH + 9), 42, 66, 40);
  fill(255, 255, 255, 75);
  rect(width - (HINTWIDTH + 9) + 66, 42, 66, 40);
  fill(255, 255, 255, 50);
  rect(width - (HINTWIDTH + 9) + 132, 42, 67, 40);
  stroke(0);
  strokeWeight(1);
  line( width - (HINTWIDTH + 9), 42, width - (HINTWIDTH + 9) + 198, 42);

  textFont(font2, 20);
  fill(0);
  text("History & Hints", 3*GUTTER + IMGWIDTH + 35, 5*GUTTER); //LABEL AT THE TOP OF THE RECTANGLE CONTAINING THE COLORED BLOCKS

  textFont(font2, 14);
  fill(255);
  textFont(font2, 18);
  text("correct color", width - HINTWIDTH + 45, 31); //TEXT HEADERS FOR HINTS

  textFont(font2, 15);
  text("total", width - (HINTWIDTH), 57);
   text("correct", width - (HINTWIDTH), 68);
    text("colors", width - (HINTWIDTH), 79);

  stroke(0);
  strokeWeight(1);
  line(width - 4*HINTWIDTH/6-10, 45, width - 4*HINTWIDTH/6-10, height - 72);

  textFont(font2, 14);
  text("corect", width - 4*HINTWIDTH/6-3, 61);
  text("position", width - 4*HINTWIDTH/6-3, 73);

  stroke(0);
  strokeWeight(1);
  line(width - 4*HINTWIDTH/6+56, 45, width - 4*HINTWIDTH/6+56, height - 72);

  text("wrong", width - 2*HINTWIDTH/6-3, 61);
  text("position", width - 2*HINTWIDTH/6-3, 73);

  textFont(font, 48);
  String colorCode = "COLOR CODE";  //TITLE BAR AT BOTTOM RIGHT
  for (int i=0; i

Arduino Code

int firstSensor = 0;    // first photocell sensor (connect to analog pin 0)
int secondSensor = 1;   // second photocell sensor (connect to analog pin 1)
int thirdSensor = 2;    // third photocell sensor  (connect to analog pin 2)
int fourthSensor = 3;   // fourth photocell sensor  (connect to analog pin 3)
int firstLED = 2;       // first LED pin (connect to digital pin 2)
int secondLED = 3;      // second LED pin (connect to digital pin 3)
int thirdLED = 4;       // third LED pin (connect to digital pin 4)
int fourthLED = 5;       // fourth LED pin (connect to digital pin 5)
int inByte = 0;         // incoming serial byte
int switchPin = 6; // switch pin (connect to digital pin 6)
int switchValue = 0; // switch input
int previousSwitchState = 0;

void setup()
{
  // start serial port at 9600 bps:
  Serial.begin(9600);
  pinMode(firstLED, OUTPUT);     // set the first LED pin to be an output
  pinMode(secondLED, OUTPUT);    // set the second LED pin to be an output
  pinMode(thirdLED, OUTPUT);     // set the third LED pin to be an output
  pinMode(fourthLED, OUTPUT);    // set the fourth LED pin to be an output
  pinMode(switchPin, INPUT); //set the switch pin to be an input
}

void loop()
{
  //turn on the four LEDs

  digitalWrite(firstLED, HIGH);    // turn on the first LED
  digitalWrite(secondLED, HIGH);    // turn on the second LED
  digitalWrite(thirdLED, HIGH);    // turn on the third LED
  digitalWrite(fourthLED, HIGH);    // turn on the fourth LED

  switchValue =  digitalRead(switchPin);
  //  Serial.println(switchValue);
  // if we get a valid byte, read analog ins:
  if (switchValue==1 && previousSwitchState == 0) {

    delay(300);
    // read first analog input
    firstSensor = constrain(analogRead(0)/4,0,255);
    // delay 10ms to let the ADC recover:
    delay(10);
    // read second analog input
    secondSensor = constrain(analogRead(1)/4 + 5,0,255);
    // delay 10ms to let the ADC recover:
    delay(10);
    // read third analog input
    thirdSensor =  constrain(analogRead(2)/4,0,255);
    // delay 10ms to let the ADC recover:
    delay(10);
    // read fourth analog input
    fourthSensor =  constrain(analogRead(3)/4 - 15,0,255);
    /*    Serial.print(firstSensor);
     Serial.print("\t");  
     Serial.print(secondSensor);
     Serial.print("\t");  
     Serial.print(thirdSensor);
     Serial.print("\t");  
     Serial.println(fourthSensor);*/
    // send sensor values:

    Serial.print(firstSensor, BYTE);
    Serial.print(secondSensor, BYTE);
    Serial.print(thirdSensor, BYTE);
    Serial.print(fourthSensor, BYTE); 
    delay(80);

    digitalWrite(firstLED, LOW);    // turn on the first LED
    digitalWrite(secondLED, HIGH);    // turn on the second LED
    digitalWrite(thirdLED, HIGH);    // turn on the third LED
    digitalWrite(fourthLED, HIGH);    // turn on the fourth LED
    delay(80);

    digitalWrite(firstLED, LOW);    // turn on the first LED
    digitalWrite(secondLED, LOW);    // turn on the second LED
    digitalWrite(thirdLED, HIGH);    // turn on the third LED
    digitalWrite(fourthLED, HIGH);    // turn on the fourth LED
    delay(80);

    digitalWrite(firstLED, LOW);    // turn on the first LED
    digitalWrite(secondLED, LOW);    // turn on the second LED
    digitalWrite(thirdLED, LOW);    // turn on the third LED
    digitalWrite(fourthLED, HIGH);    // turn on the fourth LED
    delay(80);

    digitalWrite(firstLED, LOW);    // turn on the first LED
    digitalWrite(secondLED, LOW);    // turn on the second LED
    digitalWrite(thirdLED, LOW);    // turn on the third LED
    digitalWrite(fourthLED, LOW);    // turn on the fourth LED
    delay(160);

    digitalWrite(firstLED, HIGH);    // turn on the first LED
    digitalWrite(secondLED, LOW);    // turn on the second LED
    digitalWrite(thirdLED, LOW);    // turn on the third LED
    digitalWrite(fourthLED, LOW);    // turn on the fourth LED 
    delay(80); 

    digitalWrite(firstLED, HIGH);    // turn on the first LED
    digitalWrite(secondLED, HIGH);    // turn on the second LED
    digitalWrite(thirdLED, LOW);    // turn on the third LED
    digitalWrite(fourthLED, LOW);    // turn on the fourth LED 
    delay(80); 

    digitalWrite(firstLED, HIGH);    // turn on the first LED
    digitalWrite(secondLED, HIGH);    // turn on the second LED
    digitalWrite(thirdLED, HIGH);    // turn on the third LED
    digitalWrite(fourthLED, LOW);    // turn on the fourth LED 
    delay(80); 
  }
  previousSwitchState = switchValue;
}

October 19, 2006

ICM Midterm Prep

For the midterm, I've completed the first step of what I hope to be a larger project using live video input to generate abstract graphics on the fly, the style of which will be determined by the users interaction--probably proximity.

For this first step, I have adapted an applet named Box Fitting Img by J. Tarbell to generate one example of the type of abstraction I'd like to achieve with live data.

Click here to play with the applet >>

October 12, 2006

Scraping Googlism for Data

Assignment: Create a Processing applet that uses input from a text file or URL.

This applet mines the googlism.com website for text, based on the name that is typed in by the user.

Click here to see it in action >>

Following is the Processing code:

// Kate Monahan
// ICM, 2006
// this code scrapes googlism.com for data

PFont f;

String googleContent = ""; // Var to store input from Google site
String firstName = ""; // Var to store name
String googlism1 = ""; // Var to store results from Google

void setup() {
  size(250,250);
  // Call our new getGooglism function!
  getGooglism(firstName);
  f = loadFont("OptimaLTStd-Medium-16.vlw");
  println(firstName);
  println(googlism1);
}

void draw() {
  background(100);
  textFont(f);
  textMode(SCREEN);
  fill(255);
  noStroke();
  rect(10,60,200,20);
  // Display all the stuff we want to display
  if (googlism1 == ""){
    text("Click below, type your first name & hit enter:",10,10,width-20,height-10);
    fill(0);
    text(firstName,12,63,width-10,height-10);
  }
  else if (googlism1 != ""){
    fill(100);
    rect(10,60,200,20);
    fill(255);
    text("Your #1 Googlism is: ",10,10,width-10,height-10);
    text("\"" + googlism1 + "\"",10,30,width-10,height-10);
    fill(200);
    text("Hit enter again to reset >>",10,230,width-10,height-10);
  }
}

void getGooglism(String zip) {
  //Get all the HTML/XML source code into an array of strings (each line is 
   //one element in the array)
  //note that a line break has been inserted for formatting purposes 
  //here after the "?"
   String url = "http://itp.nyu.edu/icm/proxy/proxy.php?
  url=http://www.googlism.com/index.htm?ism=" + firstName;
  String[] lines = loadStrings(url);

 // Get rid of the array and make it one very long String
  String xml = join(lines, " "); 

  // Searching for weather condition
  String lookfor = "Googlism for:</span> ";
  String end = "
\r\r\t\t\r\t\r\t<tr>\r\t\t\r\t\t<table class="; googleContent = giveMe(xml,lookfor,end); // Searching for first name end = "
"; firstName = giveMe(xml,lookfor,end); // Searching for first google response lookfor = "
"; end = "
"; googlism1 = giveMe(xml,lookfor,end); } // A function that returns a substring between two substrings String giveMe(String s, String before, String after) { String found = ""; // Find the index of the beginning tag int start = s.indexOf(before); // If we don't find anything, send back a blank String if (start == -1) return ""; // Move to the end of the beginning tag start += before.length(); // Find the index of the end tag int end = s.indexOf(after,start); // If we don't find the end tag, send back a blank String if (end == -1) return ""; // Return the text in between return s.substring(start,end); } void keyPressed() { // If the return key is pressed, send the String to google and clear it if (key == '\n') { getGooglism(firstName); firstName = ""; } // Otherwise, concatenate the String with what the // user types on the keyboard else { firstName = firstName + key; } }

October 09, 2006

Midterm Prototype Notes

Following is some of the initial Arduino and Processing code for use with our first midterm protype. The interface will entail cubes of seven colors and four slots. The player will need to choose the correct combination of colors (a la Mastermind) to solve the puzzle. Their selection will be recorded based on data from photocells located at the bottom of each slot--the glass cubes will return different results based on their relative opacity based on color. The history of their selections and the results (on the model, again, of Mastermind feedback) will be printed to a Processing interface, along with a photographic puzzle that will be solved when the correct selections are made.

Initial Arduino Code

int firstSensor = 0;    // first photocell sensor (connect to analog pin 0)
int secondSensor = 1;   // second photocell sensor (connect to analog pin 1)
int thirdSensor = 2;    // third photocell sensor  (connect to analog pin 2)
int fourthSensor = 3;   // fourth photocell sensor  (connect to analog pin 3)
int firstLED = 2;       // first LED pin (connect to digital pin 2)
int secondLED = 3;      // second LED pin (connect to digital pin 3)
int thirdLED = 4;       // third LED pin (connect to digital pin 4)
int fourthLED = 5;       // fourth LED pin (connect to digital pin 5)
int inByte = 0;         // incoming serial byte

void setup()
{
  // start serial port at 9600 bps:
  Serial.begin(9600);
  pinMode(firstSensor, OUTPUT);     // set the first LED pin to be an output
  pinMode(secondSensor, OUTPUT);    // set the second LED pin to be an output
  pinMode(thirdSensor, OUTPUT);     // set the third LED pin to be an output
  pinMode(fourthSensor, OUTPUT);    // set the fourth LED pin to be an output
}

void loop()
{
  //turn on the four LEDs
  digitalWrite(firstLED, HIGH);    // turn on the first LED
  digitalWrite(secondLED, HIGH);    // turn on the second LED
  digitalWrite(thirdLED, HIGH);    // turn on the third LED
  digitalWrite(fourthLED, HIGH);    // turn on the fourth LED
  
  // if we get a valid byte, read analog ins:
  if (Serial.available() > 0) {
    // get incoming byte:
    inByte = Serial.read();
    
    // read first analog input
    firstSensor = constrain(analogRead(0)/4,0,255);
    // delay 10ms to let the ADC recover:
    delay(10);
     // read second analog input
    secondSensor = constrain(analogRead(1)/4,0,255);
    // delay 10ms to let the ADC recover:
    delay(10);
     // read third analog input
    thirdSensor =  constrain(analogRead(2)/4,0,255);
    // delay 10ms to let the ADC recover:
    delay(10);
     // read fourth analog input
    fourthSensor =  constrain(analogRead(3)/4,0,255);

    // send sensor values:
    Serial.print(firstSensor, BYTE);
    Serial.print(secondSensor, BYTE);
    Serial.print(thirdSensor, BYTE);
    Serial.print(fourthSensor, BYTE);      
  }
}

Extremely Stripped-down Processing Code

This is the very basic Processing code for our team to begin working with.

import processing.serial.*;

Serial port;                         // The serial port
int firstSensor;                    // Initialize the first sensor variable
int secondSensor;                    // Initialize the second sensor variable
int thirdSensor;                    // Initialize the third sensor variable
int fourthSensor;                    // Initialize the fourth sensor variable
int[] serialInArray = new int[4];    // An array for our results
int serialCount = 0;                 // A count of how many bytes we receive
boolean firstContact = false;        // Whether we've heard from the microcontroller

void setup() {
  // Print a list of the serial ports, for debugging purposes:
  println(Serial.list());

  port = new Serial(this, Serial.list()[2], 9600); // define the serial port
  port.write(65);    // Send a capital A to start the microcontroller sending
}

void draw() {
  // If no serial data has beeen received, send again until we get some.
  // (in case you tend to start Processing before you start your 
  // external device):
  if (firstContact == false) {
    delay(300);
    port.write(65);
  }
}

void serialEvent(Serial port) {
  // if this is the first byte received, 
  // take note of that fact:
  if (firstContact == false) {
    firstContact = true; 
  }
  // Add the latest byte from the serial port to array:
  serialInArray[serialCount] = port.read();
  serialCount++;

  // If we have 4 bytes:
  if (serialCount > 3 ) {
    firstSensor = serialInArray[0];
    secondSensor = serialInArray[1];
    thirdSensor = serialInArray[2];
    fourthSensor = serialInArray[3];

    // print the values to the serial monitor:
    println(firstSensor + "\t" + secondSensor + "\t" + thirdSensor + "\t" + fourthSensor);

    // Send a capital A to request new sensor readings:
    port.write(65);
    // Reset serialCount:
    serialCount = 0;
  }
}

Physical Mouse Using Photocells

Assignment: What applications can you think of that could use a better physical interface for a mouse? Create a prototype in Arduino and Processing, or whatever programming environment you choose. Come up with a physical interface that makes it clear what actions map to what movements and actions.

For this assignment, I built a simple console with two photocells wired to the base of two posts. The input of each photocell—which is manipulated by the user sliding a knob up and down the post—is detected through a serial USB connection to the Arduino board and fed to Processing. There, the values are passed into x and y values for "mouse" control over an applet of a dancing bear (see it in it's full glory here).

I didn't notice the grim news report in the background until later, by the way. That, combined with my friend's sound effects, make it all a bit surreal...


Some minor problems I encountered with this project were:

  • Initially, I wasn't referencing the correct serial port in my Processing code, which resulted in the following error: "RXTX Warning: Removing stale lock file. /var/lock/LK.001.009.000". To remedy this, I referenced the list of ports that were printed out in Processing, thanks to Tom's code.
  • It took a while of fiddling with the math to get the serial input from the photocells to match the output to Processing.
  • The Processing applet is occasionally glitchy (bear jumps). Still not sure what's causing it, but it's not a significant problem. Might be the result of my math (as I'm dividing the input by 6 in the Arduino before sending it to Processing).

Breadboard Setup

Arduino Code

int firstSensor = 0;    // first photocell sensor
int secondSensor = 0;   // second photocell sensor
int placeHolder = 0;   // null placeholder to make a full byte
int inByte = 0;         // incoming serial byte

void setup() {
  // start serial port at 9600 bps:
  Serial.begin(9600);
}

void loop() {
  // if we get a valid byte, read analog ins:
  if (Serial.available() > 0) {
    // get incoming byte:
    inByte = Serial.read();
    // read first analog input and do some math to make the range 0-165 (due to processing requirement):
    firstSensor = constrain(analogRead(0)/6,0,165);
    // delay 10ms to let the ADC recover:
    delay(10);
    // read second analog input and do some math to make the range 0-107:
    secondSensor = constrain(analogRead(1)/6,0,107);
    // read placeholder value (always 0)

    // send sensor values:
    Serial.print(firstSensor, BYTE);
    Serial.print(secondSensor, BYTE);
    Serial.print(placeHolder, BYTE);               
  }
}

Processing Code

import processing.serial.*;

int rVar = 146; //set initial r value
int gVar = 195; //set initial g value
int bVar = 222; //set initial b value
int xpos = 0; //takes the place of xpos
int ypos = 0; //takes the place of ypos
Serial port;                         // The serial port
int[] serialInArray = new int[3];    // Where we'll put what we receive
int serialCount = 0;                 // A count of how many bytes we receive
boolean firstContact = false;        // Whether we've heard from the microcontroller

void setup()  {
  size(200,200);
  smooth();
  frameRate(30);
  
  // Print a list of the serial ports, for debugging purposes:
  println(Serial.list());

  // Open serial port (using number based on results of list above)
  port = new Serial(this, Serial.list()[2], 9600);
  port.write(65);    // Send a capital A to start the microcontroller sending
}

void draw()  {
  // If no serial data has beeen received, send again until we get some.
  // (in case you tend to start Processing before you start your external device):
  if (firstContact == false) {
    delay(300);
    port.write(65);
  }
  
// BEAR  
  background(rVar,gVar,bVar);
  // the following statements limit the action to the window to avoid weirdness
  if (ypos >= 107)  {
    ypos = 107;
  }
  if (ypos <= 20)  {
    ypos = 20;
  }
  if (xpos >= 165)  {
    xpos = 165;
  }
  if (xpos <= 34)  {
    xpos = 34;
  }
  noStroke();
  //circles in bg
  fill(255,35);
  ellipse(70,50,ypos+20,ypos+20);
  ellipse(165,120,ypos+20,ypos+20);
  ellipse(55,165,ypos+20,ypos+20);
  fill(40,80,0); //set head color
  ellipse(xpos,ypos,50,40); //head
  ellipse(xpos+25,ypos-13,20,13); //right ear
  ellipse(xpos-25,ypos-13,20,13); //left ear
  rectMode(CENTER);
  fill(0);
  quad(xpos-5,ypos+18,xpos+5,ypos+18,xpos+13,ypos+90,xpos-12,ypos+90); //trunk
  rect(xpos,ypos-5,49,3); //shades band
  quad(xpos-17,ypos-5,xpos-3,ypos-5,xpos-5,ypos+5,xpos-15,ypos+5); //left shade
  quad(xpos+4,ypos-5,xpos+18,ypos-5,xpos+16,ypos+5,xpos+6,ypos+5); //right shade
  rect(xpos,ypos+31,32,10); //shoulders
  strokeWeight(10); //set arm width
  stroke(0);  //set arm and leg color
  line(xpos-15,ypos+31,xpos/5+50,ypos/4+100); //left arm
  line(xpos+15,ypos+31,xpos/5+110,ypos/4+100); //right arm
  strokeWeight(17);  //set leg width
  stroke(xpos/4*6+25,ypos/2*3-20,xpos*2-200); //set pant color
  rect(xpos,ypos+80,15,3); //hips
  line(xpos-6,ypos+82,xpos/5+60,ypos/4+165);  //left leg
  line(xpos+6,ypos+82,xpos/3+83,ypos/4+165); //right leg
  noStroke();
  //note that there is no line break in these two quads--added for formatting on this web page
  quad(xpos/5+52,ypos/4+165,xpos/5+69,ypos/4+165,xpos/5+71,ypos/4+178,
  xpos/5+46,ypos/4+178); //left foot
  quad(xpos/3+75,ypos/4+165,xpos/3+92,ypos/4+165,xpos/3+98,ypos/4+178,
  xpos/3+73,ypos/4+178); //right foot
  // if someone clicks or holds the mouse, change the bg
  if (mousePressed == true)  {
    rVar = rVar + 20;
    gVar = gVar + 5;
    bVar = bVar + 40;
  }
// if the colors go off the map, reset them to a lower value
  if (rVar >= 255)  {
    rVar = 0;
  }
  if (gVar >= 255)  {
    gVar = 10;
  }
  if (bVar >= 255)  {
    bVar = 30;
  }
}

void serialEvent(Serial port) {
  // if this is the first byte received, 
  // take note of that fact:
  if (firstContact == false) {
    firstContact = true; 
  }
  // Add the latest byte from the serial port to array:
  serialInArray[serialCount] = port.read();
  serialCount++;

  // If we have 3 bytes:
  if (serialCount > 2 ) {
    xpos = serialInArray[0];
    ypos = serialInArray[1];

    // print the values (for debugging purposes only):
    println(xpos + "\t" + ypos);

    // Send a capital A to request new sensor readings:
    port.write(65);
    // Reset serialCount:
    serialCount = 0;
  }
}

October 07, 2006

Midterm Observation: Mastermind

Assignment: Choose an action that produces changes in a medium. Observe a person or people engaged in the activity. What are the physical parameters of that activity? What does the person engaged in it do with their arms, their legs, their hands or feet, their head? What physical elements of the activity make it engaging? What elements make it difficult, painful, or boring? What patterns appear when the action is repeated? What are the physical characteristics of the medium that you have to take as given? What physical input to the tool or device suggest or mirror those characteristics?

Our group chose to observe the game "Mastermind." The goal of the game is for the "code breaker" to match the color and placement of four pegs arranged (and hidden) by the "code maker." After each attempt by the breaker, the maker gives feedback on how many correct colors and/or placements they have achieved. The breaker then makes another attempt, and so on, until the code is broken or the breaker exceeds the maximum number of allowed turns.

In observing our repeated rounds of play, the code breaker's posture tended to be forward and down--focused on the board, except when receiving the feedback pegs from the maker. The code maker's posture was more relaxed, but still focused on the board, as opposed to the breaker. As documented in the video, the breaker rarely, if ever, looked up from the board for interaction or feedback from the code maker, focusing instead on the feedback pegs the maker placed. The breaker tended to use their dominant hand to place the pegs most often, but occasionally used the other hand as well, particularly when they closed in on the solution, or had trouble manipulating the pegs.

Because the gameplay focuses almost exclusively on the pegs (and deductive reasoning), the pegs are both the elements that make the game engaging, and that make it difficult and somewhat annoying. Both types of pegs are difficult to manipulate due to their size and shape, and they did not fit securely into the holes, eliminating any satisfaction from their placement. We also found the interface unintuitive, unattractive, and poorly constructed--the square orientation of the four "feedback" holes next to the horizontal code holes and the similar scoring holes was confusing; the screen to hide the code did not do an adequate job; the colors were unappealing; and the storage area for the pieces did not keep them separated, and did not close properly. The one quality that we discussed and did agree was logical and useful towards the goal was the iterative history, which was both necessary for solving the puzzle, and adequately achieved in the interface. Nonetheless, there were several instances where code breakers instinctively rearranged incorrect pegs, rather than starting a new row.

Conclusions we drew from this observation and our goals for the midterm will be dicussed in the next entry.

[[Movie deleted to conserve server space. Send me email if you'd like to see it.]]