I used Processing with the minim library and the Audioplayer library for sound, and Carnivore to detect wifi packets.

I have the sound of crickets playing constantly to set the ‘mood’ and an image of a night sky. I’m mapping the IP addresses of packets received and sent as points that look like stars on the night sky and drawing a Bezier curve between the receiver and sender to both evoke a sense of constellations in the night sky as well as inform the user of the path of incoming and outgoing packets. Using a particle system that generates a circle of fixed size and random yellow/white color for every 10 packets that come in, I’ve tried to recreate (albeit in a very rudimentary fashion) a star burst.

I kept the midi to just piano notes to keep the sketch from sounding too discordant and to give it a sense of sonic mood suited to the night. The tones are generated from reading the encrypted packets and the loudness by the size of the packet. See full code here

The encrypted packets itself yielded some very interesting visuals which I chose not to use cause I thought it wouldn’t fit with the theming of the entire sketch, but perhaps I could find a more subtle way to include it in.

For my final for Art of Graphics Programming and Cooking with Sound I sonified and visualized wifi packets over the network. My first iteration of it for cooking with sound was rather simple and just used a background image and particle system. See: Starry Starry Wifi Night

Since the focus was primarily on sound in that class, I didn’t push the visuals enough for fear of exhausting the memory.

For Art of Graphics Programming however, I decided to remove the sound altogether and push the graphics further. I used a painter effect and a simplex Noise filter on this image

and generated particles using webGL to create this effect:

main.pde

import java.util.Iterator;
import org.rsg.carnivore.*;
import org.rsg.carnivore.net.*;

import processing.opengl.*;
import codeanticode.glgraphics.*;
import java.nio.FloatBuffer;


int SYSTEM_SIZE = 100000;
int CANVAS_WIDTH = 640;
int CANVAS_HEIGHT = 480;

GLModel sys;
GLTexture tex;

int npartTotal = 1000;
int npartPerFrame = 10;
float speed = 1.0;
float gravity = 0;

int partLifetime;

PVector velocities[];
int lifetimes[];


boolean clearImg = false;
boolean changeImg = true;
float changeTime = 2.0;
float destTexTransparency = 1.0;

GLTexture srcTex, destTex, brushTex;

PainterEffect painter;

int sec0;

HashMap nodes = new HashMap();
HashMap links = new HashMap();
float currentDiameter = 10.0;
float currentHalo = 3.0;
int min_diameter = 3;
float startBrightness = 255.0;
float shrinkSpeed = 0.97;
float dimSpeed_link = 0.9;
float dimSpeed_ripple = 0;
String packets[];

int xPos, yPos;



void setup(){
  size(640,480, GLConstants.GLGRAPHICS);
  colorMode(RGB, 1.0);
    
    srcTex = new GLTexture(this, "background.png");
    brushTex = new GLTexture(this, "brush2.png");    
    
    destTex = new GLTexture(this, width, height);
    destTex.loadPixels();
    for (int i = 0; i < destTex.width * destTex.height; i++) destTex.pixels[i] = 0xff000000;
    destTex.loadTexture();    

    painter = new PainterEffect(this, SYSTEM_SIZE, CANVAS_WIDTH, CANVAS_HEIGHT);
  partLifetime = npartTotal / npartPerFrame;
    
  sys = new GLModel(this, npartTotal, GLModel.POINT_SPRITES, GLModel.DYNAMIC);
  initColors();
  initPositions();
  initSprites();

  initVelocities();
  initLifetimes();  
  
  frameRate(10);
  ellipseMode(CENTER);
  CarnivoreP5 c = new CarnivoreP5(this);
  smooth();
}

void draw() {
  
  GLGraphics renderer = (GLGraphics)g;
  renderer.beginGL();  
  
  background(0);
  
  painter.apply(srcTex, brushTex, destTex, clearImg, changeImg, changeTime);
    if (changeImg) changeImg = false;

    tint(1.0, 1.0 - destTexTransparency);     
    image(srcTex, 0, 0, width*2, height*2); 
    tint(1.0, destTexTransparency);
    image(destTex, 0, 0, width*2, height*2);

    int sec = second();
    sec0 = sec;
  
   updatePositions();
  updateColors();
  updateLifetimes();
  
   renderer.setDepthMask(false);
  sys.render();
  renderer.setDepthMask(true);
    
  renderer.endGL();
  
  //println("nodes:"+nodes.size() + " links:"+links.size());
}

void initSprites() {
       tex = new GLTexture(this, "particlebrush.png");    
   float pmax = sys.getMaxPointSize();
   println("Maximum sprite size supported by the video card: " + pmax + " pixels.");   
   sys.initTextures(1);
   sys.setTexture(0, tex);  
   // Setting the maximum sprite to the 90% of the maximum point size.
   sys.setMaxSpriteSize(0.9 * pmax);
   // Setting the distance attenuation function so that the sprite size
   // is 20 when the distance to the camera is 400.
   sys.setSpriteSize(20, 400);
   sys.setBlendMode(BLEND);  
}

void initColors() {
  sys.initColors();
  sys.setColors(0, 0);
}

void initPositions() {
  sys.beginUpdateVertices();
  FloatBuffer vbuf = sys.vertices;
  float pos[] = { 0, 0, 0, 0 };
  for (int n = 0; n < sys.getSize(); n++) {
    vbuf.position(4 * n);
    vbuf.get(pos, 0, 3);  
    
    pos[0] = 0;
    pos[1] = 0;
    pos[2] = 0;
    pos[3] = 1; // The W coordinate must be 1.
    
    vbuf.position(4 * n);
    vbuf.put(pos, 0, 4);
  }  
  sys.endUpdateVertices();  
}

void initVelocities() {
  velocities = new PVector[npartTotal];
  for (int n = 0; n < velocities.length; n++) {
    velocities[n] = new PVector();
  }  
}

void initLifetimes() {
  // Initialzing particles with negative lifetimes so they are added
  // progresively into the scene during the first frames of the program  
  lifetimes = new int[npartTotal];
  int t = -1;
  for (int n = 0; n < lifetimes.length; n++) {    
    if (n % npartPerFrame == 0) {
      t++;
    }
    lifetimes[n] = -t; 
  }  
}

void updatePositions() {
  sys.beginUpdateVertices();
  FloatBuffer vbuf = sys.vertices;
  float pos[] = { 0, 0, 0 };
  for (int n = 0; n < sys.getSize(); n++) {
    vbuf.position(4 * n);
    vbuf.get(pos, 0, 3);  
  
    if (lifetimes[n] == 0) {
      // Respawn dead particle:
      pos[0] = xPos*1.5; 
      pos[1] = yPos;
      pos[2] = 0;
      float a = random(0, TWO_PI);
      float s = random(0.5 * speed, speed);
      velocities[n].x = s * cos(a);
      velocities[n].y = s * sin(a);
      velocities[n].z = 0;  
    } else {
      // Update moving particle.
      pos[0] += velocities[n].x; 
      pos[1] += velocities[n].y;
      pos[2] += velocities[n].z;
      // Updating velocity.
      velocities[n].y += gravity;      
    }
  
    vbuf.position(4 * n);
    vbuf.put(pos, 0, 3);
  }
  vbuf.rewind();
  sys.endUpdateVertices();  
}

void updateColors() {
  sys.beginUpdateColors();
  FloatBuffer cbuf = sys.colors;
  float col[] = { 0, 0, 0, 0 };
  for (int n = 0; n < sys.getSize(); n++) {
    if (0 <= lifetimes[n]) {
      // Interpolating between alpha 1 to 0:
      float a = 1.0 - float(lifetimes[n]) / partLifetime;
    
      col[0] = 1.0;
      col[1] = 1.0;
      col[2] = 1.0;
      col[3] = a;
      
      cbuf.position(4 * n);
      cbuf.put(col, 0, 4);
    }
  }
  cbuf.rewind();
  sys.endUpdateColors();
}

void updateLifetimes() {
  for (int n = 0; n < sys.getSize(); n++) {
    lifetimes[n]++;
    if (lifetimes[n] == partLifetime) {
      lifetimes[n] = 0;
    }    
  }
}  

synchronized void packetEvent(CarnivorePacket packet){
    //println           ("[PDE] packetEvent: " + packet);
    
    xPos=getXfromIP(packet.senderAddress.toString());
    yPos=getYfromIP(packet.senderAddress.toString());
    
    
    nodes.put(packet.senderAddress.toString(), new Node(packet.senderAddress.toString(), packet.senderPort));
    nodes.put(packet.receiverAddress.toString(), new Node(packet.receiverAddress.toString(), packet.receiverPort));
    String from_and_to = packet.senderSocket() + " > " + packet.receiverSocket();
    links.put(from_and_to, new Link(from_and_to));
    
}

synchronized void drawNodes() {
  
  Iterator it = nodes.keySet().iterator();
  while(it.hasNext()){
    String ip = (String)it.next();
    Node n = (Node) nodes.get(ip);
    n.display();
    n.shrink();
  }  
}

synchronized void drawRipples() {
  Iterator it = nodes.keySet().iterator();
  while(it.hasNext()){
    String ip = (String)it.next();
    Node n = (Node) nodes.get(ip);
    n.displayRipple();
  }  
}

synchronized void drawLinks() {
  Iterator it = links.keySet().iterator();
  while(it.hasNext()){
    String from_and_to = (String)it.next();
    Link l = (Link) links.get(from_and_to);

    if(l.greyscale > 25) {
      l.display();
      l.dim();
    }
  }    
}

////////////////////////////////////////////////////////////////////////////

int getXfromIP(String ip) {
  int splitter = ip.lastIndexOf(".");
  int y = int(ip.substring(splitter+1)) * height / 255; // Scale to applet size
  String tmp = ip.substring(0,splitter);
  splitter = tmp.lastIndexOf(".");
  int x = int(tmp.substring(splitter+1)) * width / 255; // Scale to applet size
  if(x<50){x=50;}
   if(x>height-50){x=height-50;}
  return x;
}

int getYfromIP(String ip) {
  int splitter = ip.lastIndexOf(".");
  int y = int(ip.substring(splitter+1)) * height / 255; // Scale to applet size
  if(y<50){y=50;}
  if(y>height-50){y=height-50;}
  return y;
}


String fromIPfromFromTo(String from_and_to) {
  String from         = from_and_to.substring(0, from_and_to.indexOf(" > "));
  String to           = from_and_to.substring(from_and_to.indexOf(" > ")+3);
  String from_ip      = from.substring(0, from.indexOf(":"));
  return from_ip;
}

int fromPortfromFromTo(String from_and_to) {
  String from         = from_and_to.substring(0, from_and_to.indexOf(" > "));
  String to           = from_and_to.substring(from_and_to.indexOf(" > ")+3);
  String from_ip      = from.substring(0, from.indexOf(":"));
  String from_port    = from.substring(from.indexOf(":")+1);
  return int(from_port);
}

String toIPfromFromTo(String from_and_to) {
  String from         = from_and_to.substring(0, from_and_to.indexOf(" > "));
  String to           = from_and_to.substring(from_and_to.indexOf(" > ")+3);
  String from_ip      = from.substring(0, from.indexOf(":"));
  String from_port    = from.substring(from.indexOf(":")+1);
  String to_ip        = to.substring(0, to.indexOf(":"));
  return to_ip;
}

int toPortfromFromTo(String from_and_to) {
  String from         = from_and_to.substring(0, from_and_to.indexOf(" > "));
  String to           = from_and_to.substring(from_and_to.indexOf(" > ")+3);
  String from_ip      = from.substring(0, from.indexOf(":"));
  String from_port    = from.substring(from.indexOf(":")+1);
  String to_ip        = to.substring(0, to.indexOf(":"));
  String to_port      = to.substring(to.indexOf(":")+1);  
  return int(to_port);
}

boolean eitherPortMatches(String from_and_to, int p) {
  int from_port = fromPortfromFromTo(from_and_to);
  int to_port   = toPortfromFromTo(from_and_to);
  if((from_port == p) || (to_port == p)) {
    return true;
  }
  return false;
}

Link.pde

class Link {
  String from_ip;
  String to_ip;
  float greyscale;
  int from_x, from_y, bez1_x, bez1_y, bez2_x, bez2_y, to_x, to_y;

  Link(String from_and_to) {
    this.from_ip = fromIPfromFromTo(from_and_to);
    this.to_ip = toIPfromFromTo(from_and_to);
    this.from_x = getXfromIP(from_ip);
    this.from_y = getYfromIP(from_ip);
    this.to_x = getXfromIP(to_ip);
    this.to_y = getYfromIP(to_ip);
    this.bez1_x = int((from_x + to_x)/2);
    this.bez1_y = from_y;   
    this.bez2_x = to_x;
    this.bez2_y = int((from_y + to_y)/2);   
    this.greyscale = startBrightness;
  }

  void dim() {
    greyscale = greyscale * dimSpeed_link;
  }

  void display() {
    stroke(int(255 - greyscale));
    noFill();
    bezier(from_x, from_y, bez1_x, bez1_y, bez2_x, bez2_y, to_x, to_y);
    
  }
}

Node.pde

class Node  {
  String ip;
  int port;
  float x, y, diameter; 
  color c;
  float ripple_color;
  int ripple_diameter;
  float halo;
  
  Node(String ip, int port) {
    this.port = port;
    this.ip = ip;
    this.x = getXfromIP(ip);
    this.y = getYfromIP(ip);
    this.diameter = currentDiameter;
    this.c = color(255,255,255);
    this.halo = currentHalo;
    this.ripple_color = 100;
    this.ripple_diameter = int(currentDiameter + currentHalo);
  }

  void shrink() {
    if(diameter > min_diameter) { 
      diameter = diameter * shrinkSpeed; 
      halo = halo * shrinkSpeed;
    }
    ripple_color = ripple_color * dimSpeed_ripple;
    ripple_diameter += 30;
  }

  void displayRipple() {
    if((ripple_diameter < (height*2)) && (ripple_color > 2)) { 
      noStroke();
      noFill();
      ellipse(x, y, ripple_diameter, ripple_diameter);  
    }
  }

  void display() {
    
    noStroke();
    fill(c); 
    ellipse(x, y, diameter, diameter);
    noStroke();
    fill(color(100, 100, 100, 50)); 
    ellipse(x, y, diameter + halo, diameter + halo);
    

  }
}

PainterEffect.pde

// Class that encapsulates the painterly effect.
class PainterEffect
{
    PainterEffect(PApplet parent, int n, int w, int h)
    {
        this.parent = parent;
        numParticles = n;
        canvasWidth = w;
        canvasHeight = h;
          
        initParameters();
        createTextures();
        initTextures();    
        createFilters();          
    }
    
    void apply(GLTexture srcTex, GLTexture brushTex, GLTexture destTex, boolean clear, boolean change, float changeTime)
    {
        if (clear) destTex.clear(0, 0, 0, 255);
        updateBrushes(srcTex, change, changeTime);
        drawBrushes(brushTex, destTex);      
    }
    
    void updateBrushes(GLTexture srcTex, boolean change, float changeTime)
    {
        moveFilterSrcTex[0] = posTex.getReadTex();
        moveFilterSrcTex[1] = gradTex.getReadTex();
        moveFilterSrcTex[2] = texfpVel;
        moveFilterSrcTex[3] = texfpNoise;
    
        moveFilter.setParameterValue(0, new float[]{canvasWidth, canvasHeight});
        if (followGrad) moveFilter.setParameterValue(1, 1);
        else moveFilter.setParameterValue(1, 0); 
        moveFilter.setParameterValue(2, velMean);
        moveFilter.setParameterValue(3, noiseMag);         
        moveFilter.apply(moveFilterSrcTex, posTex.getWriteTex());
        posTex.swap();
        
        currentTime = float(millis()) / float(1000);
        
        if ((updateNoiseTime != 0) && (currentTime - lastNoiseUpdateTime >= updateNoiseTime))
        {
            noiseFilter.apply(posTex.getReadTex(), texfpNoise, canvasWidth, canvasHeight, currentTime);
         
            lastNoiseUpdateTime = currentTime;
        }
        
        if (updateColor) 
        {
            colorFilterSrcTex[0] = imageTex.getOldTex();
            colorFilterSrcTex[1] = imageTex.getNewTex();
            colorFilterSrcTex[2] = colorTex.getReadTex();
            colorFilterSrcTex[3] = colorAuxTex.getReadTex();
            colorFilterSrcTex[4] = posTex.getReadTex();
            colorFilterSrcTex[5] = colorCountTex.getReadTex();

            colorFilterDestTex[0] = colorTex.getWriteTex();
            colorFilterDestTex[1] = colorAuxTex.getWriteTex();
            colorFilterDestTex[2] = colorCountTex.getWriteTex();
        
            colorFilter.setParameterValue(0, brushMaxLength);
            colorFilter.setParameterValue(1, changeCoeff);
            colorFilter.setParameterValue(2, brushChangeFrac);            
            colorFilter.setParameterValue(3, brushChangePow);
            colorFilter.apply(colorFilterSrcTex, colorFilterDestTex);
            colorTex.swap();
            colorAuxTex.swap();
            colorCountTex.swap();        
        }   
         
        if (change)
        {
            println("Start changing image...");

            // Preprocessing filter is applied to the source image to generate the new image.
            imgFilter.apply(srcTex, imageTex.getNewTex());

            // The gradient of the new image is calculated and stored in first texture of newGradTex.
            newGradTex.setWriteTex(0);
            gradFilterSrcTex[0] = imageTex.getNewTex();
            gradFilterSrcTex[1] = texfpRand;
            gradFilter.apply(gradFilterSrcTex, newGradTex.getWriteTex());

            // Initializing variables to control transition between old and new image/gradient.
            changeCoeff = 0.0;       // Linear interpolation coefficient.
            swapedImageTex = false;

            // Used to control the averaging of the gradient of the new image during the transition
            // period.
            newGradTex.init();
        
            lastChangeTime = currentTime;
        }

        if ((0.0 <= changeCoeff) && (changeCoeff < 1.0))
        {
            // Updating linear interpolation coefficient.
            changeCoeff = (currentTime - lastChangeTime) / changeTime;
            if (1.0 < changeCoeff) changeCoeff = 1.0;
        }
        else if (!swapedImageTex)
        {
            // Transition period is finished.
            println("...done changing image.");
            imageTex.swap();
            swapedImageTex = true;
            changeCoeff = -1.0; // With this value, the shaders don't enter into the transition mode.
        }

        aveCount++;
        if (aveCount == aveInterval)
        {
            // Gradient average.
            aveCount = 0;
            for (int n = 0; n < numAveIter; n++)
            {
                aveGradFilterSrcTex[0] = gradTex.getReadTex();
                aveGradFilterSrcTex[1] = texfpRand; 
                aveGradFilterSrcTex[2] = newGradTex.getReadTex();
            
                aveGradFilter.setParameterValue(0, changeCoeff);
            
                if (changeCoeff == -1) 
                {
                    aveGradFilter.apply(aveGradFilterSrcTex, gradTex.getWriteTex());
                }
                else 
                {
                    aveGradFilterDestTex[0] = gradTex.getWriteTex();
                    aveGradFilterDestTex[1] = newGradTex.getWriteTex();
                    aveGradFilter.apply(aveGradFilterSrcTex, aveGradFilterDestTex);
                }
            
                gradTex.swap();
                newGradTex.swap();
            }
        }
    }
    
    void drawBrushes(GLTexture brushTex, GLTexture destTex)
    {
        brushesFilterSrcTex[0] = gradTex.getReadTex();
        brushesFilterSrcTex[1] = brushTex;
        brushesFilterSrcTex[2] = colorTex.getReadTex();
        brushesFilterSrcTex[3] = posTex.getReadTex();
        
        if (blendBrushes) brushesFilter.setBlendMode(blendMode);
        else brushesFilter.noBlend();
        brushesFilter.setParameterValue(0, brushSize);
        brushesFilter.apply(brushesFilterSrcTex, destTex);    
    }

    void initParameters()
    {
        setDefParameters();
 
        aveCount = 0;
        lastChangeTime = -1;
        changeCoeff = -1.0;
        lastNoiseUpdateTime = -1;

        startClock = millis();    
    }
    
    void createTextures()
    {
        GLTextureParameters floatTexParams = new GLTextureParameters();
        floatTexParams.minFilter = GLTexture.NEAREST_SAMPLING;
        floatTexParams.magFilter = GLTexture.NEAREST_SAMPLING;        
        floatTexParams.format = GLTexture.FLOAT;
    
        imageTex = new GLTexturePingPong(new GLTexture(parent, canvasWidth, canvasHeight), 
                                         new GLTexture(parent, canvasWidth, canvasHeight));    
    
        posTex = new GLTexturePingPong(new GLTexture(parent, numParticles, floatTexParams), 
                                       new GLTexture(parent, numParticles, floatTexParams));        
    
        gradTex = new GLTexturePingPong(new GLTexture(parent, canvasWidth, canvasHeight, floatTexParams), 
                                        new GLTexture(parent, canvasWidth, canvasHeight, floatTexParams));         
    
        newGradTex = new GLTexturePingPong(new GLTexture(parent, canvasWidth, canvasHeight, floatTexParams), 
                                           new GLTexture(parent, canvasWidth, canvasHeight, floatTexParams));     
    
        colorTex = new GLTexturePingPong(new GLTexture(parent, numParticles, floatTexParams), 
                                         new GLTexture(parent, numParticles, floatTexParams));

        colorAuxTex = new GLTexturePingPong(new GLTexture(parent, numParticles, floatTexParams), 
                                            new GLTexture(parent, numParticles, floatTexParams));
    
        colorCountTex = new GLTexturePingPong(new GLTexture(parent, numParticles, floatTexParams), 
                                              new GLTexture(parent, numParticles, floatTexParams));

        int w = posTex.getReadTex().width;
        int h = posTex.getReadTex().height;

        texfpVel = new GLTexture(parent, w, h, floatTexParams);
        texfpRand = new GLTexture(parent, canvasWidth, canvasHeight, floatTexParams);
        texfpNoise = new GLTexture(parent, w, h, floatTexParams);
    
        moveFilterSrcTex = new GLTexture[4];
        colorFilterSrcTex = new GLTexture[6];
        colorFilterDestTex = new GLTexture[3];
        gradFilterSrcTex = new GLTexture[2];
        aveGradFilterSrcTex = new GLTexture[3];
        aveGradFilterDestTex = new GLTexture[2];
        brushesFilterSrcTex = new GLTexture[4];
    
        println("Size of particles box: " + w + "x" + h);
        println("Number of particles: " + w * h);    
    }

    void initTextures()
    {
        int pix[] = new int[canvasWidth * canvasHeight];
        for (int k = 0; k < canvasWidth * canvasHeight; k++) pix[k] = 0xff000000;

        imageTex.getOldTex().putBuffer(pix);
        imageTex.getNewTex().putBuffer(pix);
    
        posTex.getReadTex().setRandom(0, canvasWidth, 0, canvasHeight, 0, 0, 0, 0);
        posTex.getWriteTex().setRandom(0, canvasWidth, 0, canvasHeight, 0, 0, 0, 0);  
    
        texfpVel.setRandom(velCoeffMin, velCoeffMax, 0, 0, 0, 0, 0, 0);
    
        texfpRand.setRandomDir2D(1.0, 1.0, 0.0, TWO_PI);
    
        texfpNoise.setRandomDir2D(0.0, 1.0, 0.0, TWO_PI);

        colorTex.getReadTex().setZero();
        colorTex.getWriteTex().setZero();
    
        colorAuxTex.getReadTex().setZero();
        colorAuxTex.getWriteTex().setZero();
    
        colorCountTex.getReadTex().setRandom(0, 0, brushMinLengthCoeff, brushMaxLengthCoeff, 0, 0, 0, 0);
        colorCountTex.getWriteTex().setRandom(0, 0, brushMinLengthCoeff, brushMaxLengthCoeff, 0, 0, 0, 0); 
    
        gradTex.getReadTex().setZero();
        gradTex.getWriteTex().setZero();
        newGradTex.getReadTex().setZero();
        newGradTex.getWriteTex().setZero();
    }

    void createFilters()
    {
        moveFilter = new GLTextureFilter(parent, "MovePart.xml");      // Compatible with NVidia GeForce 8x00 and newer.
        //moveFilter = new GLTextureFilter(parent, "MovePart-preGF8.xml"); // Compatible with NVidia video cards previous to GeForce 8x00.
    
        colorFilter = new GLTextureFilter(parent, "ColorPart.xml");      // Compatible with NVidia GeForce 8x00 and newer.
        //colorFilter = new GLTextureFilter(parent, "ColorPart-preGF8.xml"); // Compatible with NVidia video cards previous to GeForce 8x00.
    
        imgFilter = new GLTextureFilter(parent, "Blur.xml");
    
        gradFilter = new GLTextureFilter(parent, "RenderGrad2fp.xml");
    
        aveGradFilter = new GLTextureFilter(parent, "RenderAveGrad.xml");

        noiseFilter = new SimplexNoiseFilter(parent, "SimplexNoise.xml");

        brushesFilter = new GLTextureFilter(parent, "RenderBrushes.xml");
    }
        
    void setDefParameters()
    {
        brushSize = 5.0;
        
        brushMaxLength = 10;
        brushMinLengthCoeff = 0.8;
        brushMaxLengthCoeff = 1.2;
        brushChangeFrac = 3.0;
        brushChangePow = 1.0;
        
        velMean = 1.0;
        velCoeffMin = 0.8;
        velCoeffMax = 1.2;
        updateNoiseTime = 0.1;
        numAveIter = 1;
        aveInterval = 2;
        followGrad = true;
        updateColor = true;
        noiseMag = 1.0;
        blendBrushes = true;
        blendMode = BLEND;  
    }    
    
    PApplet parent;
    int numParticles;
    int canvasWidth, canvasHeight;
    float brushSize;
    int brushMaxLength;
    float brushMinLengthCoeff;
    float brushMaxLengthCoeff;
    float brushChangeFrac;
    float brushChangePow;
    float velMean;
    float velCoeffMin;
    float velCoeffMax;
    float updateNoiseTime;
    int numAveIter;
    int aveInterval;
    boolean followGrad;
    boolean updateColor;
    float noiseMag;
    boolean blendBrushes;
    int blendMode;
    
    float currentTime, lastChangeTime, changeCoeff, lastNoiseUpdateTime;
    boolean swapedImageTex;
    int aveCount;
    int startClock;
    
    GLTexturePingPong imageTex, posTex, gradTex, newGradTex, colorTex, colorAuxTex, colorCountTex;
    GLTexture texfpVel, texfpRand, texfpNoise; 

    GLTexture[] moveFilterSrcTex;
    GLTexture[] colorFilterSrcTex;
    GLTexture[] colorFilterDestTex;
    GLTexture[] gradFilterSrcTex;
    GLTexture[] aveGradFilterSrcTex;
    GLTexture[] aveGradFilterDestTex;
    GLTexture[] brushesFilterSrcTex;

    GLTextureFilter moveFilter, colorFilter, imgFilter, gradFilter, aveGradFilter, brushesFilter;
    SimplexNoiseFilter noiseFilter;    
}

SimplexNoiseFilter.pde

// This filter generates Perlin noise on the GPU.
class SimplexNoiseFilter extends GLTextureFilter
{
    public SimplexNoiseFilter(PApplet parent, String filename)
    {
        this.parent = parent;

        super.initFilter(filename);

        createPermTex();
        createSimplexTex();        
        createGradTex();
        
        srcTexArray = new GLTexture[4];
        srcTexArray[0] = permTex;
        srcTexArray[1] = simplexTex;
        srcTexArray[2] = gradTex;
    }
    
    public void apply(GLTexture srcTex, GLTexture destTex, int w, int h, float time)
    {
        srcTexArray[3] = srcTex;
        setParameterValue(0, time);
        setParameterValue(1, 1.0 / (w * h));
        super.apply(srcTexArray, destTex);
    }

    protected void createPermTex()
    {    
        permTex = new GLTexture(parent, 256, 256);
        
        int buffer[] = new int[256 * 256 * 4];
        for (int i = 0; i < 256; i++)
            for (int j = 0; j < 256; j++) 
            {
                int offset = (i * 256 + j) * 4;
                int value = perm[(j + perm[i]) & 0xFF];
                buffer[offset] = grad3[value & 0x0F][0] * 64 + 64;   // Gradient x
                buffer[offset+1] = grad3[value & 0x0F][1] * 64 + 64; // Gradient y
                buffer[offset+2] = grad3[value & 0x0F][2] * 64 + 64; // Gradient z
                buffer[offset+3] = value;                            // Permuted index
            }        
        permTex.putIntBuffer(buffer, ARGB);
    }
    
    protected void createSimplexTex()
    {
        GLTextureParameters simplexTexParams = new GLTextureParameters();
        simplexTexParams.target = GLTexture.TEX_ONEDIM;
        simplexTex = new GLTexture(parent, 64, 1, simplexTexParams);      
        
        int buffer[] = new int[64 * 4];
        for (int i = 0; i < 64; i++)
        {
            int offset = i * 4;
            buffer[offset] = simplex4[i][0];
            buffer[offset+1] = simplex4[i][1];
            buffer[offset+2] = simplex4[i][2];
            buffer[offset+3] = simplex4[i][3];
        }
        simplexTex.putIntBuffer(buffer, ARGB);
    }    
      
    protected void createGradTex()
    {
        gradTex = new GLTexture(parent, 256, 256);
        
        int buffer[] = new int[256 * 256 * 4];
        for (int i = 0; i < 256; i++)
            for (int j = 0; j < 256; j++) 
            {
                int offset = (i * 256 + j) * 4;
                int value = perm[(j + perm[i]) & 0xFF];
                buffer[offset] = grad4[value & 0x1F][0] * 64 + 64;   // Gradient x
                buffer[offset+1] = grad4[value & 0x1F][1] * 64 + 64; // Gradient y
                buffer[offset+2] = grad4[value & 0x1F][2] * 64 + 64; // Gradient z
                buffer[offset+3] = grad4[value & 0x1F][3] * 64 + 64; // Gradient z
            }
        gradTex.putIntBuffer(buffer, ARGB);
    }
    
    GLTexture[] srcTexArray;
    GLTexture permTex, simplexTex, gradTex;
    GLTextureParameters simplexTexParams;
    
    int perm[]= {151,160,137,91,90,15,
                 131,13,201,95,96,53,194,233,7,225,140,36,103,30,69,142,8,99,37,240,21,10,23,
                 190, 6,148,247,120,234,75,0,26,197,62,94,252,219,203,117,35,11,32,57,177,33,
                 88,237,149,56,87,174,20,125,136,171,168, 68,175,74,165,71,134,139,48,27,166,
                 77,146,158,231,83,111,229,122,60,211,133,230,220,105,92,41,55,46,245,40,244,
                 102,143,54, 65,25,63,161, 1,216,80,73,209,76,132,187,208, 89,18,169,200,196,
                 135,130,116,188,159,86,164,100,109,198,173,186, 3,64,52,217,226,250,124,123,
                 5,202,38,147,118,126,255,82,85,212,207,206,59,227,47,16,58,17,182,189,28,42,
                 223,183,170,213,119,248,152, 2,44,154,163, 70,221,153,101,155,167, 43,172,9,
                 129,22,39,253, 19,98,108,110,79,113,224,232,178,185, 112,104,218,246,97,228,
                 251,34,242,193,238,210,144,12,191,179,162,241, 81,51,145,235,249,14,239,107,
                 49,192,214, 31,181,199,106,157,184, 84,204,176,115,121,50,45,127, 4,150,254,
                 138,236,205,93,222,114,67,29,24,72,243,141,128,195,78,66,215,61,156,180};  

    /* These are Ken Perlin's proposed gradients for 3D noise. I kept them for
       better consistency with the reference implementation, but there is really
       no need to pad this to 16 gradients for this particular implementation.
       If only the "proper" first 12 gradients are used, they can be extracted
       from the grad4[][] array: grad3[i][j] == grad4[i*2][j], 0<=i<=11, j=0,1,2
    */
    int grad3[][] = {{0,1,1},{0,1,-1},{0,-1,1},{0,-1,-1},
                     {1,0,1},{1,0,-1},{-1,0,1},{-1,0,-1},
                     {1,1,0},{1,-1,0},{-1,1,0},{-1,-1,0}, // 12 cube edges
                     {1,0,-1},{-1,0,-1},{0,-1,1},{0,1,1}}; // 4 more to make 16

    /* These are my own proposed gradients for 4D noise. They are the coordinates
       of the midpoints of each of the 32 edges of a tesseract, just like the 3D
       noise gradients are the midpoints of the 12 edges of a cube.
    */
    int grad4[][]= {{0,1,1,1}, {0,1,1,-1}, {0,1,-1,1}, {0,1,-1,-1}, // 32 tesseract edges
                    {0,-1,1,1}, {0,-1,1,-1}, {0,-1,-1,1}, {0,-1,-1,-1},
                    {1,0,1,1}, {1,0,1,-1}, {1,0,-1,1}, {1,0,-1,-1},
                    {-1,0,1,1}, {-1,0,1,-1}, {-1,0,-1,1}, {-1,0,-1,-1},
                    {1,1,0,1}, {1,1,0,-1}, {1,-1,0,1}, {1,-1,0,-1},
                    {-1,1,0,1}, {-1,1,0,-1}, {-1,-1,0,1}, {-1,-1,0,-1},
                    {1,1,1,0}, {1,1,-1,0}, {1,-1,1,0}, {1,-1,-1,0},
                    {-1,1,1,0}, {-1,1,-1,0}, {-1,-1,1,0}, {-1,-1,-1,0}};

    /* This is a look-up table to speed up the decision on which simplex we
       are in inside a cube or hypercube "cell" for 3D and 4D simplex noise.
       It is used to avoid complicated nested conditionals in the GLSL code.
       The table is indexed in GLSL with the results of six pair-wise
       comparisons beween the components of the P=(x,y,z,w) coordinates
       within a hypercube cell.
       c1 = x>=y ? 32 : 0;
       c2 = x>=z ? 16 : 0;
       c3 = y>=z ? 8 : 0;
       c4 = x>=w ? 4 : 0;
       c5 = y>=w ? 2 : 0;
       c6 = z>=w ? 1 : 0;
       offsets = simplex[c1+c2+c3+c4+c5+c6];
       o1 = step(160,offsets);
       o2 = step(96,offsets);
       o3 = step(32,offsets);
      (For the 3D case, c4, c5, c6 and o3 are not needed.)
    */
    int simplex4[][] = {{0,64,128,192},{0,64,192,128},{0,0,0,0},
                        {0,128,192,64},{0,0,0,0},{0,0,0,0},{0,0,0,0},{64,128,192,0},
                        {0,128,64,192},{0,0,0,0},{0,192,64,128},{0,192,128,64},
                        {0,0,0,0},{0,0,0,0},{0,0,0,0},{64,192,128,0},
                        {0,0,0,0},{0,0,0,0},{0,0,0,0},{0,0,0,0},
                        {0,0,0,0},{0,0,0,0},{0,0,0,0},{0,0,0,0},
                        {64,128,0,192},{0,0,0,0},{64,192,0,128},{0,0,0,0},
                        {0,0,0,0},{0,0,0,0},{128,192,0,64},{128,192,64,0},
                        {64,0,128,192},{64,0,192,128},{0,0,0,0},{0,0,0,0},
                        {0,0,0,0},{128,0,192,64},{0,0,0,0},{128,64,192,0},
                        {0,0,0,0},{0,0,0,0},{0,0,0,0},{0,0,0,0},
                        {0,0,0,0},{0,0,0,0},{0,0,0,0},{0,0,0,0},
                        {128,0,64,192},{0,0,0,0},{0,0,0,0},{0,0,0,0},
                        {192,0,64,128},{192,0,128,64},{0,0,0,0},{192,64,128,0},
                        {128,64,0,192},{0,0,0,0},{0,0,0,0},{0,0,0,0},
                        {192,64,0,128},{0,0,0,0},{192,128,0,64},{192,128,64,0}};    
}

Final set of sounds used

1. James speaking as dog scratches himself and his collar jiggles
2. James beatboxing
3. Jacki talking
4. paper crackling
5. drink gatorade parody from tv
6. rubber band being plucked

There’s a lot of amplification, reverb, compression echo, modulation and pitch changes on each of these to get the final result

Here are a few samples of our recordings

Archana – extract from ‘Alice in Wonderland’ by Lewis Carroll

Shotgun Microphone:

Logitech USB Microphone:

Yeti Microphone:

Matt – extract from the poem ‘This is Just to Say’ by William Carlos William

Shotgun Microphone:

Logitech USB Microphone:

Yeti Microphone:

Fingers snapping

Shotgun Microphone:

Logitech USB Microphone:

Yeti Microphone:

Music played from a phone speaker that is moved around space

Shotgun Microphone:

Logitech USB Microphone:

Yeti Microphone:

Christie and I are working on developing a smoke detector that identifies a change in cigarette smoke in the atmosphere. This concept was originally developed for the purposes of detecting secondhand smoke in the Philippines, Malaysia and more recently, the UAE. Since this is ultimately about changing behavior in users, we see this project being deployed on mobile applications.

We are in the process of developing functioning circuits with various IR sensors to figure out functionality, sensitivity and calibration, but we have identified two other major phases of development:

1) Transmitting the data received by the IR sensor through the microphone port in a mobile phone and developing a basic interface for this information to be displayed/accessed.

2) Gathering all the data and making it more accessible to citizens as well as other agencies.

Our goals for this class are to get the detector, wireframe and barebones application functioning to open the product up to developers here and abroad.

Research

The Human Nose

While the human nose’s sense of smell is feeble compared to that of many animals, it is still very acute, sensitive and complex organic sensor that can tell the difference between 4000-10,000 smells. Smell is detected when odors molecules touch receptors on the olfactory nerves that then transmit the electrical impulses to the brain.

Existing Technologies to detect smoke

A number of technologies do already exist to detect smoke that employ a variety of principles.

1. Air Quality Monitor

—Air Quality Monitors use a tin dioxide semiconductor sensor to detect oxidizable gases. —They are designed to have high sensitivity to gaseous organic materials like cigarette smoke.

—A heated element inside a porous semiconductive tube that has a large surface area freely absorbs gas molecules on the semiconductor surface. Electron transfer occurs between the gas molecules and the already absorbed oxygen molecules causing a relatively large increase in conductivity for a small change in gas concentration.

Example:  Ambient Volatile Organic Compound (VOC) Monitor

The VOC Monitor —monitors ambient air quality by detecting the VOC content in the air. It is  —compact, accurate, and retails at about $35. It —requires a computer, USB port for power and specific Microsoft Windows software for detailed measurements of air quality. —On a computer, the user can see a graph of changes in air quality over time.

Specs:

—VOC’s Detected: Alcohols, aldehydes, ketones, organic acids, aliphatic and aromatic hydrocarbons.
—Sensor: MEMS metal oxide semiconductor.
—Power: Powered from PC USB port.
—Temperature Limits: 32 to 122°F (0 to 50°C).

2. Optical Smoke Detectors

Photoelectric light scattering smoke detectors, utilize a light source in order to detect obscuration
require a chamber, a photodiode, and an infrared light-emitting diode. In clean air, light passes in front of the detector in a straight line. When smoke enters the optical chamber across the path of the light beam, some light is scattered by the smoke particles, directing it at the sensor.

3. Infrared Photo Detector and Emitter

IR sensors work by producing a small beam of light between the emitter and detector across a gap exposed to the air. When small particulates cross the light beam, such as those present in cigarette smoke, the sensor detects the fluctuation in light and can send a signal to a mobile application.They are very inexpensive – usually under $5.00 and are unrestricted in terms of their circuitry as well as free of proprietary concerns. We are currently testing prototypes for sensitivity compared to the AQStick. Provided we can maintain a high degree of accuracy, this would be a superior source for a mobile application

Example: Sharp Infrared sensor

Similar to an infrared photo detector and emitter, the Sharp Infrared sensor relies on triangulation of light rather than a direct beam to give a reading. A beam of light has a certain angle of refraction in a non-smoky environment. When VOC particulates are encountered, the angle of refraction changes and the sensor reads that change.

Our Next Steps

Test out infrared sensor circuits and to get a reading which we are able to send out through serial communication using a microprocessor.

rice cooker  [the rice cooker]

ricecooker/on [power to the rice cooker]
ricecooker/off [no power]

ricecooker/waterdispenser/add [the rice cooker has a water dispenser unit. Add water when power off]
ricecooker/on/waterdispenser/add [add water to unit when power is on]

ricecooker/on/waterdispencer/amountofwater [when power is on, displays the amount of water in the dispenser]

ricecooker/off/openlid [opens the rice cooker lid when power is off]
ricecooker/on/openlid [opens the rice cooker lid when power is on]

ricecooker/off/closelid [closes the rice cooker lid when power is off]
ricecooker/on/closelid [closes the rice cooker lid when power is on]

if ricecooker/on/cook/(anything), then ricecooker/on/closelid CANNOT function. i.e. openlid can only function when the power is just on/off and no other state is being called

ricecooker/on/closelid/cook [when power is on and the lid is closed, display cook options]

ricecooker/on/closelid/cook/keepwarm [when power is on and the lid is closed, keep warm at fixed temperature]

ricecooker/off/openlid/addrice [when power is off and lid is open, add rice]
ricecooker/on/openlid/addrice [when power is on and lid is open, add rice]

ricecooker/on/closelid/cook/brownrice [ power is on, lid is closed, cook brown rice is selected]
ricecooker/on/closelid/cook/brownrice/amount [ power is on, lid is closed, cook brown rice is selected, amount of cups present is displayed]
ricecooker/on/closelid/cook/brownrice/dispenseWater
[ power is on, lid is closed, cook brown rice is selected, dispenses water based on the number of cups]

ricecooker/on/closelid/cook/brownrice/error/not-enough-water-in-dispencer
[displays an error if there is not enough water to cook rice]

ricecooker/on/closelid/cook/brownrice/error/insufficient-rice
[displays an error if there is less than 1 cup of rice]

ricecooker/on/closelid/cook/brownrice/error/too-much-rice
[displays an error if there is more than 4 cups of rice]

ricecooker/on/closelid/cook/brownrice/ricecooking/
[start cooking brown rice]

ricecooker/on/closelid/cook/brownrice/ricecooking/timetoCompletion
[time till brown rice is finished cooking]

ricecooker/on/closelid/cook/brownrice/ricecooking/complete
[brownrice is cooked]

[similarly for white rice]
ricecooker/on/closelid/cook/whiterice
ricecooker/on/closelid/cook/whiterice/amount
ricecooker/on/closelid/cook/whiterice/dispenseWater
ricecooker/on/closelid/cook/whiterice/error/notenoughwater
ricecooker/on/closelid/cook/whiterice/error/notenoughrice
ricecooker/on/closelid/cook/whiterice/error/toomuchrice
ricecooker/on/closelid/cook/whiterice/ricecooking/timetoCompletion
ricecooker/on/closelid/cook/whiterice/ricecooking/complete

[similarly for steaming]
ricecooker/off/openlid/steamtray/add
ricecooker/off/openlid/steamtray/addvegetables

ricecooker/on/closelid/cook/steam/error/no-steamtray-present

ricecooker/on/closelid/cook/steam
ricecooker/on/closelid/cook/steam/dispenseWater

ricecooker/on/closelid/cook/steam/error/notenoughwater

ricecooker/on/closelid/cook/steam/countdowntosteamstart
[displays the time it takes for the water to come to boil and start steaming]

ricecooker/on/closelid/cook/steam/time
[displays the time the contents have been steaming for]

ricecooker/on/closelid/cook/steam/end
[stop steaming]

We bought this amazing underwater mic for $25 called the Super Spy Phone.

Refraction of Sound

We found that with no liquid, the sound was pretty consistent at around 20hz, with some variation because of the mic and the plastic.

In water, we saw both more higher and lower tones in the wave, but the tones were relatively adjacent to the original 20hz.

The wine was very similar to the water, with slightly higher level of higher tones.
Then, with the beer, we suddenly saw a tone appear in the spectrum analysis that was around 250hz. This had never appeared before. All of the other tones had been between 20 and 100. Additionally there appeared a very slight increase in the tones above 500.

The milk was actually the cleanest tone, with almost all of the sound right around 20hz.
The honey showed an even split between three tones all around 20hz.

We would suggest that the carbonation in the beer, which was lacking in all of the other liquid, was what caused the dramatic difference in tones.

Visualization

To see the waves in the various liquids, we placed a small amount of the liquid in a plastic cup measure and placed it on the speaker.

  tge0-1-0-0.core1.nyw.ny.rcn.net (207.172.15.84)

  tge0-0-0-0.core1.nyw.ny.rcn.net (207.172.15.75) tge0-1-0-2.core2.phdl.pa.rcn.net (207.172.19.20) tge0-0-0-2.core2.phdl.pa.rcn.net (207.172.19.228) tge0-1-0-2.core1.phdl.pa.rcn.net (207.172.19.216) tge0-0-0-2.core2.lnh.md.rcn.net (207.172.19.11) tge0-1-0-3.core2.lnh.md.rcn.net (207.172.19.225) tge0-1-0-3.core2.lnh.md.rcn.net (207.172.19.225) tge1-4.border1.eqnx.va.rcn.net (207.172.19.179) tge2-3.border1.eqnx.va.rcn.net (207.172.19.7) tge2-4.border1.eqnx.va.rcn.net (207.172.19.205) equinix-iad11.netarch.akamai.com (206.223.115.102) ge-4-1-0.mpr1.iad10.us.mfnx.net (206.223.115.86) a72-247-242-50.deploy.akamaitechnologies.com (72.247.242.50) 209.66.92.238.t00817-32.above.net (209.66.92.238) ae11.bb01.iad1.tfbnw.net (74.119.78.68) ae8.bb02.frc1.tfbnw.net (31.13.24.46) ae2.dr04.frc1.tfbnw.net (31.13.27.82) po1022.csw03a.frc1.tfbnw.net (31.13.25.203) ]]> http://itp.nyu.edu/~apk284/blog/understanding-networks/2012/10/02/traceroutes/feed/ 0