Category: Computational Media

First semester work in computational media.

Final Project – TweetBots need your emotions

TweetBots is a project that involves 2 Braitenberg inspired Bots, acting and reacting under the influence of human emotions.
The initial attractor for developing this idea into a project was effective causality resulting in unpredictable and unprogrammed behaviours.
I was very interested in these hopefully unforeseen physical manifestations defined by what would appear to be basic intelligence. At this point I started to narrow down the basic cause and effect relationship of an intelligence based on reactions to certain stimuli. It was clear to me that if something appeared to be intelligent enough to avoid or be attracted to a certain environment, then clearly it would appear to be “alive” and hence possessing of very basic intelligence.
The stimuli I chose to isolate were SOUND, LIGHT, SMELL and TEMPERATURE. This is mostly due to the availability of electronic sensors capable of evaluating and transmitting these qualities in an environment as data.
For this particular experiment I chose LIGHT as the the main and only stimulus.
This seemed like a fairly good approach but it had limitations. The Bots appeared to be functioning on a basic level but were limited to one specific behaviour- either attraction or repulsion to light.
I started wondering how this behaviour could be initiated via human influence and even could this be done remotely. Could these bots act based on remote input?
However, the more I thought about this, the more I realized that there was an inherent contradiction in this idea. If I wished for the Bots to have a certain essence of autonomy, after all they were “alive” , is it possible at all to bring human influence in a way that doesn’t involve a direct input. I wanted to the Bots to sense their environment, but I was also interested in a more advanced way of controlling them, specifically a control that is not only remote but that doesn’t involve human interaction based specifically on a stimulus, LIGHT.
How can I have human input that is not just someone holding/withholding a light source to disrupt and influence the bots?
I chose Twitter as the source of this influence. The reasoning was that tweets can not only contain inane pointless statements but also can be a great platform for simple emotional expression. I chose to focus on these emotional expressions. How can I convert a tweet, for example ” I am so happy, I just got accepted to ITP” to a command that the Bots can execute in a physical space.
I was concerned with creating a database of “happy” words, and “sad” words, mostly due to the conflict of taking them out of context. I have to admit, there is something conceptually very interesting about losing the meaning of a tweet in translation, and misinterpreting it but for this specific project I wanted this to be as close to the original intention as possible.
The solution was fairly simple- look for the words “sad” and “happy”. More often than not , these words will illustrate their true meaning in a sentence, and hence the minimize the chance of misinterpretation. I wanted to avoid making SchadenFreude Bots.
So the basic scheme of this experiment becomes this. An origin of tweets being parsed is set as the longituted and latitude of ITP, with anything within 10mile radius being parsed for “active” words.
If a “sad” is in a tweet, it gets extracted and sent to the bot, the bot then switches his behaviour to “photophobic”, however if “happy” is present, the bots become “photophilic”
Each bot contains a RGB led that changes color based on the word present acting as a status, and also functions as a stimulus(light source) to provoke a response in the bots to their ever changing environment. Ultimately the bots roam around in a physical space, changing their physical proximity to each other based on the virtual emotional space around them(Twitter). The resulting behaviour is fascinating.

BUILD
*parts were mostly sourced from Polulu, Sparkfun and Jameco.

- Tamiya 70097 Twin-Motor Gearbox Kit at a ratio of 204:1
- Round robot chassis from Polulu( amazing for balance)
- Tamiya toy tires
- SN754410 Quad Half H-Bridge( for controlling each motor individually)
- Arduino Pro 328 – 5V/16MHz
- 2.4GHz XBee
- photocells
- rgb led
- round acrylic half globe(Canal Plastics)
- various wires and breadboard

CODE
-PHP using Twitter API php code(TwitterSearch) for querying for tweets based on “ID” and geographic location
-Processing to parse and define “happy” and “sad”
-Arduino to receive “happy” and “sad” messages and control Bots and behaviour

Download Code

Code and other bit Thanks to Eric Mika, Daniel Shiffman, Heather Dewey Hagborg, Rory Nugent and Tom Igoe.

Tweet Bots/ Emotion to Motion

This is a further development/revision on Intelligent Emotional Bots project.
The overall behaviour characteristics are heavily inspired by Valentino Braitenberg’s ideas in Vehicles, Experiments in Synthetic Psychology.

Theory
The goal is to create fairly sophisticated and intelligent cause and effect behaviours.
The bots initial cue for emotional response is dependent on human emotion, specifically shorthand expression for emotional states-emoticons.
Thus the bots are influenced and interfered by human interaction rather than being controlled directly.
The focus of this project is the re-interpretation of emotional content into motion consequently resulting in emphatic emotional bots that have characteristically complex and organic behaviours.

Construct
The project consists of two bots.
Bots are 3 wheel with two rear wheels driven by a twin motor gearbox at a 58:1 ratio.
Photosensors are connected individually to each motor and logically to each wheel.
Communication is achieved via XBee radios.

Interaction Logic
Twitter is set up as permission via OAuth to an app called Emotbot sampling from 60 ITP student Twitter accounts.
PHP is linked to Emotbot and listens for messages containing emoticons- :) :(
If found, message is parsed and this is sent to Processing.
Processing sends “emoticon” via serial wireless through XBees to an Arduino- random function.
Arduino receives and interprets message as specific behaviour.
Behaviours are either photophillic or photophobic.
Receiving a :) results in a photophilic behaviour.
Receiving a :( results in a photophobic behaviour.

IntelligentEmotional? Bots

e·mo·tion (-mshn)
n.
[French émotion, from Old French, from esmovoir, to excite, from Vulgar Latin *exmovre : Latin ex-, ex- + Latin movre, to move; see meu- in Indo-European roots.]

This project will involve using information acquired via Twitter to control physical bots.

The aim is to have a bots that will be driven by mental states that arise spontaneously rather than consciously.
It may be equally as important to consider both if those moods arise from artificially created cause and effect circuitry that mimic biological reactions or from emotions parsed from twitter feeds into commands for kinesis.
This is an experiment that aims to explore these two areas; The function of anthropomorphism, and consequent modes of control.
The ultimate goal is to create something that feels “alive” rather than constructed and based solely on Aristotelian physics(flawed?) and explore the close ties between perturbations in physical motion and emotional states.

A few points I want to explore as I move forward:
Effective intelligence based on cause and effect behaviours triggered by either circuit/sensor or Twitter?
Importance of the bots to be aware of each other/seeking each other?
Bots to act as one unit/is this beneficial?
Is there a way of outputting emotional information externally as a guide to spectators for the bots mood?
Is the motion useful?
How much human involvement?
Clarity of commands i.e (turn left vs Kurosawa Hidden Fortress is great)?

M5 Bus Ride

I wanted this assignment to function as an experiment for answering specific questions.

How can this trip be functional?
Can it provide more than just experience?
If so, how can I attempt to modify an experience along a route that is set by the MTA?
Is there different way of thinking about the journey, in terms of specific values?
If I were to collect a data set, how different would my experience be?
Would it make a difference, if I use a different mode of of moving through the physical world?

As an experiment, I decided to use my bicycle and my own power to cover the entire distance. The interest and fascination in this experiment lies partly in the idea of using something of the everyday, in my case a bicycle and use it out of it’s usual context of the familiar streets that lead me to Brooklyn. This also brings an interesting point of psycho geography, and the freedom and levity a bicycle allows one. It both lets you move rapidly but does not limit you to the road. A bicycle can be used on various surfaces- the sidewalk, grass, water and mud to name a few.

A very interesting side-effect I wish to explore more, is the immense effect I  found one bicycle can have on road patterns, other vehicles and people.  In this specific sense riding a bicycle affects the experience of many people who travel along the road. This reminds me Vito Acconci’s pieces of following strangers on the street, and how his actions were able to influence set patterns. More to come on this, once I gather enough documentation.

Like any experiment it is useful to set limitations, especially when this experiment involved gathering data in various forms. To discipline my ride I set the following rules:
1.) never stop(brakes were removed), to rest, for traffic, lights or pedestrians.

2.) follow the bus route as closely as possible.

3.) In case of an accident, recover as quickly as possible and continue on the set route.

4.) create a record of the ride (data as video and coordinates).

I followed the M5 bus route starting at Houston and 6th Avenue. My original intention was to ride along with the bus.  However as the bus, was both limited by its size within traffic and tasked with making stops every couple of blocks,  it soon became apparent that this would not be a viable option since it would be effectively breaking the rules I had set for myself.

My overall time from ORIGIN (Houston St and 6th Av) to DESTINATION (George Washington Bridge) was 43 min and 23 seconds. This includes time lost from 2 wrong turns on the route,  and an accident/collision with a taxi cab. The trip was documented with a rigged helmet cam consisting of an Sanyo Xacti CG10. Longitude and Latitude coordinates together with Altitude and Time were logged using my Android phone and Offline Logger application at 1 second sample size.

As further documentation I put together a video of my experience. The video is of two parts. The first showcasing the entirety of the route via satellite view and the second the actual detail of the route as it happened. This part has been time-remapped intentionally to showcase the anxiety and chaos of the encountered traffic patterns. All data captured (Longitude, Latitude, Altitude and Time)during the trip is overlayed on screen moving in a pattern from left to right. The video was created to document a first person experience of doing this route on a bicycle, as a facsimile of an  experience of the M5 route  in a different way. Following the video through the entire duration takes the viewer through the entire route as it would be done, at a chaotic 2000% speed. Taking the time out of the context was necessary as it serves to condense the experience.

By gathering data and using my own body and energy to transport myself through space, I feel I have gained a more intimate connection to spaces and lines that were previously unknown to me. This  intimacy, I feel would not be possible within the space of a bus. The ability to interact with your environment and have minute control within the confines of the M5 route intensified the tension that is inherent in the amalgamation of 18KM streams of socio-political architectural distortion.

Processing Video/Multiple Screens 

Processing sketch using camera and mouse interaction to create multiple instances of screens.
Mouse moves through the screen to select point of interest.







Capture video;



void setup() {
  size(300,300);
  video = new Capture(this, 300,300,15);
}

void draw() {
  if (video.available()) {
    video.read();



  // tint(mouseX, mouseY, 255);

  //blend(video, mouseX, mouseY, 33, 100, 67, 0, 33, 100, LIGHTEST);

  //blend(video, mouseX, mouseY, 0, 0, 0, 0, 0, 0, DIFFERENCE);
  image(video, 0,0,400, 300);

  copy(mouseX, mouseY, 100, 100, 0, 0, 50, 50);
  copy(mouseX, mouseY, 100, 100, 0, 50, 50, 50);
  copy(mouseX, mouseY, 100, 100, 0, 100, 50, 50);
  copy(mouseX, mouseY, 100, 100, 0, 150, 50, 50);
  copy(mouseX, mouseY, 100, 100, 0, 200, 50, 50);
  copy(mouseX, mouseY, 100, 100, 0, 250, 50, 50);

  copy(mouseX, mouseY, 100, 100, 50, 0, 50, 50);
  copy(mouseX, mouseY, 100, 100, 50, 50, 50, 50);
  copy(mouseX, mouseY, 100, 100, 50, 100, 50, 50);
  copy(mouseX, mouseY, 100, 100, 50, 150, 50, 50);
  copy(mouseX, mouseY, 100, 100, 50, 200, 50, 50);
  copy(mouseX, mouseY, 100, 100, 50, 250, 50, 50);

  copy(mouseX, mouseY, 100, 100, 100, 0, 50, 50);
  copy(mouseX, mouseY, 100, 100, 100, 50, 50, 50);
  copy(mouseX, mouseY, 100, 100, 100, 100, 50, 50);
  copy(mouseX, mouseY, 100, 100, 100, 150, 50, 50);
  copy(mouseX, mouseY, 100, 100, 100, 200, 50, 50);
  copy(mouseX, mouseY, 100, 100, 100, 250, 50, 50);

  copy(mouseX, mouseY, 100, 100, 150, 0, 50, 50);
  copy(mouseX, mouseY, 100, 100, 150, 50, 50, 50);
  copy(mouseX, mouseY, 100, 100, 150, 100, 50, 50);
  copy(mouseX, mouseY, 100, 100, 150, 150, 50, 50);
  copy(mouseX, mouseY, 100, 100, 150, 200, 50, 50);
  copy(mouseX, mouseY, 100, 100, 150, 250, 50, 50);

  copy(mouseX, mouseY, 100, 100, 200, 0, 50, 50);
  copy(mouseX, mouseY, 100, 100, 200, 50, 50, 50);
  copy(mouseX, mouseY, 100, 100, 200, 100, 50, 50);
  copy(mouseX, mouseY, 100, 100, 200, 150, 50, 50);
  copy(mouseX, mouseY, 100, 100,200, 200, 50, 50);
  copy(mouseX, mouseY, 100, 100, 200, 250, 50, 50);

  copy(mouseX, mouseY, 100, 100, 250, 0, 50, 50);
  copy(mouseX, mouseY, 100, 100, 250, 50, 50, 50);
  copy(mouseX, mouseY, 100, 100, 250, 100, 50, 50);
  copy(mouseX, mouseY, 100, 100, 250, 150, 50, 50);
  copy(mouseX, mouseY, 100, 100, 250, 200, 50, 50);
  copy(mouseX, mouseY, 100, 100, 250, 250, 50, 50);

  noFill();
  // Rectangle shows area being copied
  //rect(mouseX, 25, 50, 100);
}

ICM Trabi – week 1

15 hours of processing for a simple schematic of a East German car(Trabant).

Trabant Schematic

Somehow I don’t feel this program is intended for hard-coding detailed schematics. It appears that drawing through code is terribly inefficient. I miss Illustrator…

The process was fun overall, and equally as frustrating and some knowledge for basic 2D shapes such as arcs, curves, bezier curves, rectangles, triangles etc.

This command helped me immensely towards the end of the process.
Its a means of tracking the pointer x/y coordinates, hence making it easier to know where to place your coordinates for each 2D shape. Thanks Bolta!

void draw () {
println(mouseX=mouseX;
(mouseY=mouseY);

Apparently doing this is not the most efficient way, since it references from 0/0 on the frame. A better way would be to have X and/or Y as reference to each shape/line, so that each can be moved later in relations to a relative value rather than a specific.

Thanks to some 2nd year peeps for teaching me how to insert some trackpad interactivity.
I used this code as a quick fix to add some x axis motion.

translate(mouseX-250,0);