A Thinking Sam @ITP

When coming to ITP, one of the general themes I knew I wanted to explore was data visualization and how it can actually affect people’s day-to-day decision-making and behaviors. There’s so much information out there in the world, but structuring this glut of information in forms that can actually be useful in guiding decision-making is something we’ve barely scratched the surface of.

This general theme inspired the main project of my first semester at ITP: economz, a visualization project that helps people make sense of their carbon FOODprint. By creating meals using popular food items, economz shows the associated carbon footprint of the meal using a gauge we’re all familiar with: car miles traveled. The goal of the project is to raise awareness about how food choices have a great impact on one’s carbon footprint, to suggest greener meal alternatives, and to hopefully plant the seed of change in people’s minds.

The Short Version

The economz website hosts more information about the project. Please see it here (it will be moved to economz.herokuapp.com soon; as well, the featured visualization will be updated to a newer version).

View the code:

• The Processing code of the digital visualization
• The Arduino/Processing code of the physical installation and accompanying digital visualization
• The Ruby/Sinatra code of the economz website

Now, onwards to each component of the project.

Creating the digital visualization in Processing (The Processing code)

A version of the visualization built with Processing and Processing.js is available here. From the very beginning, I knew I wanted the user to interact with the visualization by placing food items on a plate (creating a meal), and then seeing the associated carbon footprint of that meal in the form of a car travelling across the screen (because car travel is one of the main rubrics people think about for carbon emissions). One of the most fun parts of the project was creating the digital dashboard with menus, prompts and buttons. I wanted it to be a self-contained program that could live on the Web with splash screens, buttons, Tweet functionality and more. I am quite happy with how it turned out.

Future work (both planned and possible):

• Ability for user to save the meal visualization they have created (done. pending next version)
• Ability for user to tweet about the meal visualization they have created (done. pending next version)
• Ability for user to toggle different attributes of their meals; for example, how big is the portion? do they eat it once a day? once a week?  is it organic? (this is possible, but not currently planned. there is something nice about the simple/straight-forward dashboard with limited options)

Creating the physical installation with Arduino, RFIDs, and Processing (The Arduino/Processing code)

The economz physical installation is powered by serial communication between Arduino and Processing. Physical “fake” food items (like this steak here!) are tagged with RFIDs and communicated to Processing via an NFC/RFID shield and Arduino. Processing is used for the digital visualization.

Here’s how it works:

• The user approaches the economz physical installation and encounters 23 different fake food items (various veggies, meats, dairy items, starches, and snacks).
• They select the food items they want to create a meal out of, and then scan each item across a grocery-store-style checkout system (and then place them on a plate). As the items go across the checkout counter, the RFID tag of that food item is detected by the RFID shield on the Arduino, and the screen in front of the user is updated to show the new food item on the plate. If they rescan the item, they can remove it from the plate.
• When they are done creating the meal, they scan a toy hummer SUV to run the car animation and to see the carbon footprint of that meal. At the end, a splash screen appears on the monitor to give them more information about the meal they just created, and to suggest actions for a healthier carbon FOODprint.

Creating the economz website with Ruby/Sinatra (The Ruby/Sinatra code)

The economz website is viewable here (soon to be moved to economz.herokuapp.com). It is built with Ruby/Sinatra, and houses the main Processing.js visualization, information about the project and the physical installation, and a gallery of food items with more information about each item’s carbon impact. The gallery of meals is particularly important to the website. It allows visitors to learn more about each food item’s carbon footprint and to view, submit, and exchange recipes for those that are “economz-friendly” items. The gallery also has a bit of slick JQuery interactions and a nice color-oriented visual display.

The main future work planned for the website is allow users to save the meal visualization they have created directly to the economz Web gallery of meals. Currently, the user can save the meal visualization locally (to their own computer), but saving it the economz server is a tricky task. Code needs to be written to generate an image from the Processing.js canvas, send it to the economz application server, and then read the image and save it to the database at the application. It turns out generating an image from the canvas and sending it to the application is easy (using html5 and JQuery/AJAX), but actually reading, encoding and saving the image on the application side is much trickier (code needs to be written so that the application can properly encode the image at its end).

Conclusion

economz was a really fun and important project to work on. Special thanks go to my wonderful ITP professors for their support and guidance: Daniel Shiffman (for ICM/Processing), Benedetta Piantella (for Physical Computing/Arduino/RFIDs), and Zevensuy Rodriguez (for Web/Ruby/Sinatra). Also a special thanks goes out to all my ITP colleagues who helped out: Esteban Cordova, Maria Paula Saba, Jess Jung, Caroline Sinders, Wajma Mohseni, and all the other classmates who gave great feedback.

For my final projects in ICM, Pcomp and Web, I’m creating one big project around how food choices influence one’s carbon footprint.

The project is called economz (for now), and the website for it can be found here: http://economz.tumblr.com/

About economz

For our physical computing midterm project, my team (which included the wonderful Wajma Mohseni and Wang Yang) developed a fun installation to track and encourage healthy eating in a shared kitchen or break room.

Check out the project website here. The code for this project is available on Github.

And now on to the details.

Collectabite was an idea borne from our belief in:

•  HEALTH, particularly nutrition
•  COMMUNITIES influencing eating habits
• SOCIAL ACTIVITIES that are fun and that can positively influence healthy eating!

How does it work?

Collectabite is a shared nutrition system that tracks how often people are eating fruits and vegetables, and highlights the positive habits of individuals and the group.

A day in the life of a Collectabiter:

Solange is a busy techie at a startup firm. She eats an apple at 11am instead of grabbing her usual danish. Feeling happy with herself, she high fives Lil’ Collectabite in the kitchen to register her apple consumption. “You go girl!” it yells out in encouragement. It takes a photo of her from a camera which is projected onto a public screen for everyone to see: “Solange the Healthy Superstar” using a cool picture format. The eat-o-meter goes up by one on the screen, getting closer to the company’s combined goal for the day. Now everyone will know Solange is eating healthy and contributing to the community goal.

The collectabite information is fed into a website, where pictures of healthy eaters are stored, the meter is monitored, and discussion forum + social media feeds are streamed.

Once the community goal is reached, Lil’ Collectabite shouts out and cheers. The information will be fed into a tweet or website so all members of Collectabite are notified (and if companies or organizations wish, rewards can even be provided!). The meter is refreshed for a new week of healthy eating. If the goal is not reached, it will be recorded and tracked.

Technical Overview

The code for this project is available on Github.

Collectabite is built using serial communication between an Arduino micro-controller and the Processing development environment. The user registers a healthy eating action by high fiving our apple figure (aka “Lil’ Collectabite”). Using force-sensing resistors that we calibrate, we are able to determine exactly when a high five has been registered by the system.

Once the system has been triggered, Processing takes over. Each High Five produces the following action:

• An encouraging sound or song is played (using the Processing minim library). For example, “You Go Girl!” or a series of cheering.
• A series of photos are taken to document who just registered the healthy action and at what level the meter is currently. An animated .gif is created (using the gifAnimation library in Processing), shown on the public screen for everyone to see in person, and uploaded to a website that everyone in the group has access to. The website allows for even more public recognition of the healthy individuals as well as a tracking system of the meter.
• The meter goes up by 1.
• If the goal has not yet been reached, the system remains open and someone else can high five Lil’ Collectabite.
• If the goal has been reached, a final “winning” song is played and the system resets.

Video on the User Interaction

Visual Display of the digital system:

Image of the physical installation:

Video on the making of Collectabite

Pictured here is our playtest set-up in the ITP kitchen area. People were asked to either “High Five for Health” or “Throw Ball Through the Mouth” whenever they ate a fruit or vegetable. We need more incentives and interaction.

For our Pcomp midterm project, we have created a concept around improving the shared nutrition of a group. The working title is “Collectabite,” and it seeks to make a community more aware of healthy eating habits and shared nutritional intake. Ultimately, the goal is to encourage a community to eat better. We will publicly display the group’s fruit and vegetable consumption in a shared community space. The process will be fun. We will use a visual display and a series of sounds to encourage an interactive process.

Read more about it in the concept brief.

Preliminary Bill of Materials:

• Wood (for carnival idea)
• Paint (for carnival idea)
• Balls Material (felt, fake fruit, etc. for carnival idea)
• Cylinder (for carnival idea)
• Buttons (for button idea)
• Giant Stuffed Fruit (for high five idea)
• LEDs (for physical meter idea)
• Screen (for digital meter idea)
• Sensors
• Speaker (for sound interaction)

Timeline

• Oct 22nd – Present Concept
• Oct 29th – Playtest user interaction
• Nov 5th – Build input system, begin mapping to meter
• Nov 12th – Finish input system, finish meter system

For my Week3 PCOMP lab, I ran a servo motor with my Arduino.

After getting the basic lab working, I decided to “supe” up my Indian auto-rickshaw with some hydraulics (no joke, during my time living in India, it always felt like the rickshaws had hydraulics anyways, and not in a good way).

This was my inspiration:

This was my attempt (the motor came free near the end):

Funny Note.

I wrote the below before realizing my circuit wasn’t connected to ground on the micro-controller. An hour of my time lost, but a learning experience

—————-
Sam’s Silly (incorrect) Note:

I had an issue where the mapping was very irregular (it would give a different value each time).

The block of code that I fiddled with was:

int analogValue = analogRead(A0); // read the analog input
Serial.println(analogValue); // print it

// if your sensor’s range is less than 0 to 1023, you’ll need to
// modify the map() function to use the values you discovered:
int servoAngle = map(analogValue, 0, 1023, 0, 179);

The issue I faced was that, sometimes it would print 1023, but after I added the servoMotor code, it would change to 735 (but nothing changed about analog input).
——————-
Actually, it just wasn’t connected to ground on the microcontroller.

As I read through Tom Igoe’s Physical Computing’s Greatest Hits (and misses), I was struck by just how much one can innovate on existing ideas. While I had come across almost all of these project types before, each one of the projects presented by Tom was a decidedly unique take in its own right. Human creativity is decidedly wonderful.

I’m always most intrigued by the projects that cause a double-take: the seemingly random object that mirrors your movement as you walk by; the  ”fake” flowers that sway in the wind with your movement; the video-game that you’re part of without knowing it. I think random day-to-day delight and whimsy is something humans thrive on, but don’t get to experience nearly enough. It’s certainly a concept I would love to explore more closely.

I’m also quite drawn to physical computing projects that explore alternative forms of energy. For example, the Solar Sinter project (in the above video) utilizes sunlight and sand as the raw energy and material to produce glass objects using a 3D printing process (and Arduino is the micro-processor!). While it’s still experimental, it is a new take on how to use abundant energy supplies. More physical computing projects should explore this area.

Another project type I find to be compelling (and the most meaningful) are those projects that equip a certain group of people with capabilities they lacked before. In particular, projects that allow people with disabilities or paralysis to better explore, interact with their world, and express themselves via a new form of physical computing. Some examples include:

• The Neater Eater – Allows people with disabilities to easily program and customize their own feeding mechanism. While it’s something most people take for granted, the ability to feed yourself (rather than rely on someone to hand feed you) is incredibly empowering.
• Eyegaze Edge – Quadriplegics or others who can’t use a standard keyboard and mouse can control a computer simply by moving an eye (the device uses a high-speed infrared camera mounted under the system’s monitor and a small external processing unit to translate eye motion into on-screen action).
• Jouse2: A computer is controlled with mouth movements and gentle puffs of air. The person manipulates the stick with her mouth to move the cursor and can click on an item by blowing into the straw.

However, there is one major issue I have with almost every physical computing project. It tends to be designed for a certain privileged sect of the world population (generally, the “richer” countries and populations). Physical computing and interactive art projects are not yet designed for use in the majority of the world. I think that’s a real shame. While there is still a digital divide in access to technology in these places, I wonder how we as technologists can start bridging this gap and coming up with simple devices and interactive physical projects that can be used by people all over the world. How can we bring delight, whimsy, and fun projects to people who are just now discovering computers? How can physical computation empower more people to live more meaningful lives in all corners of the world?

These aren’t easy questions, and it certainly will be a challenge — but one I feel is critically important to the field. This is especially true as physical projects become more and more ubiquitous with the rise of 3D printers and other similar technologies.

Week 2 PCOMP labs were not only a lot of fun, but they allowed me to start my (hopefully long-term) relationship with my new friend Arduino.

Here’s what I did:

Digital Input and My Very Own Switch!

Using Arduino and a switch, I programmed a basic digital input/output that controls the 2 LEDs. When the switch is open, one light is on; when the switch is closed, the other comes on. The switch is the digital input, and the LEDs are the digital output.

I then had some fun with it! This past weekend, I escaped the city and went on an excursion to upstate New York. On our trip, we stopped at a thrift store that had all sorts of weird/fun items, including some old-school games!

I bought the classic game “Operation,” and it proved to be perfect for my very own switch. When I touch the metal part of the game (when performing the “operation”), not only does the patient’s nose light up, but so do my LEDs!

The set-up:

Analog In with an Arduino and the Coffee Luv-o-Meter

In the next lab, I used Arduino to explore Analog input. As a somewhat silly way of playing with analog inputs, I attached a force-sensing resistor to a coffee cup. The harder you squeeze the cup, the brighter an LED shines. The brighter your light, the greater your love for coffee (or something like that …)

Apologies for ridiculous midnight commentary.

Thoughts and questions that came up:

• I wanted to make it a bit more difficult to get the light to shine bright when you squeezed it (i.e. I wanted the person to have to squeeze really hard to get it bright; as-is, you don’t have to squeeze too hard to get it from “no light” to “bright”). I fiddled with my Arduino code, but that didn’t do the trick. Maybe something physically on the breadboard would help?
• Also, umm, I really need to learn Arduino code!

When I was young, I used to love going to big electronic stores, mainly because it meant I could demo the video game systems I didn’t have at home. But I always remembered being disappointed and somewhat confused by the whole set-up.

I wanted to see why this was (and if it was even still the case), so I set out to observe how in-store electronic displays are designed and how people interact with them.

tl;dr: In-Store Displays Suck!

I was shocked by just how bad these set-ups are designed (true story: they are exactly the same as when I frequented them 15+ years ago!). What’s inexplicable is that these set-ups are presumably installed with the intention of selling more product — the fact that they are so terrible to interact with is an enoromous waste of an opportunity for these stores!

So what sucked? (note: I am mainly talking about the interaction with the display, not so much the electronic itself)

In his hilarious and informative book “The Design of Everyday Things,” cognitive scientist and usability engineer Donald Norman points out that a well-designed product has obvious visual cues and a clear and natural map of what does what. You can see exactly what you need to see, and it’s fairly obvious what each component will do. A big part of getting it right is designing for the context of your user.

The problem with in-store displays is that they completely miss the context of how the product will actually be used by an in-store customer. Instead, it’s basically just the product sitting there, waiting for someone to do random shit with it.

Sounds like that could be OK, but here’s what I found:

My Experience

I initially saw an open Wii across the room, and being the Nintendo fan that I am, I went over to see what it was all about. Here’s what I saw on the screen as I approached:

Now, for many people, this would be an immediate deal-breaker on exploring the wonders of the Wii. There was no information about how to reset the system or any other helpful info or visual cue for the first-timer. However, me being the tech wiz I am, I grabbed the Wii remote and re-connected the device. Fantastic right? But were’s what I saw next:

I came in on a game that was already in progress, with absolutely no instructions or prompts of what I could do. I was left fiddling with the controller, pressing random buttons, and ultimately just getting confused and frustrated by the game I couldn’t get to work.

And this is my problem with these displays.

They are there to show off the product and allow people to play around, maybe even moving them one step closer toward making a purchase. But there’s absolutely no customization of the display for the context of a user in the store. Games don’t reset after each user (instead, normally you have to start where the last person left off). There’s no clear “getting started” instructions. There’s no real visual cues to let you know what you should do, so you’re left pressing random buttons (they seem to assume people will already know how the product works, which is ironic given its goal of engaging new users).

It’s a frustrating process, even for those of us that have played lots of games in our lives.

A better in-store display would take into account the context of the user: someone browsing the store who may be very new to that technology. It would thus have a customized process to orient the new user to the system in the best possible way. For example:

• A visual cue to let people know what they need to do first. Grab the controller? Press a button? Select a game or other program? This stuff isn’t always clear.
• A specific demo version of a game that gives newcomers some initial instructions (that can be skipped by the expert), and then resets after each use. Coming in midway through someone else’s game is confusing.
• A fun prompt after the end of the game to let the user know what they can do next.

My thoughts were backed up by some hilarious observations of other in-store users: whether the user was trying an Xbox, iPod, or PS3, the interaction with the display was rather uniform — they would initially look excited, grab the device or controller, press random buttons and try to figure out how the damn thing worked, get confused, and then put the thing down and leave after 2-3 minutes of not making much progress. If the goal was to have the user press random buttons to see how the device physically felt, then mission accomplished! If the goal was to get the user more engaged with the product, then it was a massive failure.

Again, the context matters.

A chair would be cool too.

And then there was light! (video of my first blinking LED at ITP)

Working with my wonderful lab partners Caroline and Vanessa, we were able to set up our breadboards and get the LEDs to blink to our liking:

However, we certainly ran into some difficulties (and had some questions):

• Soldering. The instructions said to touch the soldering iron directly to the joint (not the wire), but the joint never actually got hot enough to melt the wire (so we had to touch the wire directly)
• At first we thought we did not set up our 9-12V DC power supply correctly. Turns out we just didn’t understand how to use the meter properly. It worked fine (although we forgot to attach a battery for a bit!)
• Specific ways of wiring to create proper circuit. We understand that the wires have to be connected/run in specific ways, in order for the circuit to work, but we’re still a bit unclear on the specifics.