I really enjoyed reading Tom Igoe’s Physical Computing’s Greatest Hits (and misses) because he’s got a pretty exhaustive list of interesting (if overused) physical computing categories. I’ve seen examples of almost all of these, but I’m still a little surprised they’ve become tropes. Maybe I didn’t realize that there are enough people experimenting with this level of physical computing to make “Fields of Grass” an entire category.

 

The ones I respond most to are those that enable novel interactions, not just re-imagine existing interactions. For example, even though it may not be very “useful”, I enjoy video mirrors and mechanical pixels because you’re engaging in a new kind of experience. You have to explore the system a little to find the boundaries and parameters.

Contrast that with “Body/Hand-as-cursor” or “Touch Screens” which take a well known behavior and give the user a new vector through which to use it. These categories are usually conceptualized far before they’re actualized, so they tend not to feel very exciting when you use them in person.

I don’t know where “LED fetishism” falls along this spectrum, but it’s hilarious and true. That being said, “LED Throwies” was genius.

Here are a few other examples of common Physical Computing idioms that I’ve seen:

Exercise Machines

This generally takes the form of a screen mounted on a treadmill that displays a simulated running course. The course responds to the pace and difficulty level of the runner. Running on a simulated suburban sidewalk is now possible!

 

Virtual Reality

An oldie but a goodie, the virtual reality rig is a helmet with a screen on the inside of the visor that fills the user’s field of vision. The imagery displayed on the screen responds to the orientation of the user’s body, and creates the illusion of being in a virtual 3D environment.

 

User Superimposition

This approach augments the image of a user with computer graphics, or inserts them into a wholly artificial scene. It’s often proposed as a way to “try on” clothes in a shopping context to see how they would look without having to actually try them on. It’s becoming more feasible with robust body tracking hardware like the Kinect.

 

This weekend I visited the American Museum of Natural History to observe people using interactive technology in the wild. The reason I chose the museum is because it’s got an interesting collection of hands-on exhibits that use technology which the visitors have never seen. I was able to watch them interfacing with it for the first time.

For this post, I’ll focus on the ice core exhibit. The primary focus of the exhibit was a ≈16 foot long ice core from Greenland which was positioned horizontally about 4 feet off the ground. Around the core were a number of signs which included photos, data and information. Above the core was a video interview of one of the scientists. And positioned in front of the core was a touch screen that slides along a track for the length of the core. This screen is what I was primarily interested in.

I chose to observe this exhibit because it seemed like a pretty innovative way of exploring this unique object. At first glance the screen looked like a something that might allow you to peer deeper into the core either literally or figuratively, like a hybrid between an electron-microscope and a VH1 popup video.

It didn’t end up being quite that magical. The idle screen simply says “Touch the Screen to Begin.” It doesn’t say to begin what, and there’s nothing other than its form-factor that gives you further hints. But the handles on either side are a pretty good indication about how to use it, even if you don’t know what it is. Once you touch the screen you’re given a menu of 4 different data sets, such as climate and volcanic eruptions over the past 10,000 years. After making a selection, a graph appears that slides along the x-axis as you slide the screen, implying that the data is aligned with the segment of ice below the screen. Using the device was very intuitive—the physical affordance of the sliding track made it obvious that the core itself was a timeline.

The greatest let-down of this interface was that the screen actually obstructed the ice that the data was purportedly relating to. It felt less like like it was enhancing the core and more like it was getting in the way. I would have loved to see a magnified video of the ice, possibly even down to the level where pollen and particles of volcanic ash were visible. Instead you get a power-point, only a small slice of which you can see at any given point and limiting you from taking in the whole arc at once.

After I finished using it I started observing other people from a dark corner of the room. It was a little creepy. The first thing that I noticed was that people were actually pretty excited about seeing this ice core. There’s something inherently cool about capturing time in a physical object. Lots of “Oooohs” and pointing and pressing faces in to get a closer look.

I’d estimate that 1 in 3 visitors who looked at the exhibit decided to engage with the sliding screen. It was clearly taking a back seat to the ice core itself. The extent of the engagement was also surprising; most people would slide it down the track for a few inches, get the idea, and then move on. A few people navigated into the menu a few clicks, but it was a small minority.

A rather large design oversight is that the ice core, and thus the track, were positioned high enough that it was above the heads of a few kids who walked by. They didn’t even notice it.

The general impression that I left with is that the physical object was novel, intuitive and well constructed. However, the content was dull and the screen actually detracted from the primary focus of the exhibit—the core that it was obstructing.

As the brains and sensors in the objects around us become more sophisticated, there are great opportunities to design the interaction that mediates our relationship with them. After reading the initial chapters of Chris Crawford’s The Art of Interactive Design, I agree that the most meaningful interactions require a full feedback-loop between the object and the human that includes Listening, Thinking and Speaking from both parties (if not always literally).

An excellent display of this is relationship between a car and the driver. The car is constantly “Listening” to your body movements, “Thinking” by converting those your movements into turning, accelerating, etc. and “Speaking” through meters, sounds and force feedback. This “loop” is so tight, that both actors are constantly doing all three things.

The strength of this interaction is that the fidelity of the loop is high and constant. It’s not limited by resolution or a frame-rate or an inability to interpret the data. Those aren’t always assumptions that can be made with human-computer interaction, especially when it comes to technologies like camera vision (I’m looking at you Kinect). The more “understanding” that each actor has over the intent, abilities and expectations of the other actor, the greater the quality of interaction becomes.

Interaction comes in degrees and a high-quality loop isn’t always necessary. In its simplest form, an interaction can happen between a human and a light-switch—albeit not a very interesting one. The light switch “Listens” by virtue of it being a switch, “Thinks” by interpreting it’s position into the appropriate state, and “Speaks” through the light bulb. Playing any 3D game on a laptop is not much more than this same interaction happening millions or billions of times per second.

Remove this reactionary relationship, and the interaction ceases to exist. There are examples of digital technology all around us that wouldn’t be considered interactive. My favorite might be the GPS system. We can sample from it like listening to music from a cello, but our actions have no impact on the behavior of the satellites. Far from making it a “bad” technology, this lack of interaction is one of the beauties of it’s design. Similarly, automated machinery, clocks, and CCTV are sophisticated digital technologies, but they are not interactive.