Glimmers
At ITP we believe that computational media will make possible exciting new artistic forms. Unfortunately there is not so much evidence to back up that belief. In the arts computers are used to more efficiently create work in traditional linear or static forms. Doubters say that you will always break the spell on audiences imagination when you ask them to interact. We have found some interesting glimmers in our Physical Computing classes at ITP by taking the opposite approach, demanding much more of more parts of the user. I will talk here a little bit about the theoretically background for Physical Computing and then very practically about how we implement it in our curriculum.
Computers for the Rest of You
Your body is making contact with the world using hundreds of muscles and millions of nerve endings. Even more numerous are the internal connections within your imagination. Your mind distills all this activity down to a very meager executive summary called consciousness. The fact that our bodies are capable of instantly finding the most pertinent bits out of this flood of sensation and imagination is impressive especially in dangerous situations. In the more enjoyable activities in our lives such as sports, sex and art, the trick is to not worry so much about the summary but to deal directly with the full flows into and out of your body and mind. In the simple act of walking, you are not conscious of the millions of decisions involved in having your feet correctly meet the terrain but you would say that it is you making those decisions.
Despite the fact that we are capable of processing much more, we spend most of our time conscious only of our consciousness. Indeed we come to equate consciousness with the whole of our self. If we are making computers in our own likeness, as I believe we are, it is not surprising that they primarily cater to that tiny part of our experience that reaches consciousness. Your body is bored with just that. Maybe it is time we started making computers for the rest of you.
Computers are seen as information processors and the Internet as being for information transfer. Information by some definitions is analogous to consciousness, difference that makes a difference. Many people in the arts cannot get too excited about the web because they also want the difference that doesnt make a difference. They want to include some of what was cast off in distilling the information, allowing the user to do more processing to complete the picture. You would think computers would be well suited to this. Containing context along with the content is a unique capability of the non-linear media of computers because they can store more associations than forward and backwards. Ultimately the non-linearity of computers media will better capture and portray our raw subjective experience.
It is tempting to blame the current computer harware as limiting how rich an experience we can convey. Although computer networks and processors have developed fantastically in the past decade, the bandwidth between the person and the machine remains along the same old pipelines of the keyboard and mouse, monitor and speakers. New calls for broadband refer to bandwidth between machines and not to the bandwidth coming back from the person into the computer. This results not from shortcomings of the technology but from an impoverished view of people. We should not be looking at the computer but at the person. Imagine what a limited being the computer sees when it looks back at you, an eyeball and two ears for input, and fingers (probably the most consciously controlled part of our body) for input. That looks more like a Tralfamadorean than a person.
For starters it would be nice to reverse things so your eyes are used for input into the computer and your fingers are used for sensing output. Your eye movements are not very consciously controlled and thus people pay close attention to them. People are just as interested and more trusting in what you give off, your non-conscious actions, than what you give out. It is the difference between trying to say something and trying to express your self. Most communication over the Internet filters what you give off leaving only what you give out. This controlled communication might seem attractive for business but will not go far in building trust or letting expression flow once trust is built.
Better yet, lets get away from the head and the hands and look at the neck and big muscles in the arms, legs and butt. Researcher into convergence shows that people lean forward at computers and backwards when watching TV. Well people have many more positions than that. In fact sometimes they are not sitting at all. Sometimes they leave the house and sometimes they dance.
Physical Computing at ITP
Being within New York Universitys Tisch School of the Arts, ITP attracts students who sense the potential of nonlinear media but for expression instead of information. We are a graduate program so our students usually have a background outside of computers, for instance in architecture, filmmaking, or sword swallowing. In their first semester students learn to program the computer and to create digital media but are often frustrated that most examples in this new media are a step back from traditional media in their expressive potential (for the student or their audience).
At ITP I started an area of study that I called Physical Computing. In these classes students are asked to go back to a wider view of what people do with themselves than is currently in evidence on the World Wide Web. Students are also asked to stretch their conception of the place and shape of a computer beyond the gray box in an office. The results are projects that range from room sized installations to clothing to musical instruments.
We have shown several hundred projects at our semi-annual shows and have in the process collected and developed many techniques for artists interested in Physical Computing For instance, Danny Rozin, a former student and now director of research at ITP has created many installations such as The Wooden Mirror and The Easil shown all over the world. As a by-product he developed an XTRA for Director called Track Them Colors that allows students to plug a video camera into the computer and very easily do some simple machine vision and video manipulation. The first thing many students need for their projects is the position and movement of the users body, their whole body, not just their fingers. Bill Buxton, of Alias/Wavefront rightly points out that the toilets in the airports are smarter than your computer in this respect but Dannys software is rectifying the situation. With the millions of pixels per second video blows by the limits of your consciousness and begins to challenge the capabilities of your full body. This XTRA opens this richness of video up as an input in addition to the usual output. Each year the projects get better because students have a technical and conceptual leg up thank to alumni like Danny.
The introductory course for physical computing is pretty wide open. It covers some skills without too much methodology or theory and then gets out of the way of the students pent up vision. We work with microcontrollers which are just very tiny and cheap computers. The skills presented are at a medium to low level. The students have to build their own circuits but they can program the microcontroller in Basic instead of assembler. The class looks for interesting transducers like photocells, motors and thermistors for converting between electronic signals and the many types of energies like light, pressure and heat that your body can create or sense.
Physical Computing fits well with the other departments at the Tisch School of the Arts like film and theater if for no other reason than the job market is not easy. Physical Computing has many pedagogical advantages that benefits for students who will go on to more purely software pursuits in the web industry. Students deal with bits instead of bytes with these tiny microcontrollers which affords them a look down to the metal of computers which they dont get in the higher level languages like lingo which are widely used at ITP. For the less technically adventurous, computer programming can suddenly become clear when it has tangible components like wires, buttons and motors.
The class has a simple structure and limited assignments but students work harder in this class than in any others. One motivator is that the work usually has some physical manifestation where they can see people using it, unlike the distant and anonymous use of their web projects. The open-endedness of the class encourages a playfulness that can bring out the zeal of the eccentric. I think the main reason for the obsession shown in this class is that it taps into the belief that computer revolution has just begun. It reminding students that computers will not always be as limited as they are now. The potential of computational media in the arts is well served by a big step back back in perspective when looking at both the machines and the people that use them.
There are now many classes that follow the introductory class. Designing Experience, taught by Masamichi Udagawa, and Public Places, taught by Gideon DArchangelo, provide more methodological backbone, one coming from the industrial design, and the other from museum design. Expressing with Technology taught by Danny Rozin is more for the fine art crowd. Physical Computing 2, taught by Tom Igoe, offers more professional project management skills and deeper technical chops. Other classes like Sonic Design, taught by Ben Rubin, and Virtual Reality, taught by Jean-Marc Gautier, take on a whole new flavor when you have been infected with the possibilities of Physical Computing.
Step Back
The great success of the World Wide Web has somewhat cemented our vision of what computers can do as processors of information. As we try to push that conception to suit the more expressive parts of our lives we are hampered by a very consciousness centered view of ourselves. We should make computers notice and service the parts of our experience that never reach consciousness. This calls for some practical training in the area digital and analog input and output from the computer. At ITP we try to do this in our Physical Computing curriculum.
Dan OSullivan (dan.osullivan@nyu.edu) is an Assistant Professor at the Interactive Telecommunications Program (ITP) at New York Universitys Tisch School of the Arts. ITP is a two year masters program with approximately 240 students exploring creative uses of computer and networks. Dan was the originator of Navigable Movies which became the product QuickTimeVR at Apple Computer. He was the creator of various interactive television programs including YORB and Dans Apartment.. His current work is in the area of data mining as a means of creating social spaces.
[1] The User Illusion : Cutting Consciousness Down to Sizeby Tor Nrretranders
[2] Steps to an Ecology of Mind by Gregory Bateson
[3] Joy Mountford often talks about this. http://www.idbias.com/people.html
[4] Creatures with hands for a head and eyes in the palm in Slaughterhouse Five byKurt Vonnegut
[5] The Presentation of Self in Everyday Life by Erving Goffman
[6] http://itp.nyu.edu/
[7] http://fargo.itp.tsoa.nyu.edu/~dano/physical/physical.html
[8] http://fargo.itp.tsoa.nyu.edu/~danny/bio.html
[9] http://www.smoothware.com/
[10] Mindstorms, –Seymour Papert