Physical Computing

Yeseul Song | Syllabus | IMNY-UT 245 | Tues 3:40pm to 6:40pm in 370 Jay St, Room 411 Meetings:14
Last updated: July 3, 2025
Show Course Description

This course expands the students’ palette for physical interaction design with computational media. We look away from the limitations of the mouse, keyboard and monitor interface of today’s computers, and start instead with the expressive capabilities of the human body. We consider uses of the computer for more than just information retrieval and processing, and at locations other than the home or the office. The platform for the class is a microcontroller, a single-chip computer that can fit in your hand. The core technical concepts include digital, analog and serial input and output. Core interaction design concepts include user observation, affordances, and converting physical action into digital information. Students have weekly lab exercises to build skills with the microcontroller and related tools, and longer assignments in which they apply the principles from weekly labs in creative applications. Both individual work and group work is required.

Prerequisite: Creative Computing or equivalent programming and physical computing experience.

Creative Computing

Dan O'Sullivan | Jack B. Du | Ellen Nickles | David Rios | Carrie Sijia Wang | Daniel Shiffman | Syllabus | IMNY-UT 101 | Fri 3:40pm to 6:40pm in 370 Jay St, Room 409 Meetings:14
Last updated: July 3, 2025
Show Course Description

This course combines two powerful areas of technology, Physical Computing and Programming, and asks students to consider their implications.  It will enable you to leap from being just a user of technology to becoming a mindful creator with it.

The course begins with Physical Computing, which allows you to break free from both the limitations of mouse, keyboard and monitor interfaces and stationary locations at home or the office. We begin by exploring the expressive capabilities of the human body and how we experience our physical environment. The platform for the class is a microcontroller (Arduino brand), a very small inexpensive single-chip computer that can be embedded anywhere and sense and make things happen in the physical world. The core technical concepts include digital, analog and serial input and output.

The second portion of the course focuses on fundamentals of computer programming (variables, conditionals, iteration, functions & objects) as well as more advanced techniques such as data parsing, image processing, networking, and machine learning. The Javascript ‘p5’ programming environment is the primary vehicle. P5 is more oriented towards visual displays on desktops, laptops, tablets or smartphones but can also connect back to the physical sensor & actuators from the first part of the class.

What can computation add to human communication? The ultimate question of this class is not “how” to program but “why” to program. You will gain a deeper understanding of the possibilities of computation in order to see how it applies to your interests (e.g. art, design, humanities, sciences, engineering). In addition to weekly technical assignments there are blogging assignments, usually reacting to short readings, allowing you to reflect in writing about the nature of computation and how it fits into your life and into human society. 

There is an even workload each week of a technical production assignment and a writing assignment but none of them are big.  The course is designed for computer programming novices but the project-centered pedagogy will allow more experienced programmers the opportunity to go further with their project ideas and collaborate with other students.

Real-Time Media

Carrie Sijia Wang | Syllabus | IMNY-UT 285 | Tues 3:40pm to 5:10pm in 370 Jay St, Room 407> Thur 3:40pm to 5:10pm in 370 Jay St, Room 407 Meetings:14
Last updated: July 3, 2025
Show Course Description

This course focuses on designing, developing and delivering real-time, performative work using audio and video elements. The class will have an emphasis on using MaxMSPJitter and other tools to create performative experiences that dynamically combine interactive elements such as video, sound, and code, allow for the unfolding of engaging narratives, and generate compelling visuals in real time.

We will look at various examples of both multimedia performances and installations, explore how we can apply the technologies we have learned to design real-time systems, and discuss methods we can use to make our work more engaging.

The class is three-fold and divided into tech tutorials, discussions of existing examples, and in-class performances.

Playful Experiences

Gregory Trefry | Syllabus | IMNY-UT 290 | Mon 3:40pm to 6:40pm in 370 Jay St, Room 409 Meetings:14
Last updated: July 3, 2025
Show Course Description

Forget the screen. People want to be part of the action. They don’t want to watch detectives and control superhero avatars. They want to solve the mystery and be the hero. They want to experience it. We see this craving for playful experience in everything from immersive theater to escape rooms to the Tough Mudder to gamified vacation packages. Designing live experiences for large audiences that demand agency offers a distinct set of challenges, from how much choice you give each participant to how many people you can through the experience. We’ll look at examples from pervasive games to amusement parks to immersive theater, examining both the design choices and technology that make the experiences possible. Along the way we’ll create large, playful experiences that put the participant at the center of the action.

This class focuses on the particular design problems of large-scale games and playful systems. In this class students develop a foundation in design fundamentals from which to approach the problems of design particular to experiential entertainment. We will analyze existing digital and non-digital games and playful experiences, taking them apart to understand how they work. We will also work on a series of design exercises that explore the social, technological, and creative possibilities of play.

The class will be broken into three sections: People, Time and Space. People will focus on experiences that coordinate the actions of a large number of participants. Time will focus on experiences that stretch out in time and begin to integrate with our everyday lives. Space will ask you to design an experience that takes advantage of physical space and integrates other elements of the class.

Information Design (Topics in Design)

Katherine Dillon | IMNY-UT 270 | Wed 3:40pm to 6:40pm in 370 Jay St, Room 409 Meetings:14
Last updated: July 3, 2025
Show Course Description

Information Design Introduces students to the art and science of transforming complex data into clear, compelling visual narratives. Students explore fundamental principles of information design while developing practical skills in data analysis and visual representation. Through hands-on projects, students learn to craft effective charts, diagrams, and maps that communicate complex information with clarity and purpose. The course emphasizes both technical proficiency and storytelling, teaching students to select appropriate visualization methods and combine them into cohesive narratives that engage and inform their audience. By the end of the course, students will have developed a robust toolkit for turning raw data into meaningful visual experiences.

At the completion of this course, the students will:

– Think critically about information design problems and have a framework for defining, assessing, and solving them

– Develop confidence with the vocabulary of information design and the appropriate application of the visual models

– Be familiar with best-in-class examples of information design

– Understand the power of information design as a tool of communication and persuasion

Generative Filmmaking in the Age of Hypercinema (Topics in Media Art)

Gabe Barcia-Colombo | IMNY-UT 260 | Wed 3:40pm to 5:10pm in 370 Jay St, Room 407 Meetings:14
Last updated: July 3, 2025
Show Course Description

The history of generative art dates back to the 20th century, with Dada and Fluxus artists using chance games and techniques to create spontaneous works of art. Today, generative art and entertainment continues to evolve with advances in artificial intelligence, but how do these advancements impact media making and the film and television industry? Video diffusion models, artificially intelligent avatars and generative soundtracks are among the emergent AI tools that can be used to create a film. This IMA and Film course will allow students to explore new tools in the creation of hypercinema. From RunwayML and Stable Diffusion to real time avatar puppeteering and virtual production, this course will provide a hands-on survey of generative media in the film industry.  How does generative AI change the way we tell stories? How do we deal with the ethical issues surrounding the use of generative media in film production? How will generative AI lead to dramatic changes in production and postproduction, distribution, and intellectual property ownership in the film industry? Class time will be split between lectures surveying the field of generative media and hands-on production using brand new tools and platforms.

Typography and Technology (Topics in Design)

Kelli Anderson | IMNY-UT 270 | Thur 3:40pm to 6:40pm in 370 Jay St, Room 408 Meetings:14
Last updated: July 3, 2025
Show Course Description

When we see the shape of an uppercase serif letterform, we may subconsciously be reminded of the Roman Empire. What we may not consciously realize is that this association has its roots in the technology used to make these letters, thousands of years ago. Serifs are a wedge-shaped artifact that occurs when a chisel hits stone—the tool used by the Roman Empire to carve their letterforms into monuments called capitals (now a word synonymous with “uppercase” due to this same history.) Though some debate exists among historians, it is widely believed that “capital” letters get their geometric shape from the constraints of the tool of the chisel itself. To understand how the wide stylistic variety of letterforms arrived in our font library (and to understand where our own hazy associations with letterforms originate), one must look to the technology which produced them. From the exigencies of the sign painter’s brush to the psychedelic warping of 1960s Phototype to the 8-bit pixel-based typefaces found in 80s video games, letterforms contain the technological history of the world in microcosm. The subtle choices in each typeface’s form bear the imprint of their moment’s philosophical, technological, and visual conditions, capturing an era’s zeitgeist with a miraculous economy of expression. The letters that we use today are more than 2,000 years old—persisting longer than any other artifacts in common use—but have undergone dramatic fluctuations alongside tech’s major physical transitions from stone to paper to metal to celluloid to digital information. Parallel to this technological history, letters shifted context from cuneiform to letterpress to Linotype to phototype to digital screens in a continual reinterpretation of the the fundamental question “what is a letter?” In the 1970s, technologists and computer scientists found themselves grappling with this same fundamental question as they carried letterforms over into the digital realm: What are letters? Are they fixed visual information? Or are they an idea—a set of executable, gestural instructions? Are letters best understood as reconfigurations of a set of modular parts— building-block components rather than the choreographed gestures of calligraphy? Are they the organic product of the human hand or the output of a system? Early digital technologies wagered “is this what computers are for?” with typefaces in tow—choosing which aspects of the old analog world to reconstruct—in deciding what attributes to port-over. The world we live in today has been impacted by how technologists answered these questions. Questions which, just as easily, could have been answered differently. This course will begin from a place of reflection on our own lived associations with typographic morphology. We will then explore the possible technological origins of those associations while reflecting upon how [what seemed like] tiny digitization decisions delivered us the typographic reality we inhabit today. Students will be asked to look to history for “reasons” for typographic form (which is fun!) But we will also practice looking to history for alternate futures—to examine the “dead ends” that might have otherwise been and daydream about where these paths lead. Typographic technological history offers a manageable jumping-off point for such a thought experiment. This thought experiment scales up to larger problem-solving (and conceptualization) skills related to understanding the implications and effects of tech.

Collective Narrative

Lily Crandall | Syllabus | IMNY-UT 286 | Mon 3:40pm to 6:40pm in 370 Jay St, Room 410 Meetings:7-Second Half
Last updated: July 3, 2025
Show Course Description

This two-point workshop is centered on the examination and creation of collective storytelling environments. We will examine a wide-range of storytelling spaces including participatory and user-generated environments, site-specific works, community based arts practices, and transmedia storytelling. Coursework consists of weekly multimedia assignments to reflect on storytelling in our own lives and the lives of others, field trips, and student presentations. By the end of this course, students will have experienced and experimented with alternative, collaborative, and site-specific storytelling methods.