With emerging research and development with soft circuit technologies and its integration into textile and clothing design, the garment as a reactive interface opens up new possibilities in engendering self-expressions, sensory experiences and more. This 14-week class is to introduce students to this realm by creating connections between hardware engineering and textile crafting. The class is for students with basic physical computing knowledge to explore the possibility of wearables, and arouse discussion about the potential in re-imagining our relationship with personal devices, textiles and garment design as an interactive media.
What do WhatsApp and Nelson Mandela have in common? How about Mr. Robot and the UN Council for Human Rights? When most people think “digital security”, they rarely think of these connections; but the connections are there. Digital security is much more than an industry buzzword— it encompasses techno-social idealism, open source development, and symbiotic coordination between sectors in tech, the humanities, and civic society. Certainly, we’re going to talk about Signal, Tor, VPNs, and OTR. But let’s dig even deeper. In this course, students will learn the principals of digital security; from end-to-end encryption, to circumvention technology, resilient communications, and beyond. Part lab exploration, part oral history, and part prototyping workshop, students will come away with the tools they need to undertake development projects with end-user security in mind; and foster an appreciation for digital security’s integral, timely, and often life-saving role in human rights struggles across the globe.
Rules of play shape competitive games from checkers to football. But how do rules of interaction shape everyday life? And what happens when there are no established conventions and the rules are being made up as we go along as they have been for the past 6 months of living almost exclusively online? In this course, we will explore, code and test design strategies for playful, serious and bizarre group interactions while at the same time interrogating both what it means to play and how individual identities and group behaviors emerge. (The interactions we create in this class will be designed specifically for synchronous, in-person, embodied experiences.) Some of the questions we will ask and attempt to answer: What motivates participation? What hinders it? When does participation become oppressive? What’s the difference between self-consciousness and self-awareness? Who has power? Who doesn’t? Are leaders necessary? What’s the difference between taking turns and engaging in conversation? What happens when the slowest person sets the pace? And most importantly, what happens when there are no rules? Interaction inputs we will play with will include: mouse, keyboard, mobile device sensors, and microphone. Outputs can include visuals, text and sound. We will use p5, websockets and node.js for real-time interaction. Class time will be split between group improvisation exercises, playing with and critiquing examples and translating design strategies into code and logic.
This course re-conceives interactive media as a form of choreographic intervention. Instead of asking how moving bodies can control media, we will ask how interactive systems can influence movement. How do you make someone feel soft inside? How do you shake an entire room? How do you orchestrate duets between strangers? To accomplish this, the class facilitates a semester-long collaboration between ITP students and dancers from the Barnard/Columbia Dance Department. Choreographers will learn to apply computational thinking to choreography and creative coders will learn to apply choreographic thinking to computation. To whatever extent possible, we will attempt to embody code. Using computer vision and visual media, we will look at directing both how people move (quality of movement) as well as where they move (pathways and spatial relationships). We will evaluate the strengths and weaknesses of the various sensing technologies available to us today. How wide is the gulf between what we can see and feel (strength, hardness, contortion) and what a computer can see and interpret (locations, contours, velocity, acceleration)? Class time will be split between movement exercises, playing with examples and deconstructing code. The class will culminate in a final showing of student work. All classes will take place at NYU with a separate weekly technical lab for ITP students.
One of the most transformative consumer products in history, the iPhone remains the standard bearer for great design and user experience. With the latest version of iOS and the introduction of the iPhone X, Apple puts depth sensing and augmented reality in our pockets. How do we take advantage of this incredible platform to produce our own compelling experiences? This course will be a hands-on workshop where we explore the world beyond generic apps and push the boundaries of what’s possible on iOS hardware. We will cover both the design and technical elements that pertain to end-to-end mobile app development. Topics will include augmented reality, the Internet of Things, new interfaces for musical expression, voice interaction, wearables, and others… While the class will involve iOS programming instruction, the instructors will help facilitate coding using an established code base to ensure technical skills are not a bottleneck to implementation. Students should expect to spend additional time outside of class learning Swift and other related programming concepts. Full-time access to an iOS device and a Mac laptop running the latest operating systems are required. As part of the design process, we’ll host workshops and guest critiques with designers from top studios around New York City. At the end of the class, students will have a working app to add to their portfolios and a strong basis for future mobile application work.
We’ve been tinkering with the living systems that generate our foodstuffs for millennia. But climate change is radically and rapidly shifting these food landscapes, and the impacts include the extinction of many of the foods we love: chocolate, wine, beer, coffee and more importantly starvation for those in the world who are already food insecure. In this class, we’ll explore biotechnologies and bioengineering along with microbes and mushrooms to design and create pathways for the restoration of some of the damage we’ve wrought on our food system. We’ll also use art and design and systems thinking to build speculative and actionable projects that will focus not just on the future of food but the future of our planet and all of its inhabitants. This class is part of the Biodesign Challenge.
Magic windows that allow us to peek into different realities without leaving our physical space, lenses that reveal hidden layers of objects or navigating new universes within the same room. More than ever, mobile devices are getting a human-scale understanding of space and motion allowing us to create more intimate interactions with our surrounding spaces, leveraging them as a canvas to experience other realities. We now have the potential to give life to inanimate objects, tell stories through space, customizing private views of public spaces and recognize places we’ve never been. We’ll question what it means and how can we blend reality exploring themes such as: augmented space and new paradigms in social interaction, public space and privacy; storytelling and navigating the physical space like turning pages in a book; tangible interfaces, mixed objects and animism; Magic windows, x-ray vision, time-machines and impossible universes; Far away so close: telepresence and remote collaboration. The course will survey the past, current and up and coming technologies and experiences in Mixed Reality including environmental augmented reality and interactive projection mapping, handheld devices while fostering a strong user experience perspective on the affordances and constraints of each. We’ll research and discuss the design principles and guidelines for creating mixed reality experiences focusing on the links between real and virtual objects, interaction space and asymmetries between physical and digital worlds, environmental semantics and multimodal and tangible interaction. Technologies explored will be focusing on mobile platforms (phones, tablets) including Vuforia, SLAM, image and object recognition, depth sensing, projection mapping. Unity3D will be the development platform: students must have previous working knowledge of Unity3D and feel comfortable with independently developing using this platform. A working knowledge of Unity3D may be gained by going through the Unity 5 3D Essential Training Lynda Course prior to the course (log in to Lynda for free via https://www.nyu.edu/lynda).
According to anthropologists Filip de Boeck and René Devisch, divination “constitutes a space in which cognitive structures are transformed and new relations are generated in and between the human body, the social body and the cosmos.” In this class, students will learn the history of divination, engage in the practice of divination, and speculate on what forms divination might take in a world where the human body, the social body, and even the cosmos(!) are digitally mediated. Starting with an understanding of ritual and folk culture, we will track the history of fortune-telling from the casting of lots to computer-generated randomness to the contemporary revival of Tarot; from reading entrails to astrology to data science; from glossolalia to surrealist writing practices to the “ghost in the machine” of artificial intelligence. Weekly readings and assignments culminate in a final project.
The medium of motion pictures will be will be transformed by virtual reality technologies. But the emerging hybrid form will likely have less to do with the iconic VR headset and immersion, than in the newly possible flow of expression in the other direction, out of the participant. This class looks at the true potential of virtual reality as its mutability, to put ordinary users in the role of director of visual media as they already are in their dreams and fantasies. Democratizing media by breaking it down into discrete more easily remixable parts has historically, from DNA to alphabets to movable type led to an explosion of expression and knowledge creation. Motion pictures, perhaps our most convincing medium, is now undergoing such a transformation thanks to virtual reality technologies. Specifically, we will look at how tools like depth cameras, motion capture, and machine learning can treat a scene as a collection of elements instead of a collection of pixels. We will work with the real-time 3D rendering capabilities available, even in your phone, for the instant contrivance of visual reality using compositing, the transformation of images and models as well as virtual camera moves. The class will also consider how the more conceptual dissection of film by cinema scholars and psychologist into things like plots and character types might be templated into reusable formulas for non-experts to create an emotionally satisfying dramatic arcs.
Web browsers were originally used only for displaying simple HTML pages, but over the years they have become supercharged all-powerful web execution machines. In this class we’ll explore experimental new features and HTML5 APIs that allow browsers to communicate with the OS and their environment. APIs that will be covered may include: Battery Status, Geolocation, notifications, accelerometer usage, video access, speech recognition, and text-to-speech. We’ll cover the mechanics of bookmarklets and Chrome extensions, with a sustained multi-week focus on building extensions and exploring Chrome’s extensions APIs. Class workshops will include projects such as building an ad blocker, programmatically replacing text and images on a website, and making sites that respond to external events. Students will give weekly in-class presentations on web capabilities, complete small weekly assignments, and present a final project. This class leans heavily on web technologies, and experience with HTML, CSS and modern JavaScript (ICM with p5.js or Commlab Web/Networked Media) is required.
Hardware is not hard, and rapidly prototyping circuit boards is easier than ever with new tools available at ITP. Students will learn how to grow from a breadboard to a custom surface mount board, all without leaving the floor. This class is about artists and designers taking control of their hardware, and exploring the potential of embedding their projects into the world around them. Students will learn the multitude of tools and processes required to make a DIY circuit board. These include Eagle CAD, micro-milling machines, drawing schematics, ordering parts, surface-mount components, acid etching, solder paste and stenciling, reflow, pick-and-place, and others. In-class demonstrations will be done for each of the above, and students will complete assignments using online reference notes and videos. Two smaller projects and one final project will be assigned (3 total), each a circuit of the student’s’ choosing. These three assignments will be designed to work off the most recently taught subjects, and to get the students to fail early. It’s a lot of new material to be learning across multiple domains, so the students will need the entire semester to iterate.
Subtractive fabrication is a common manufacturing process that produces durable and functional objects. This class will cover multiple techniques on machining and milling raw material into custom parts. We will focus on both traditional and digital fabrication tools: lathe, CNC router, 4 axis mill, etc. We will cover CAD, CAM, and machine setups as well as research affordable desktop milling solutions for personal shops. The class will be hands on and fabrication heavy, paying close attention to precision, accuracy, and craftsmanship. There will be weekly fabrication exercises, a midterm, and a final project. It’s mill-er time.
You live with illusions. The nature of these illusions has long been described in mystical practices but is now increasing corroborated by modern research such as neuroscience, behavioral economics, social psychology, embodied cognition, and evolutionary psychology. What does this have to do with computational media? With technology, we have the ability to revisit some of these vestigial illusions that made sense in ancient environments but that might limit our personal happiness or the overall functioning of modern society. Will the computer’s ability to run more objective statistical analysis on data gathered tirelessly over time, across individuals and locations allow us to more accurately see ourselves and the world as it is. Can we build computer interfaces that give a fuller expression of our experience when we are not limited by an illusory view of ourselves? The insights into how into how to reach people more fully comes with a responsibility to then ask what should say to them. As the computers are able to understand us better than we understand ourselves will we relinquish control to them? At a practical level, the class looks at interfaces for digitizing signals from the less consciously controlled parts of your body using things like biosensors and cameras. It then moves on to opportunities for finding insights in from patterns in the mounds of already digitized expression you have produced every day for years, for instance in your email. Then we look at the possibility of bypassing consciousness with techniques like subliminal priming to affect behavior. Finally, we will look at how we can use things like browser extensions to manipulate shared media in an attempt to counter some filter bubble effects that emerge at a group level based on the illusions discussed in this class. This class will use skills from Physical Computing and ICM.
Code without content gets boring fast. This seven week course will show you how to create stories around which you can weave the technology learned in other classes. When content comes first, interesting problems arise to solve. Participants will get solid grounding in how to tell a visual story using words and images in a traditional format, so then they can take that format and reimagine it in entirely new and unique ways. The first few classes are devoted to getting basic comic skills. The remaining classes will hone and expand these abilities while posing the question: what can be done differently, and how can technology add to what we have created? At the end of the semester you will have a something that sets you apart; – original content AND technological know how. Students will combine words and images, look at each other’s work, look at examples of published works. Reimagine how these stories can be told in new and unique ways. This a demanding course. There is a lot of work involved, they will end up with a lot of original content. During the first half we look at and make traditional comics. Second half we experiment with comic format WHILE honing storytelling skills. Relevant speakers will come in to discuss what they do and how they work etc.
As a response to developing technologies, artists working in areas of new/digital media are continually inventing new concepts for self-expression – interactivity, the passage of time and resolution, just to name a few. Yet these concepts are new only in the sense that they are being adapted to new media. For example, the notion of interactivity, frequently observed as original and specific to the user-interaction component of computer-mediated works, was equally, if differently, specific to Gianlorenzo Bernini’s 17th-century Baroque sculpture and architecture. Indeed the very concept of new media, and the concomitant implication of critically significant artistic development, applies throughout history. Oil revolutionized painting in the Renaissance, as did house-paint (on canvas) in the 1950s; in the 1910s, the found object indelibly altered definitions of art, the importance of the object being subsumed by that of the concept in the 1960s. This course examines how artists working before the boom of digital technology utilized other media, techniques and approaches to effect formal, conceptual and experiential dynamics comparable to those being investigated by new media artists today. The objective of the course is to provide students with not only knowledge of the immensely rich history of artistic creativity, but also a platform through which that knowledge might be utilized to reconsider new media strategies of artistic expression. It is the goal that through observation, discussion, reading and projects (both written and hands-on), students acquire mental tools to approach their own work with an expanded understanding of artistic possibility. Organized thematically, each class focuses on a different concept derived from the field of new media production and examined with regard to artistic precedents. The course focus primarily, though not exclusively, is on 20th/21st-century art. It is conducted as a combination lecture/discussion class. Critical theory is incorporated into the readings and discussions, but this is not strictly a theory course. The course has been conceptualized and designed to enhance understanding through a variety of means, from basic observation, to exploratory conversations, to more rigorous thinking informed by lectures, readings and focused discussions.
This course teaches the ins and outs of using image processing software with an aim towards some type of real-time use (e.g. a performance or installation). The class looks at ways to manipulate different visual media (time-based, still, vector, and rendered) in real-time to allow students to develop interesting real-time performance systems. While the focus of this class is on using Max for visual work (through a software package called Jitter), it also looks at how to integrate interactive elements (sound, physical interfaces, etc.) into the work. Class time is spent on interface design and software development issues as well. The class explores some interesting capabilities of the software in terms of real-time camera input and tracking, generative graphics systems, and media transcoding. Throughout the class students develop and share ideas on live performance as a medium for visual expression, and learn the software tools necessary to put these ideas into practice in the form of idiosyncratic performance systems. A final presentation in the form of a group performance will be arranged. Students should have some working knowledge of Max/MSP before taking this class.
This class focuses on the art of computer graphics and image processing. We explore the concepts of pixilation, image representation and granularity and the tension between reality and image. Students are introduced to the tools and techniques of creating dynamic and interactive computer images from scratch, manipulating and processing existing images and videos, compositing and transitioning multiple images, tracking and masking live video, compositing and manipulating live video as well as manipulating depth information from Kinect. The class uses Processing.
Course introduces student artists to the 4th dimensional concepts of TIME & MOVEMENT as the necessary building blocks of Art created by Photography, Video, Digital & Performance medias. Students will explore the attributes of TIME & MOVEMENT through project assignments that use the methods of photography, video, digital, & performance art. The course also includes a brief art history of 4D concepts spanning a wide array of subject matter from cave painting to contemporary art.
Video art has become one of the most widely used media art forms because it allows both the artistic concentration of photography & the free flowing imagery of movement. Students acquire rudimentary skills in shooting & editing while working toward a personal statement in video.