“The smartphone is not only the primary site for digital communication and consumption, it also hosts emerging forms of media production. Let’s investigate the potential of the mobile touchscreen as a creative instrument! This is a project based course, and we will explore by creating and testing a series of functioning web-based toys – including drawing apps, character creators, and writing tools. You can expect to sharpen your skills in javascript and design. “
In this course, students will be exposed to different qualitative research methodologies and disciplinary approaches to those methods (in terms of technique, ethical standards, approaches to citation, etc.). Students will practice using research methods to study subjects of their choosing and produce reflections, sketches, or prototypes based on weekly research findings. By practicing the act of “doing research” students will think about ways to incorporating research into their practice and better understand and articulate how research is already part of their practice.
When technology advances, teaching styles regress. Every new wave of technology touted as a boon to education — radio, TV, DVDs, the internet — has tended to revive the idea that the ideal class structure is the lecture, where faculty broadcast information to disconnected student recipients. Yet we’ve known for decades that lectures are poor ways to create learning experiences, and that actively involving students–with the class, the teacher, each other–is far more effective. The thesis of Imagination and Distributed Learning is that the range of possible experiences available online is larger, more varied, and more interesting than most of what is on offer from colleges and universities’ online courses. Students will read teaching and learning research, and do field work where they set out to learn something online and reflect on the experience, and then set out to teach something online, and reflect on that experience. The goal of the course will be for students to build up alternatives to existing (and often quite dull) online educational practices today. The final project will be a proposal, plus designs or partial implementation, for online tools or experiences that will create experiences users can learn from. These proposals will be presented to a panel of people managing existing online programs at NYU and elsewhere.
Today’s internet, made up of mostly text documents and two-dimensional images and videos, is the result of historical limitations in bandwidth, graphics processing and input devices. These limitations have made the internet a place where the mind goes, but the body cannot follow. Recent advances in motion capture devices, graphics processing, machine learning, bandwidth and browsers, however, are paving the way for the body to find its place online. This course will explore embodied interactions in the browser and across networks. Specifically, we’ll explore TensorFlow.js models like PoseNet and BodyPix, and Microsoft Kinect in p5.js and Three.js. Assignments will consider designing engaging embodied experiences for individual and social interactions online. Experience with Node, HTML and JavaScript is helpful but not required. ICM level programming experience is required. The course will have weekly assignments that explore embodied interaction online. Assignments will begin with exploring single points of interaction (i.e. one mouse or one joint), and progress to considering full bodies and multiple bodies in one browser. Students will have a 2-3-week final project with which they will delve more deeply into the subject matter in one piece of work. Students will have readings/watchings focused on embodied and networked user experience. Some influential works that will likely be assigned/discussed are Laurie Anderson’s “Habeas Corpus,” Todd Rose’s “The End of Average,” and Myron Krueger’s “Artificial Reality.” The course examples will be taught in Javascript using web technologies/frameworks. However, students are welcome to work in their preferred medium.
In this class students learn how to use data. If you can learn to live with uncertainty, you can make something beautiful and true. Students will learn about data as another form of evidence. We will collect our own datasets to learn about challenges and opportunities. We will explore sources of uncertainty, and how imagination and empathy can help uncover ways that data can lead to insight or alternatively, lead one astray. We cover basic stats principles to show how even properly collected data may lead you astray; we cover design principles, we introduce technical tools for visualizing data. Students collect their own data set, share reactions to assigned readings, and find their own examples of data usage that are positive or misguided.
What can cybernetics, the study of how we shape and are shaped by systems, teach us about the sexual and social reproduction of gender and sexism? How does sex become gender and what are the politics surrounding who gets reproduced? We will explore how social regulatory systems are encoded into technological platforms and disentangle how they produce social pressure and govern behavior through somatic exercises, discussion, and project making. In this class, we will not shy away from difficult conversations and work closely together to cultivate a space of openness and mutual support. Discussion and project-making is core to this class. Together we will read the work of scholars such as Donna Haraway, Ruha Benjamin, Paul Preciado, Silvia Federici, & Audre Lorde. Along with lecture, discussion, and in class activities, students will be encouraged to explore their own research interests and personal histories. When projects are discussed, we will practice communicating ideas through presentation as a medium and will co-create a culture of constructive feedback.
How can light, sound and design transform the human experience within a given space? How can psycho-geography be manipulated through audio-visual techniques? In what ways have and will technology allow spaces for sonic entertainment to be more immersive and experimental? Through an exploration of audio-visual techniques (i.e. VJing, MIDI-ing devices, sound synthesis, projection mapping, experiments with spatial sonic composition) along with discussions on how counterculture movements have used music and design as a vehicle for political dissent and community building, students will be invited to imagine new club spaces for social contexts beyond pure aesthetics. Assignments will include the development of different forms of interactive spaces for expression. Ableton (and free DAWs), MaxMSP, Isadora, and Unity will be used within this course.
Course description (optional): We are currently living in a society that operates under the principle that one body equals one agent, one vantage point, one identity. But emerging technologies may create a future in which the notion of a single personal identity becomes outdated. That future includes: machine learning techniques that make emulating the style and behavior of other people fast and easy; widely available AR/VR headsets that get people to identify with however many faces and bodies they choose, instead of just those they were born with; cryptocurrencies enabling the use of pseudonymous economic identities to transact across the planet in a permissionless manner. This is a course where we will get to explore and anticipate the utopian and dystopian aspects of this weird future of identity.
Masks have been used around the world since antiquity for ceremonial and practical purposes, as devices for protection, disguise, entertainment and bodily transformation, made to be worn or displayed. Sociologist Erving Goffman wrote about the everyday life as a masked theatrical performance. The performative aspect of our lives today is ever so present in our use of social media, where we present a curated version ourselves for the immediate visual consumption of others. In our `Selfies`, we can assume a multitude of identities and characters. Recent tools and platforms have evolved social media portraiture to an art form and have created new opportunities for artists to create and distribute interactive augmentations, forming new relationships between artists and viewers. This class explores the developing language of social media portraiture enhanced by Augmented Reality. Students will: – review masks in art history, leading up to today – ideate, design and develop an interactive mask (AKA effects/lenses/filters) – learn to use the Meta Spark software to create AR effects. This course requires CL: Hypercinema or equivalent experience.
A programming course where we’ll explore various techniques and solutions for tracking and sensing people or objects in space. Students will get familiar with the terminology and algorithms behind many sensing topics such as computer vision, depth cameras, positional tracking, coordinate mapping, machine learning, and pattern recognition. As these subjects are explored, we will also dig into communication, and how this information can be transmitted from one tool to another, for example using OSC, Spout/Syphon, MIDI, DMX/ArtNet. The goal being to use the right tool for the job and not limit ourselves to a particular piece of software. The first classes will consist of theory and in-class exercises covering these techniques, and remaining classes will be dedicated to a special project, which should use a combination of what we’ve learned to create a new work. Students will work in small groups to build this special project, but we’ll review proposals, milestones, and work in progress collectively on every class, encouraging discussion and collaboration.
In an age of pressing and complex problems like climate change, extreme inequality, and surveillance capitalism, “problem solving” is a central feature of innovation, design, and planning. But can these wicked problems actually be “solved”? And why does the cutting edge of problem solving look so limited? Machine learning. Predictive analytics. Algorithmic decision-making…Is planning for the future being outsourced to machines? In this class, we’ll take back control of the future by learning how it has historically been predicted, planned, and produced in board meetings, think tanks, writers’ rooms, and policy circles, and how those methods are being impacted by new technologies. During a series of discussions and hands-on workshops, we will learn specific, tangible, and collaborative practices for prediction and planning that can augment and transcend computational capabilities, making for marketable future-proof skills that can help redefine the future for humanity.
Time-based art, performance – and theater most specifically – should be perfect manipulators of experience. Many creators of time-based art look for the “universal” in content and overlook what we all have in common in form: brains. What happens in the minds of all truly happens (what happens in the lobby also truly happens). How can we use art to make our brains experience the same things? What behind-the-scenes work can we employ to manipulate experience. Film scoring works on us in ways we don’t perceive in the moment. Can we pay closer attention to this when making work? Sometimes what we have for lunch effects our experience of art more than the hours of labor that went into the most minuscule of decisions when making that art. Is empathy always ethical? Show don’t tell. How does the art take place in the room in front of us? How can public performance not be awkward? How do we experience a performance rather than watch it.
he virtual expansion of screens began during the 1960’s with the exploration of head-mounted displays. Since the 60’s, virtual reality has been explored in a multi-disciplinary context including philosophy, design, arts, behavioral therapy. Baudrillard, with his publication of Simulacra and Simulation (1981), declared that human experience is being replaced by a simulation of reality (HyperReality). His theories brought the dystopian narrative of the virtual to mainstream pop-culture, as seen in films such as The Lawnmower Man and The Matrix . Contrary to Baudrillard, Canadian VR Pioneer Char Davies brings a more positive perspective to Virtual Reality, “facilitating a temporary release from our haitial perceptions and culturally biased assumptions about being in the world, to enable us, however momentarily, to perceive ourselves and the world us freshly.” Throughout the class, the friction between Baudrillard and Davies will create the foundation of our exploration of Virtual Reality, where we will use room scale headsets and game engines to create meaningful “temporal experiences” exploring themes from behavioral sciences to narrative storytelling. We will be exploring ● existing VR projects, popular culture references and theory. ● concepts such as sense of embodiment (SoE), social VR design, and interactive storytelling techniques. ● methods for designing, modeling and rigging avatars for VR. ● live and pre-recorded animation. ● spatial audio techniques such as ambisonic sounds engines. ● packaging and distributing applications for social VR. This is a production class, along with a theoretical foundation, in which we will prototype projects with networking, inverse kinematics, raycasting and face tracking technologies to explore questions such as “how does the viewer become part of the experience?” and “how does the real space relate to the virtual worlds we design?” In the second half of the class, students will work in groups to build a final social VR project based on their exploration of the above framework.
If we consider drawing as one of the most ancient forms of interaction, it has the power to engage users like no other technology. With the right setup and call for action – I’m always taken aback by how a cup full of crayons and a pile of paper can bring the child out of a serious businessman. But how do you pass the strong barriers of users who are afraid to draw? In this weekend workshop we’ll examine various techniques that can spark an interaction which is all focused on the action of drawing. Workshop topics include an exploring into collaborative drawing platforms, interactive drawing installations, drawing machines and drawing as a form of interactive storytelling. We’ll discuss the differences between digital and analog drawing and how to marry the benefits of both mediums. We’ll cover the work of important artist and researchers who are creating inspiring work in the field of drawing and interaction such as Shantell Martin, Zach Lieberman, Tobias Gutmann, David Ha and more There will be various collaborative drawing exercises in class. This workshop is meant for students who wish to focus on drawing as the driving force of their interactions and possibly expand it into thesis. Each student will design and prototype an interaction which is based on the subject of drawing. Alon Chitayat graduated from ITP in 2015, He works at Google as a Sr. UX and motion designer and on a collaborative digital whiteboard called Jamboard.
Small, affordable single board computers enable you to blend the principles of Physical Computing with media playback and capture. This course uses the Raspberry Pi computer as a platform for creating portable devices that have the capability to display graphics, play video, play audio, take photographs, and capture video. As a foundation for the course, students will learn the basic workflow of using the Raspberry Pi computer for physical projects. This foundation includes an gaining an understanding of the Linux software, Python, and digital input and output. Students will work independently or collaboratively to build on this foundation to create their own media playback and capture devices.
Genspace is collaborating on this course with ITP so that students can learn science literacy through several specialized workshops that will take place Genspace – topics include Biohacking (with an introduction to CRISPR) Biomaterials. Students will create projects throughout the semester utilizing both Genspace and ITP resources. Additionally, students will learn the basics of biodesign and bioinformatics to help them frame and conceptualize their research and their projects and how best to use these skills ethically and responsibly in aesthetic and scientific ways. Since 2009 Genspace has operated a community biology laboratory in Brooklyn stemming from the hacking, biohacking, and DIYbio movements. It currently supports citizen science and public access to biology, biotechnology, synthetic biology, genetic engineering, citizen science, open source software, open source hardware.
As game and interaction designers we create systems and choices that can either prey upon our psychological foibles or help us avoid decision pitfalls. It is our responsibility to understand how we decide, to consider the ethics of the systems we create and to practice designing systems in a purposeful manner. Game Design & The Psychology of Choice will provide interaction and game designers with an understanding of the factors that influence behavior and decision-making by looking at the intertwining of cognitive psychology and economics through the development of behavioral economics. These disciplines study behavior on the individual and group level, often revealing some of the why behind the rules of thumb and folk wisdom that game designers come to intuitively. But understanding the why—why we fall into decision traps; why certain tradeoffs tax our brain more than others; why we are overconfident about our abilities; why certain decisions make us uncomfortable—allows us to more purposefully apply our design craft, both in and out of games. Finally, as a class, we will take what we learn about how we think and create series of game experiences based around key cognitive science concepts. Assignments may include: •Mod a cognitive science experiment into a game or experience •Analyze and present a game through the lens of cognitive science and behavioral economics •Create game or experience based around a particular insight from cognitive science or behavioral economics
Computers are able to understand human speech better than ever before, but voice technology is still mostly used for practical (and boring!) purposes, like playing music, smart home control, or customer service phone trees. What else can we experience in the very weird, yet intuitive act of talking out loud to machines? The goal of this course is to give students the technical ability to imagine and build more creative uses of voice technology. Students will be encouraged to examine and play with the ways in which this emerging field is still broken and strange. We will develop interactions, performances, artworks or apps exploring the unique experience of human and computer conversation. Students will learn how to use text-to-speech and speech-to-text technologies, voice assistant devices, generative text techniques, open speech APIs, Node.js, and conversational UI design. There will be weekly assignments leading up to a final project. ICM or comparable programming experience required.
Beginning with the release of Crowther and Woods’ “Colossal Cave Adventure” in 1977, the potential and unique affordances of computation as a means of storytelling have become more and more apparent. Combining approaches from literary theory, anthropology, computational creativity and game design, this class considers how narrative structure can be represented as data and enacted through computation, and invites students to implement practical prototypes of their own interactive and procedurally-generated narratives using a variety of technologies. Topics include (but are not limited to) hypertext fiction, “choose your own adventure”-style branching narratives, text adventures, visual novels, story generation from grammars and agent-based simulations. Students will complete a series of bite-size weekly assignments to present for in-class critique. Each session will also feature lectures, class discussion, and technical tutorials.