The Nature of Code is an intermediate course based on Daniel Shiffman’s The Nature of Code course at NYU ITP and was adjusted for undergraduate students. This course explores the fundamentals of programming, such as Object-Oriented Programming, and the application of simple principles of mathematics and physics in order to recreate natural behaviors in a digital environment. Prerequisites: This class uses p5(p5js.org) and requires Interaction Lab, Communication Lab, Application Lab, or similar programming background. Knowledge of other languages, such as Processing, three.js and OpenFrameworks, is also encouraged.
In this course students will learn the fundamentals of computation, software design, and web technologies, through a series of creative projects. The course is intended to equip students with the skills to develop artistic and business projects that include a significant computational component. Topics such as variables, functions, components, and functional and reactive programming will be brought together to create interactive applications, generative art, data visualization, and other domains. Within the framework of these creative projects students will develop a greater understanding of how computer programs operate, be exposed to various concepts used to create experiences and interactions, and become more familiar with some of the technologies that constitute the internet. This course is intended for students with no prior programming background. Prerequisites: None
Utilizing technological and scientific research / case studies / artifacts, this class introduces students to the topic of enhanced / extended perception and how technological augmentation allow us to sense and perceive alternative layers of our surrounding world, reconfiguring our understanding of what reality really is. Students will be asked to develop their own prototypes that demonstrate a conceptual or functional outcome on how perception can be extended, enhanced, or even hacked.
Ours is a time of unceasing image production. CCTV cameras wrap around the planet, machines make visual learning sets for other machines, hundreds of billions of hours of video material is livestreamed online every year, artists speculate about the possibility of making virtual reality for animals, and seemingly natural phenomena such as climate change can only be apprehended through risk simulations and data visualisations. Simultaneously, from deep fakes to computergenerated influencers and webcam models, digital images without real-life referents are set to become a part of social life, posing questions about the agency of media makers and consumers in an increasingly simulated world. Contemporary visual culture is not made by humans alone and sometimes not even meant for human eyes. What are we to make of aesthetics when they become automated? How can human collectives be thought from the perspective, or collaborate with, the semi-autonomous technological systems around us? What kind of conceptual capacities do we need to be able to think about the future of such media globally, and in specific geographical locations? In this class, we will explore the shi< that media theory and philosophy of technology have made towards the inhuman. From biometrics, insect media, and eco-media to so<ware art, augmented reality, and satellite imagery, we are witnessing a paradigm change in how we think about the relationship between human, animal, and machine agency in visual culture. Looking at a variety of contemporary media objects, we will read state-of-the-art theory, write a research paper, and collaborate on a post-human media studies syllabus spanning texts and objects from across the globe.
An attempt to better understand and participate in the communications revolution we are undergoing through an investigation of the nature and consequences of previous communications revolutions. Using readings ranging from Plato to Sontag to Kundera, the course will look closely at the history of spoken language, images, writing, printing, photography, film, radio and television. How were they understood? How were they initially used or misused? What were their effects upon social patterns, politics and thought? How did innovations occur? What can that tell us about the potential and potential influence of digital communication? Students will be asked to undertake innovative experiments of their own in forms of new media. Prerequisites: None
As no two cultures define “artificial” the same way, and no two define “intelligence” the same way, so too they will not define and imagine “artificial intelligence” in the same way. As China and the West engage in what is sometimes called a new scientific and technological race, the fundamental terms of a shared conversation across cultures may be missing or contested. As planetary-scale economies shift toward increasing automation, AI moves out of the laboratory and into the fabric of supply-chains, urban infrastructures, and everyday life. This research seminar will allow students to explore different ways that different cultural connotations of “face”, “city”, and “governance”, for example, imply different paths for the development of this fundamental technology. NYU Shanghai will host an international conference in the Spring 2020 that examine these same topics. Prerequisites: Interaction Lab / Application Lab / Communication Lab / What’s New Media
This class focuses on the curiosity behind the greatest discovery of electromagnetism. By replicating experiments both with magnetic and electrical fields, we will explore the major breakthroughs that enabled us to power up devices, connect people, and store information. During the course we will have seminar discussions analyzing texts that contextualize the lab experiments and we will work toward conclusions on the implications of these discoveries. We will analyze different perspectives that led to the development of theories about the electromagnetic field, radio waves transmissions, and the quantum properties of electrons. Students will propose their own creative experiments, linking their personal interests with how electrons behave. As part of their final project report, they will submit an essay describing the technical methodology, critical framework, and the results of their experiment. Throughout the course they will acquire a working knowledge of components like capacitors, lasers, antennae, and circuit prototyping tools. Prerequisites: None Fulfillment: CORE ED; IMA/IMB elective.
People use their bodies in the workplace whether they are dancers or athletes, managers or engineers. Physical wellbeing, social teamwork, and cognition may be affected by our movement practices. How do people use physicality and motion to think? What is the interaction between body, motion, place, and goals? We will explore these questions by building physical-computing-based systems that encourage us to bring movement into new times and places in daily life, that coach users and develop learning environments for movement practices, and that test our understanding of ways that we “think with the body”. In this course we will bring practices such as fitness, dance, sports, and martial arts into a series of interactive installations, movement learning projects, and workspace modifications built on computing, sensing and actuator technologies. In this course we will also explore these questions through review of existing creative projects in this area, readings, presentations, and knowledge-sharing sessions. Prerequisites: Creative Coding Lab, Interaction Lab or equivalent programming experience Fulfillment: IMA/IMB elective.
Data is at the heart of the increasing role technology has in our lives. Data collection and algorithmic processing are not only central to recent technical breakthroughs such as in AI and automation but have created new economic paradigms where data equals value and shape political approaches to power and control. Decisions based on algorithms affect society at large whether it’s changing the way we transport and distribute goods, or influencing the things we buy, the news we read or even the people we date. The world that algorithms see is data. For the average person, however, data is seldom more than an abstract idea. So what exactly is data? How is value extracted from it? And why should we care? How can we ethically balance the positive uses of data-driven systems with the threats they pose to discriminate and infringe basic human rights? This class seeks to untangle some of these issues practically and theoretically. Prerequisite: Creative Coding Lab or equivalent programming experience. Fulfillment: CORE AT; IMA/IMB elective.
Toys are not only for kids. Toys are part of our culture and an important medium to develop essential skills like creativity, problem-solving, and socialization. They can also be a great contribution in education, medicine, and business, and can improve the quality of life for children and adults alike. The emphasis in this class is on designing for play and entertainment. Students will be introduced to the underlying essential concepts in designing toys and they will create their own utilizing hand-making craft skills and new technologies. This course will equip the students with a basic knowledge about various design topics including: brainstorming, sketching, graphic design, concept development, basic mechanisms, 3D modeling, rendering, and rapid prototyping. This is a hands-on class and students are required to bring their imagination in addition to a willingness to experiment and explore creative solutions for class assignments. Prerequisites: Interaction lab / Communication lab Fulfillment: IMA/IMB elective.
Application Lab is an intensive project-driven course where students explore current challenges and opportunities at the intersections of emerging media and innovation through the lenses of design, prototyping and innovation. The course seeks to help students understand how these high-level concepts intersect with skills to form the basis for new applications of technology and human industrial art. At the end of this course, students will be able to think critically and holistically about not only what makes innovations possible but will also how to utilize emerging media technologies and ideas to bring innovations into the world that respect and acknowledge the values of design, iteration and innovation. Prerequisite: None. Fulfillment: IMA Major Other Foundation/Electives; IMB Major Emerging Media Foundation/ Interactive Media Elective.
Exhibition: Next class is an exploration and observation of the fields of exhibition design and museum studies. This class will explore how emerging and interactive technologies can be applied to museum and exhibition design to enhance visitors’ experiences. What is an exhibition in a museum of today and how should it be experienced? What is the role of a museum in contemporary society? How does it engage the audiences of tomorrow? The class discusses curatorial practices, various exhibition concepts and forms, museum visitor experience, and exhibitions’ social values. Students will visit and immerse themselves in many museums and exhibitions as a professional observer who will be asked to write reflections of their observations as an essay for each museum visit. Students will choose a research topic at the beginning of the class and they will start collecting materials, building objects, designing experiences, and writing a statement for their final exhibition based on this topic. Students’ design work will be frequently reviewed and given feedback by the instructor, classmates, and guest speakers/critics. After the midterm, the instructor will initiate a collaboration with a local museum or art space. Based on the specific circumstances, students will face a design challenge to propose an exhibition proposal or provide a creative solution to the partner organization. By the end of the course, students will install and present their work as a group art show in the student art gallery. Prerequisites: Interaction Lab or Creative Coding Lab Fulfillment: IMA Major Electives; IMB Major Interactive Media Elective.
In this foundation course students will be asked to think beyond the conventional forms of human computer interaction (i.e. the keyboard and mouse) to develop interfaces that consider the entire human body, the body’s capacity for gesture, as well as the relationship between the body and its environment. Students will learn the fundamentals of electronics and programming as they build projects using the Arduino microcontroller platform. Arduino is a small computer based on open source hardware and software. When used in conjunction with various sensors and actuators, Arduino is capable of gathering information about and acting upon the physical world. In addition to these physical computing techniques, students will also learn to harness the methods of traditional computation. The fundamentals of programming will be explored using the Processing programming language. Processing has a simplified syntax and an approachable computer graphics programming model, making it an ideal platform for first-time programmers. Students will gain a deeper appreciation of the expressive possibilities of computation as they learn to author their own software and systems and not simply use off-the-shelf solutions. Additional topics will include algorithmic drawing and animation techniques, digital modeling and fabrication, data exchange, manipulation, and presentation, as well as control of images, audio and video, including computer vision techniques. Structured weekly exercises are aimed at building specific skills, however students are free to pursue their own diverse interests in their midterm and final projects. Prerequisite: None. Fulfillment: CORE AT; IMA Major Other Foundation; IMB Major Emerging Media Foundation.
User experience design (UXD, UED, or XD) is the process of enhancing user satisfaction with a product by improving its usability, accessibility, and desirability provided throughout the user’s interaction with a product. The class is designed for those who are passionate about creating user-centered experiences with interactive media. Students are encouraged to empathize with users, engaging them to make informed design choices from prototype right through to project completion. Prerequisites: None Fulfillment: IMA/IMB elective.
Virtual Reality and Augmented Reality represent, respectively, visions of “being” somewhere else or augmenting your present environment. These visions are not new, but new technologies have made it possible to produce experiences unlike anything before, particularly through the use of headsets, spatial audio, touch sensors, and custom location-based installations. These new technologies are becoming small, powerful, and inexpensive, and as a result we are witnessing the birth of a powerful new medium, new art form, and new industry – all very quickly. The speed of VR and AR growth has created both opportunity and confusion. “VR / AR Fundamentals” takes a long, deep perspective. We will overview such basic elements as audiovisual resolution and fidelity; spatiality and immersion; other senses such as touch, smell, taste (and even mind); input and interactivity; and live and social. We’ll look at distinctions such as cinema versus games, movies versus models, public versus personal, real world versus fantasy worlds, linear versus interactive, and narrative versus ambient. These elements and distinctions will be presented partially as technical but in an understandable way for general liberal arts students, and will rely heavily on experiencing content and keeping up with current events. That’s the first half-semester. The second half-semester we’ll concentrate on collectively producing a series of timely and relevant projects, all short, entertaining, and useful to others exploring the world of VR / AR. Prerequisites: None Fulfillment: IMA Major Electives; IMB Major Interactive Media Elective.
In this foundation course students will explore the possibilities of emerging media by successively producing projects that make use of digital images, graphics, audio, and video. The course is designed to provide students with a framework to effectively communicate and tell stories through digital means. Students learn through hands-on experimentation in a laboratory context and the principles of interpersonal communication, media theory, and human factors will be introduced in readings and investigated through discussion. Adobe Creative Cloud and other relevant software applications will be examined to establish a diverse digital toolkit. Both traditional and experimental outputs will be explored. Weekly assignments, group and independent projects, as well as documentation of projects will be assigned in each of the core areas of study. Prerequisite: None. Fulfillment: IMA Major Other Foundation; IMB Major Emerging Media Foundation.
Intro to Game Animation builds a foundation for animating in interactive media. From a basic overview to a deep dive into the animation process, this course will teach students how to craft a performance to create unique characters and tell a story. Students will be taught to look through the eyes of an animator, investigating how successful games utilize animation principles, and how these principles and techniques functionally serve overall game design. This class encourages students learn through practical experience, teaching familiarity with the animation principles in a 3D environment. This class is beneficial for an artist seeking to be an animator. This class will also benefit developers and designers by providing them a better understanding of the importance of animation and how it fits into the game development pipeline, providing a comprehensive understanding of the game industry as a whole.
Music and Gameplay is an intensive course concerned with digital games in which the gameplay is fundamentally influenced by, or oriented around a musical system. In this course, students will engage with music games in a variety of ways: through critical play, design practice, and hands-on development. This multifaceted approach will foster an understanding of how interactive game mechanics can be linked to music expression. Throughout the course, we’ll be drawing inspiration from a variety of music games across 3 major categories: