The spectrum of Extended Reality (XR) – encompassing Virtual Reality (VR), Augmented Reality (AR) and Mixed Reality (MR) – brings with it several opportunities for new possibilities for game design, interactive storytelling, and more. While implementations of these technologies and associated theory have existed for a number of years, the recent technical and commercial resurgence means that developing skills in critical thinking and creative aptitude with regards to AR and VR is incredibly timely. To this end, rather than studying only contemporary discourse around XR technologies which frames it as a new development or passing fad, this class will also look at more well-established principles of both AR and VR – such as object presence and other aspects of existing theory. Furthermore, it analyses such technologies through a theoretical and critical lens, placing them within the wider history of the arts.
Code Lab 2 is a continuation in exploring how to craft game with programming. In Code Lab, we examined how to make games in openFrameworks, starting from scratch. This class will be a workshop, building off of that knowledge, but focusing on learning how to work with code that is already written. Students will learn to work with a new Integrated Development Environment (IDE), eclipse, learn to work with a version control system, and work in depth with Java and Processing. Over the course of the class, students will be given several versions of classic games (Pong, Space Invaders, Asteroids, etc.) that are incomplete or have an obvious bug. They will learn to read the code, identify how to correct the issue with the game, and then eventually modify it to make their own new version of the game. These skills are essential to work with code from other developers, whether they are members of the same team, open source projects, or examples provided in tutorials and readings.
Note this class is called “Devised Theater: History and Practice.” This intensive focuses on both historic evolution of ritual-based/early theater models through contemporary theater philosophies (accentuating history of Futurist/Dada theater innovations to present), and on anatomizing the nature of performer, performance, story and storytelling via the non-traditional philosophies and methods of contemporary experimental theater. The class will be rigorously participatory in terms of discussing/physicalizing these experimental methods and will culminate in the creation and performance of simple class collaboration-generated stage narratives. Students will investigate the meaning and application of physical/environmental ’neutrality’ on stage as they simultaneously investigate and define for themselves the most essential markers needed for the viewer to perceive ‘story’ in performance. As the staged pieces are constructed from these anatomized building blocks of performance and story, more complex qualities of character, identity, archetype, mannerism, linguistic disfluencies (verbal and non-verbal) and psychological subtext will be introduced as tools for each performer’s role in the story. In the final phases of piece creation, simple analog elements of music, sound, light, mask, craft materials, dance, virtuosic/specialized skill, props will be introduced as tools. The final performance will aspire to clear and effective applications of the performance/story elements discussed (or discovered) in class. Techniques and exercises derived from the worlds of Futurism/Dada, Richard Maxwell, Blue Man Group, Elevator Repair Service, Ann Bogart, Joshua Fried, and others will be discussed and employed.
“””Portraiture stands apart from other genres of art as it marks the intersection between portrait, biography, and history. They are more than artworks; when people look at portraits, they think they are encountering that person,”””” says Alison Smith, chief curator at the National Portrait Gallery in London. For thousands of years, artists have used cutting-edge tools and resources to create portraiture, giving viewers a glimpse into the subject’s life. A successful portrait embraces technology to bring the viewer closer to the subject but is not overshadowed by it. In this course, we will delve into portraiture through the lens of volumetric capture using the Depth Kit system. Through hands-on assignments, students will learn the entire pipeline of volumetric capture, from configuring the system to capturing our subjects and final output. Simultaneously, we will focus on fundamental aspects of portraiture, such as lighting, storytelling, production techniques, and historical foundations. The course will explain the techniques and considerations involved in creating volumetric portraits. We will explore various approaches to capturing subjects, employing advanced technologies to record their presence in 3D. Students will gain proficiency in the Depth Kit system to produce high-quality volumetric portraits that can be integrated into different mediums, including game engines, augmented reality (AR), or traditional 2D outputs.”
This course focuses on designing, developing and delivering multimedia live performances via a virtual platform. The class will have an emphasis on experimenting with different possibilities of virtual performances, pushing the boundaries of the performative medium, and using emerging technologies to create experiences that allow for the unfolding of engaging narratives, and/or generate compelling visuals in real time. We will look at various examples of both online and offline performances, explore how we can apply the technologies we have learned to design performative systems, and discuss methods we can use to make our performances more engaging. Students will practice quickly coming up with ideas and performing in class. A few weeks into the course, students will propose final project ideas and then develop the performances in the following weeks with support from the instructor. The class will culminate in a virtual event featuring solo and/or group performances by the students.
This interdisciplinary course combines ethnographic readings, representations, and interpretations of city and urban cultures with a video production component in which students create short documentaries on the city of Accra. The interpretative classes will run concurrently with production management, sights and sound, and post-production workshops. The course will have three objectives: (1) teach students the documentary tradition from Flaherty to Rouch; (2) use critical Cinema theory to define a document with a camera; and (3) create a short documentary film.
This interdisciplinary course combines ethnographic readings, representations, and interpretations of city and urban cultures with a video production component in which students create short documentaries on the city of Accra. The interpretative classes will run concurrently with production management, sights and sound, and post-production workshops. The course will have three objectives: (1) teach students the documentary tradition from Flaherty to Rouch; (2) use critical Cinema theory to define a document with a camera; and (3) create a short documentary film.
The foundations of Max, a powerful visual programming language for music and multimedia, will be covered in this course. We will examine how computers can be utilized to create situations for music creation, performance, and collaborative improvisation as well as applied to building interactive, generative music. In addition to learning Max’s fundamental building blocks, we will also use fundamental music theory as a tool to better understand music making. We will create programs that examine rhythm, melodies, chords, scales, and recognize other qualities of music like timbre, texture, and dynamics while taking into consideration the principles of harmony, melody, and rhythm defined in basic music theory. The final will require you to develop a collaborative piece of interactive computer music, a collaborative performance environment, or another final project that has been discussed and agreed upon together. This class does not require any prerequisite programming skills or prior music theory knowledge.
This course focuses on three important areas in music technology: spatialization, computer aided composition, analysis & synthesis techniques. In each area, concepts & implementations will be explored in a variety of artistic & technological contexts. Students will work with the latest technologies including IRCAM Tools, Spat plugin, Max Bach library, Ambisonics, & Wave Field Synthesis. The course includes a 3-hour weekly lecture, 3-hour studio lab, & workshops at IRCAM. This course is taught in collaboration with IRCAM in Paris, one of the world leading institutions in computer music and acoustics.
Advanced work in studio repair and maintenance focusing on digital equipment. Students will develop practical skills tracing schematics, using test equipment, and replacing components as required daily in the studio environment.
A continuation of MPATE-UE 1010. This is an advanced & detailed study of the audio-visual production & post-production process including digital recording techniques, with special emphasis on synchronization & the interfacing of SMPTE time code. Sound design, advanced Foley topics, * creative workflow in audio post production will also be discussed.
“As game and interaction designers we create systems and choices that can either prey upon our psychological foibles or help us avoid decision pitfalls. It is our responsibility to understand how we decide, to consider the ethics of the systems we create and to practice designing systems in a purposeful manner. Game Design & The Psychology of Choice will provide interaction and game designers with an understanding of the factors that influence behavior and decision-making by looking at the intertwining of cognitive psychology and economics through the development of behavioral economics. These disciplines study behavior on the individual and group level, often revealing some of the why behind the rules of thumb and folk wisdom that game designers come to intuitively. But understanding the why—why we fall into decision traps; why certain tradeoffs tax our brain more than others; why we are overconfident about our abilities; why certain decisions make us uncomfortable—allows us to more purposefully apply our design craft, both in and out of games. Finally, as a class, we will take what we learn about how we think and create series of game experiences based around key cognitive science concepts. Assignments may include: •Mod a cognitive science experiment into a game or experience •Analyze and present a game through the lens of cognitive science and behavioral economics •Create game or experience based around a particular insight from cognitive science or behavioral economics”
This course introduces students to modalities for creating site-specific and immersive art and performance. Assignments will examine the work of artists who challenge the limitations of the physical, psychological and transactional spaces that have come to define conventional production models. Students will regularly receive prompts from which collaborative work will be workshopped, generated and presented. The sites and practices explored will de-center script/text as spine, institutional space as gathering place, linear storytelling as narrative, and separation between audience and artist as social contract. Through group performance projects and presentations, students will investigate how Site evokes Narrative and Event differently in brick & mortar, virtual, historic, liminal, dead, found, contested, democratized and community spaces. Our work will unpack the challenges and opportunities presented when we relinquish creative control of such unfixed elements as serendipity, impermanence, improvisation, audience agency, public space, weather, and pandemic.
Throughout history, musicians have channeled their creativity into outrageous fashion statements and invented personas: think MF DOOM, Sun Ra, Ghostface Killah, Daft Punk, Leikeli47 and Rammellzee. By embracing their alter egos in extreme and outlandish ways, artists have found their authentic creative voices. This course will introduce participants to the art of masquerade using their resourcefulness to create costumes from found materials, and performance as an exploration in creative expression using new media and technology. Students will be introduced to ideas surrounding abstract storytelling, experimental audio video production, and A/V performance using a combination of technical and hands-on approaches. This course requires CL: Hypercinema or equivalent experience. Prerequisite: CL: Hypercinema (ITPG-GT 2004)
To be a VR creator, it’s not enough to learn the hard skills—it’s also our responsibility to prime ourselves for the human impact of our work. As a means to design VR that is both enjoyable and accountable, this class proposes we borrow design principles from Hedonomics, a branch of ergonomic science that facilitates pleasurable human-technology interaction. Through the Hedonomic Pyramid, we’re able to section our thinking off into regions (Safety, Function, Usability, Pleasure and Individuation) and map out industry-tested VR design guidance for each. The result is a hierarchical checklist of proven principles, specifications and practices—that promote a culture of inclusive and holistic design—built to serve as a quickstart guide to designing accountable VR interfaces and systems. This class, divided into units that represent each level of the Hedonomic pyramid, will unpack both technical and conceptual strategies for creating VR, from visual interface fidelity to avoiding locomotion cybersickness to designing safer social VR spaces.
“In this course, we will explore how to create narratives that leverage our lesser used senses like touch, taste and smell as well as lesser-known ones like space, time, balance and scale. We will dig into the history of experiential storytelling, starting from immersive theater and Smell-O-vision to cutting-edge haptics and mind-bending illusions of proprioception. To help center this back in practical applications, we will also explore how this evolving art is commonly used in exhibition design, experiential marketing and brick and mortar retail. The class will be a healthy mixture of game theory as well as experienced based learning (meaning there will be a couple field trips and multisensory VR projects to explore). A basic knowledge of game engines is ideal but not mandatory because we will be using predesigned templates in Unreal engine to be experienced and manipulated in real-time through virtual reality hardware.”
“How can motion capture (MoCap) be used to archive, preserve, and share intangible heritage forms, such as performing arts, rituals, and other social practices and traditions? This course approaches motion capture through the lens of ethnography — drawing on techniques of observation, participation, and qualitative design research. This class will offer an overview of different motion capture technologies, such as 2D-3D pose estimation and depth mapping, with a practical focus on learning the OptiTrack system at ITP. We will start by covering the basics of OptiTrack and build up to other workflows and techniques used across animation, game design, and virtual production (e.g. OptiTrack to Unreal Engine or Unity).” Prerequisite: CL: Hypercinema (ITPG-GT 2004)
“What is the medium of memory? In this 14-week studio class, we will dig into this question through creative storytelling. Starting from a lens-based practice, this class will introduce traditional and bleeding-edge documentary methods to inform our own varied approaches to activating archival material. Through weekly “readings” (articles, podcasts, films), written reflections, and creative assignments, we’ll explore: • how technology has impacted our relationship to memory; • how visual interventions can can surface alternative narratives; • how to make under- and unrecorded histories visible, and call into question the power dynamics embedded in “official” records; and • how we might recast objects and sites of memory-keeping, like heirlooms, journals, and memorials, as a mode of engaged preservation. Mid-way through the course, students will identify either personal or collective histories to open up to their own individual creative reexamination, memorialization, or transformation––each producing a final project with the technology and approaches of their choosing that serves to answer the question we started with––what is the medium of memory?”
This course explores the fundamentals of storytelling through animation and takes students from traditional animation techniques to contemporary forms. In the first part of the course, students will focus on traditional animation, from script to storyboard through stop motion and character-based animation. The course then examines opportunities afforded by new technologies, such as interactivity, projection mapping and game engines. Drawing skills are not necessary for this course, however students will keep a personal sketchbook.
This course would focus on the questions of 1) what makes people healthy? and 2) how can we design tools and environments that support healthy lifestyles? Key topics to be covered include public health concepts like the multiple determinants of health and the social-ecological framework, plus a little evolutionary biology; the role of behavior in health, key tenets of behavioral economics and behavior change strategies; and systems thinking concepts from Donella Meadows and others. Students will come away with a much more sophisticated understanding of the complex system of factors and forces that affect people’s health; understanding of key systems concepts and some techniques for understanding systems; and experience designing for behavior at scale. A potential final project could be to reimagine/redesign a popular commercial service so that it would have a more health-producing impact — or, alternatively, to focus on designing changes to the ITP environment that would promote better health for students, faculty and staff. About Steve Downs: www.stevedowns.net
This introductory course will focus on giving students a practical understanding of how to make ‘serious’ video games. That is, games that challenge the player to think and learn. This is a class where we will be ideating and producing playable games each week over 7 weeks using the open-source Godot game engine. By the end of the course, you’ll have a small portfolio of ‘serious’ games to show and build upon. Throughout the course, students will become familiar with concepts such as iterative design, play testing, object-oriented programming, user-interface design, animation and basic art for 2D games. They will take a design-based approach to content creation, developing a proficiency in the Godot game engine and the overall indie-game development pipeline.
In an era of remote everything, how can we create artwork that brings us back together IRL? This course explores our connection to physical objects within the context of community. How can sculpture, installation, immersive, and public art nurture our neighborhoods via collaboration, play, ritual, self-expression, and awe? Students will work collaboratively to radically imagine bold, sculptural, immersive works using innovative and lo-if techniques integrated with technology. Hands-on workshops include experiments creating found sculptures, AR prototypes, projection mapping, real-time interactive multimedia content, and more. We’ll reference ancient monuments, sacred objects, NYC relics, street art and contemporary works to envision new artifacts that create awareness by reflecting the needs of our communities. Prerequisite: Comm Lab: Hypercinema About Ali Santana: http://www.alisantana.com/bio
Where does healing reside in media-making? How do we approach creating artistic processes and tools that move towards minimizing harm, supporting collective care, and understanding what healing means for ourselves and with one another? This course examines socially-engaged artistic processes and frameworks that reconstruct, reclaim, and decolonize ‘healing.’ Together, we will gather embodied data from our bodyminds, build language through readings, and map out artists in the field exploring disability, racial trauma, queerness, and diaspora within media and performing arts. Subsequently, we will develop our own processes as our final project: weaving together storytelling, embodied strategies, tool-making, performances, and/ or space-making.
As you know, projection mapping and Light Art are becoming popular again because of large-scale pop-up installations worldwide: ARTECHOUSE, SuperReal, Meow Wolf, and TeamLab. Technology has advanced over the years, but how people enjoy light art have not changed so much. How do your ideas and artwork fit into these site-specific installations? This class is for anyone interested in creating a site-specific installation using mapping technologies to create new experiences for the public audience. This class guides students through conceptual and technical processes of project and artist development. It consists of three parts: Project & Artist Development, Projection Mapping, and LED Mapping. We will research and discuss the history of visual artwork, public engagement, and technical exercises using real international contests and festival sites. The student will learn the latest Projection and LED Mapping techniques using Madmapper. And we will also focus on advanced techniques like multi-projector projection, projector calculation, Interactive Mapping, and software & hardware to culminate in a final project. The class will also invite guest speakers to discuss the nuts and bolts of their art and business. About Chika Iijima: www.mappathon.com, www.imagima.com
In this course, students will explore the fundamentals of augmented reality by dissecting the interaction between camera, computer, and user. Each week we will focus on a different AR modality: image, face, body, environment, and object, and consider their real-world applications. Through weekly explorations, we will examine the existing affordances of AR as well as their impact. This course will culminate in a final project, and our tool of choice will be Lens Studio. Course syllabus: https://docs.google.com/document/d/1wMWVnDdzgz2bbsCNp7jRAiCe1job4glq61o67sWAP00/edit?usp=sharing
The goal of this class is to gain an understanding and proficiency with Computer Aided Design (CAD). We will become familiar with CAD software, mechanical design, and simulation. The class will cover common CAD modeling techniques. We will use our designs to get physical parts made as well as use them in virtual projects. We will create parts both real and impossible.
This course will explore the history and meaning of the ubiquitous concept of “cursed” media, and provide students with a survey of digital art tools for the creation of their own cursed animation, video, photography, music, and web art. Many people were first introduced to the concept of cursed media when it exploded into mainstream internet discourse in 2016 with the @cursedimages Twitter account, which posted found photos bound by their unsettling effect on the viewer. Cursed media predates this account, however, stretching back to medieval notions of cursed objects. We will demonstrate how throughout time, cursed media has functioned like a slip of the tongue that provides a window into the cultural unconscious, where we encounter uncensored thoughts and feelings about race, gender, class, and what it means to be human. From Amazon Muzak generators to Artbreeder’s GAN tools for image creation, from machine learning text generators to robots who work at Walmart, cursed media and tools for its creation bring into view the ways that that culture reacts to tension between the increasingly precarious position of human beings in the capitalist 21st century and the threat of human erasure by the powerful forces of nature. Students will be introduced to digital art tools for creating music, manipulating photography and video, working with 3d animation, and building web art. Students will attempt to create their own cursed media, and in the process will gain a deeper understanding of the unconscious biases and ethical implications of contemporary digital creation tools.
This course will use the open source COVID-19 Impact Dashboard as a basis to explore ways to humanize the unfolding data on the coronavirus pandemic. Students are invited to collaborate on the COVID-19 Impact Project. Students will discover how data flows from public github repositories and tools needed to visualize the data. We will review other data-centric open source projects related to COVID-19 and discuss the questions they are trying to answer or problems they are trying to solve. We will examine historical and contemporary data visualizations. Using data visualization as a scaffold, we will explore ways to support community driven mourning and memorialization. Students can choose to participate as javascript coders, p5js explorers, UI/UX designers, citizen journalists, or data science explorers.
Computational simulation and videogame engines offer thinkers and makers a new way to reflect on the question “If I can make a world, what would it be like?” In Worldmaking with Unity, students will be exposed to various theories and approaches of worldmaking, and realize their own creative visions by constructing an original, conceptual, and playfully simulated world with indie game development engine Unity. This seven-week seminar/studio course will include a gentle introduction to computer programming, 3D modeling, character and scene design, and rendering techniques with Unity, as well as related production tools such as Blender. Projects may address system, agency, narrative, generative design, critical computing, and more. Student projects created in Unity can be compatible with augmented reality (AR) and virtual reality (VR). This course is designed as an intro-level game design and development course. There is no prerequisite. More advanced production techniques such as shader language and AI might be introduced as optional topics only. More information at: https://docs.google.com/document/d/1UppRjSyFlMpGPNLMowqKmP7wwq29s5Oexnpvrj4Gubs/edit?usp=sharing
Cooking programs with an image based language is a fundamental skill in the production and design of modern digital processes. Visual programming is not only an alternative way to code, but a solution to approach generative and interactive media. This class reviews the past, present and future of visual programming languages used to procedurally generate and manipulate media such as Max/MSP(Nato.0 55 3d), Isadora, Quartz composer, Touch designer, Houdini, cables.gl and Unreal Engine among others. The core of this course is the study of Unreal Engine’s Blueprint Visual Scripting system as a way to produce an interactive program in an executable form using only Visual Programing. We will study how to create actors, functions, interfaces and how they communicate with each other. We will also take a look into 2 other visual editors, The material/shader editor for the creation of HLSL like shaders and visuals and the new Metasound editor for the manipulation, generation and sequence of sound within the engine. A general understanding of Unreal Engine is a prerequisite for this class. Students will learn how to use blueprints to produce an interactive program that can be a video game, an installation or a Real Time digital Performance.
Listening is not something we do through hearing alone. Engaging with multiple perspectives, from Deaf studies and critiques of ableist hearing ideologies, to the possibilities and pitfalls presented through machine listening and imaginative sonic speculation, we will playfully deconstruct and question what it means to listen at all. Each week, readings will be assigned and students are asked to respond to prompts in the form of light-weight exercises that will orient most of our in-class discussion. Time will be spent discussing readings, presenting, and providing critique for each other’s projects in order to help draw connections between theory and practice. Through in-class presentations, we will encounter works by Christine Sun Kim, George Lewis, Pauline Oliveros and read from Jennifer Lynn Stoever, Tina Campt, and François Bonnet. The class has no technical prerequisites, and students are welcome to respond with works using tools and techniques from other classes (audio/video presentation, programming and physical computing, installation and fabrication, etc.). Through this work, we will unpack how the way in which we attend to the physical world and its inhabitants through our listening has real consequences. In much the same way our interpretation of the world is informed and influenced by factors outside of ourselves, our capacity to listen is conditioned through societal, political, economic, historical, and racial dimensions. We will disentangle listening from hearing and consider listening as a practice that begins not with how we hear sounds but how it allows us to (or prevents us from) interfacing and relating to our exteriors (and interiors). By the end of the class we will attempt to converge and coalesce our own ideas and perspective of what it means to listen.
This course is a unique collaboration between the Collaborative Arts and IMA Tisch departments, and CultureHub at La Mama. During the pandemic many performing artists moved their work online, leading to an increasing acceptance of experimental practices that their predecessors developed in on-line work for the past 30 years. In Experiments in Hybrid (IRL/URL) Performance, students will have the opportunity to design, prototype, and present collaborative projects that build on this tradition, blending both physical and virtual elements. Over the course of the semester, students will have the opportunity to study at the CultureHub studio where they will be introduced to video, lighting, sound, and cueing systems. In addition, students will learn creative coding fundamentals allowing them to network multiple softwares and devices generating real-time feedback systems. The class will culminate with a final showing that will be presented online and broadcast from the CultureHub studio.
In Jorge Luis Borges’s 1941 short story, The Garden of Forking Paths, a character named Ts’ Ui Pen seeks to build a labyrinth in which all men will lose their way. Borges reveals that labyrinth to be a sprawling book in which all moments in time are followed by every possible subsequent moment. Multiple futures and timelines exist as one in this garden of forking paths. Fast forward to today, where we find that Ts” Ui Pen”s seemingly incomprehensible vision has come to fruition via video games and interactive fiction. Both often eschew linearity and instead offer users the opportunity to drive the story and shape their own narrative experience. “In this course, we will first explore the trajectory of interactive fiction, from its early overtures in works by Borges and Italo Calvino, forward through Agusto Boal”s Forum Theatre, early computer text adventures, Choose Your Own Adventure books, non-linear film, and into the various digital interactive fictions of today, such as hypertext literature, adventure games, and visual novels. “From there, students will embark on their own adventure through interactive creative writing. The journey will begin with students taking apart and diagraming existing interactive works, and using digital tools to craft and share their own short interactive scenes and dialogs. We will confront the challenges and limitations of writing interactive story and establish some key critical theories governing non-linearity. The course will culminate in students designing and writing a complete short work of interactive fiction in a medium of their choice.
Augmented reality (AR), Virtual Reality (VR), “AI (Artificial Intelligence and Machine Learning) art, projection mapped art, NFT art, as well as other types of digital artworks, have come to be an integral part of new media culture. The genesis of and their relevance to new media are cultural imperatives to study and analyze — from a creative, historical, psychological, philosophical, marketing, and technological perspective. Many artists (painters, photographers, sculptors, filmmakers, animators, and writers), scientists, and technologists at NYU, as well as nearby VR World, Artechouse, and Hall des Lumières , and other NYC organizations are central to producing realistic and immersive three-dimensional environments ” AR, VR and projection mapped (360 digital imagery) worlds. Through lectures, group discussions, GoogleDoc reports, field trips to museums and new media organizations, and workshops specific to new media innovations and applications, students will gain a framework to understand the importance of these evolving technologies and their impact on the arts, ecology (sustainability initiatives), social justice, and behavioral science. Students from varying creative and technical backgrounds will participate in the development of new media art projects, intended to be showcased at the Gallatin Arts Festival. They will be introduced to and encouraged to take advantage of NYU”s LaGuardia Studio, LaGuardia Co-op, and LinkedIn Learning instructional tutorials, as well as using their own resources to aid in developing their new media projects.
Multichannel Media Installation & Performance is a course designed for composers & artists who want to work in a performance or installation context with immersive sound & image technology. The course focuses on software & hardware workflows for the creative applications of multi-channel sound & immersive video for the creation of fixed, generative, reactive, performance-based, & interactive systems that can be experienced in a gallery context or a live performance. Students will develop a semester-length project to use scale & immersion to creative effect. The course will feature regular creative critique as well as an overview of relevant interaction design strategies for creating interactive spaces using sensors & cameras.
This course explores the art and craft of mixing records, with special attention to “mixing in the box” (via a digital audio workstation). Focus on methodology and technique, with particular emphasis on establishing balances, using such tools as compression and automation to enhance dynamics and develop unique coloration. Examines intersection of technology, budgets, and the marketplace. Students execute their own mixes, with guidance and critique from the instructor. Basic level of DAW proficiency required.
An introduction to MIDI (Musical Instrument Digital Interface) with an emphasis on sequencing, production and arranging techniques. Open to students without previous experience in music technology.
A general introduction to the fundamental concepts of music technology, including: MIDI and sequencing, the basics of digital audio, sound recording, mixing and sound synthesis,. The course will also briefly overview advanced topics and applications in the field.
Introduction to the physical aspects of sound, psychoacoustics, basic electricity, principles and practice of magnetic recording and an overview of the recording studio, including an introduction to multi-track recording techniques. Students perform various duties just as they would in a professional recording session with live musicians in the recording studio. Open to students without previous experience in recording technology.
An introduction to maintenance and troubleshooting concepts used in the recording studio. Procedures discussed are those necessary in utilizing sophisticated audio equipment and understanding essential aspects of studio design.
This workshop seeks to bring students from varying backgrounds together to engage in evaluating and sharing digital new media for the Internet and other new media art mediums. Each student brings to the class a set of experiences and skills, such as research, writing, design, film, music, photography, computer gaming, performance, animation, computer literacy, software knowledge, virtual reality, augmented reality, mixed reality, A.I. (Artificial Intelligence) Generative Art experiences, among others. The class will discuss new media concepts, content strategies, and frameworks that bridge theory and practice. Through lectures (including a survey of digital new media innovations), group discussions, virtual or other lectures and/or workshops, students will develop individual projects, based on their new media skills. The class intends to be a part of the development of the Virtual Reality (VR) Museum (“Virtual Museum XR”), or other Gallatin arts initiatives, such as Rabbit Hole. Digital new media projects may include digital photography, animated films, podcasts, sound art installations, TikTok, music videos, VR, AR, A.I. Generative Art, to name a few. Class projects, readings, and week-to-week journal-keeping reports are essential components of this workshop. They will reside in a designated Google Docs site, specific for this class. Students are encouraged to supply their own media and take advantage of NYU’s LinkedIn Learning new media tutorials and access NYU’s LaGuardia Studio and LaGuardia Co-op hardware and software opportunities.
This course examines mobile food networks as a method to research and map the contemporary city. In Shanghai, over the past few years, there has been an enormous transformation in the way the city feeds itself. Street food stalls, restaurants and marketplaces have all migrated online. This tendency towards virtualization was intensified during the Coronavirus pandemic, when, during lockdown, people used their phones to order food, which was delivered straight to their door.This course treats mobile food delivery as a media infrastructure. It examines how these new delivery systems form part of a distributed urban ecosystem that underlies the emergence of a Sentient City. Students will use the tools of critical cartography and digital storytelling to explore the cultural, economic and political issues that are raised by the explosive growth of mobile food delivery. Research topics include the economics of company platforms; logistical networks; the reorganization of food production; the socio-economic conditions of delivery workers; changing cultural habits of urban residents; the shifts in the city’s built environment as well as the design of the apps themselves. Prerequisite: None. Fulfillment: IMA Major Electives; IMB Major Interactive Media Elective.
In this era of virtual production, time-based media faces new opportunities and challenges in terms of pipelines, workflows, and distribution. Decentralizing, hybridizing, and outsourcing among film studios, production houses, broadcast design, interactive studios, and the gaming industry have become major topics of discourse in academia and industry. This course focuses on the history/context, present practice, and the emerging trends of VFX studies and its applications. Through collaborative research with academia and industry, the course investigates the theory and practice of VFX studies and further examines the feasibility of emerging technologies through the spirit of entrepreneurship. Prerequisite: Interaction Lab / Creative Coding Lab / Communications Lab / Application Lab Fulfillment: IMA Major Electives; IMB Major Interactive Media Elective.
The possibility of truth telling has been challenged and debated throughout the history of art and across various genres. What does it mean to tell the truth in art? How does art cross the boundaries between the real and the fake, truthfulness and misrepresentation? This course will examine social engagement in art and the construction of authenticity in creating works based on trust rather than the truth and will investigate a variety of artistic sources and genres including, but not limited to, propaganda, documentary, video art, public art, and installation art. Students are expected to develop projects demonstrating a comprehensive understanding of the subject. Prerequisite: Interactive Lab, Communication Lab, Application Lab or permission of instructor Fulfillment: IMA/IMB elective.
This course explores unruly images, bodies, faces, spaces, aesthetics, and even feelings that exist at the margins of categorization, making these powerful subjects for artistic work. We will manipulate image-making tools and give form to expressions that reveal the hidden structures of power. Through lectures, discussions, workshops, and readings, we will look at topics such as memes and glitch art, cyberface and AR filters, monsters and glitched bodies, uncanny valley and AI images, as well as liminal space and collaborative world-building. We will pay particular attention to selected writings from Hito Steyerl, Legacy Russell, Rosa Menkman, and Adam Greenfield, and apply these critiques to emerging perceptual technologies (volumetric 3D capturing, machine learning, AR/VR, web-based 3D game engines) that students will use to create their projects.
Sound is physical. It can move objects, vibrate surfaces, perceptually alter our emotions, and shape the way viewers engage with spaces. As artists, we are conditioned to aspire to situate our work within traditional settings. How can we reposition sound as a main element of a work within unconventional contexts? This course will use the question of a sonic utopia as a platform to create interdisciplinary projects that exist between installation, sculpture, video, performance, movement, and music. Students will learn sound theories through lectures and in-class workshops and are encouraged to incorporate their individual interests into each of the four main projects that aim to position sound as a primary element of a work in order to expand the possibilities of working with sound.
The current design methods and technologies in the field of digital entertainment, from design concepts to design tools, are still based on 2D screens, but the product is often 3D. This contradiction results in the designers not being able to see the final consequences of their work directly. Can existing VR technologies solve this contradiction? Immersive design is a new design method for digital creativities beyond the screen. It usually takes VR, AR, or MR devices as design interfaces to craft 3D models, textures, game environments, and other gaming experiences. Not only game assets creation but also game design concepts can be changed with these new technical inputs. In this course, students will answer the following questions: ○ Is immersive design more intuitive? ○ Is immersive design more efficient? ○ Do immersive design techniques have lower learning costs? ○ What traditional design lessons and concepts can be applied in an immersive design environment? In view of these questions, students from different disciplines will give answers through discussions, projects, collaboration, documentation, and VR games. Prerequisite: Communications Lab. Fulfillment: IMA/IMB Elective.
The aim of this course is to explore the relationship between the virtual self and environment and to assess both as a space for learning and collaboration using virtual reality. This course takes place entirely in virtual, immersive environments. Students will be provided Oculus Quest 2 virtual reality headsets and specialized software. See the principles above for further details. Prerequisite: IMA Major with junior or senior standing. Fulfillment: IMA/IMB Elective.
Learning with Turtles explores programming languages, systems, and activities designed to help learners in computational environments. Starting from a constructionist principle that systems designed for beginners must be able to embody the most powerful ideas in computing, we master some of those systems, explore how those have been designed, and engage in contemporary debates. The environments we learn with include Turtle Geometry, Craft Computing withTextiles, Modelling, and other interactive projects using programming and modelling systems such as Snap!, TurtleArt, Turtlestitch and NetLogo. Individual and group projects involve students in advancing their computational knowledge and skills and provide opportunities to design for others, to teach, to study learning and expertise, and present projects in community and public forums. The course is fundamentally about ideas, and how some powerful ideas from computation can empower a learner to be a better creator and problem solver. Writing, presentations, and discussions will emphasize reflection on our own learning within the course. Prerequisite: None. Fulfillment: IMA Major Electives; IMB Major Interactive Media Elective.
Auto Fictions is a studio class focusing on the creation of immersive, multi-path and interactive experiences based on personal narrative. Documentary art has included the art of installation for decades, but new technologies have given artists affordable tools that allow them to rapidly prototype and then refine immersive media experiences. Auto Fictions is an interdepartmental course that may include students from Film, ITP and Theater disciplines. Students will create production teams. Each team will make a project and each student will help the other create their work through intensive collaboration. The intention here is to practice collaboration across disciplines, methods, values, and artistic cultures to create a work of immersive fiction based on materials gathered from the past, captured or created in the present and/or imagined for the future. We will look at several possible approaches to the creation of immersive media works. Including multiplexing software programs. Students with the requisite knowledge in these tools or gaming engines with similar capabilities may use them for the creation of their works. But it is not required. There may be an opportunity for building intuitive interactivity into some of the experiences. In response to the Covid-19 pandemic we will be exploring how to adapt our working methods and our work itself to these conditions as they develop. The goal of the course will be to physically realize the works but to do that we will go through a detailed comprehensive design, planning and budgeting process so that if conditions permit all aspects of the work will be fully created or planned and ready to build. If conditions permit, we will meet one day at the beginning of the semester to install and introduce the platform, create teams, and review the assignment. Otherwise we will meet online and begin work there. Students will be provided with access to online tutorials for relevant software and systems.
This is an interactive media art class for people who want to develop creative experiences that are about something specific with a specific point of view! This sounds vague, but your projects won’t be! You’ll be provided with a step-by-step methodology for making research-driven creative and critical projects with a focus on experience design. For the first part of the semester, you’ll make a low-tech guide to a somewhat wacky random topic you’ve been assigned to. For the second part you’ll choose a topic you are passionate about and make an interactive experience that engages deeply with that topic. Skills touched upon: research methods, ideation, critique, experience design, intro-level Unity3d. Class time will consist of: creative exercises; discussion of readings, methodologies, and artworks; student presentations; critique sessions; guest artists and researchers; and a handful of demos. Why critical? In this class, critical means: discerning, eager to participate differently, cast new light on, re-examine, course-correct. Why experience? The work in this class will be looked at through the lens of its ability to change a user, participant, audience, viewer. Interactivity is one way of doing that, but through the lens of experience design, all art is temporal and embodied. A research-based art practice brings together an eccentric mixture of skills, including traditional forms of research (library and interview techniques, informal ethnographies) and experimental hands-on research (hunch-following, experimentation, systems thinking, prototyping, daily practice, user experience design, and user-testing). We’ll try ’em all!
This two-point workshop is centered on the examination and creation of collective storytelling environments. We will examine a wide-range of storytelling spaces including participatory and user-generated environments, site-specific works, community based arts practices, and transmedia storytelling. Weekly assignments, field trips, and student presentations.
If so much of life is circumstance, being in a certain place at a certain moment in time… Can we shape a life or at least a few brief moments of a life by designing the circumstances in which that life inhabits a space? In this course, we will treat space as a time-based medium and ask how interactive spaces can generate narratives that are lived rather than told. We will do so by interrogating four so-called “space-narrative” forms: Wandering The Desert, Processions, Circles and Territories. Through play, discussion and hands-on workshopping of both technical topics and ideas we will ask and attempt to answer some of the following questions: What constitutes a space? How do we experience a space over time? How does space shape our experience of time? How can space form a personal narrative? Media outputs we will employ include: lights, projection and sound. Interaction input sources will come from cameras and microphones. We will use p5.js, websockets and node.js for real-time interaction. Class time will be split between group improvisation exercises, playing with and critiquing examples and translating design strategies into code and logic.
This course aims to provide students with the means to understand immersive media experiences, and conduct experiments from both a practical and a theoretical perspective. The course consists of lectures, research, discussion and studio-based practice. Students will learn to produce stereoscopic – 3D images and photogrammetric 3D models, utilize multi-channel video and sound systems, and be introduced to the Unity game engine and VR hardware. For the final project, students produce VR environments in experimental and meaningful ways. Prerequisite: None. Fulfillment: IMA/IMB elective.
The Emerging Media Studio courses explore methods to creatively think through and hybridize artistic photographic practice with emerging media technologies from medicine, the military, archaeology, urban planning, environmental science and other industries. Projects may take open-ended forms such as video, virtual reality environments, site-based performance, spatial imaging, 3D fabrication and photographic documentation. Critical readings and ideas drawn from artists as well as professionals in other fields are discussed. Our practice is learning how to adapt to and position ourselves as artists making unique contributions to the social dynamics of culture and a constantly shifting universe of media.
Visual Effects and Compositing is a lecture and workshop course that covers the application and practical creation of 2D (greenscreen, color correction, morphing etc.)and integration of existing 3D (CGI, animation, virtual sets etc.) visual effects. The assignments will require students to explore how to create, manipulate and combine 2D and 3D images seamlessly.
Introduction to Visual Effects for Animated and Live Action Films provides an in-depth overview of VFX production from the director-producer perspective. This course will offer a basic understanding of how VFX are utilized and will prepare students for the visual effects industry, and its constantly evolving digital and in-camera solutions. It will also provide an historical overview of the evolution of VFX. Students will also learn how powerful visual effects have been, and will continue to be, in helping to tell stories, especially projects with limited budgets. They will gain a detailed understanding of the type of camera coverage necessary on-set to facilitate the VFX process and what communications, with their primary crew, are necessary during early stages of preproduction to optimize footage for VFX. Areas of study include rig removal, green/blue screen compositing, motion capture, shooting backdrops, matte paintings, background plates, miniatures – models – forced perspectives, match lighting, sky replacement, crowd replication, integration of 2D and 3D CGI elements, motion control and practical effects shot in-camera.
Course description (optional): Not all innovation starts in the West and gets exported to other parts of the world. In many places with less developed capitalist economies and infrastructures, technology is rapidly developed and adapted for hyper-local use. We’ll gain inspiration from a broad spectrum of creative uses of technology in the developing world(s) — from art and design, hacktivism, and community-oriented work that increase social good, and then conceive of and prototype our own projects. Special attention will be paid to speculative 3D fabrication tools and processes, web-based platforms, circuit bending and making them all work together! Students will experiment and research at the intersections of art and technology to develop the beginnings of work grounded in post-colonial reality and the late capitalist near-future.
This course explores sound as an art form and technical practice in its own right. Topics include contemporary techniques in composition, sound art, and interactive installation. Students will produce sound with narrative elements that evoke social, cultural & critical-thinking. Their final projects can be experimental podcasts, music (performance and/or recordings), multi-channel audio installations, or multimedia projects. | Prerequisite: DM-UY 1113 or MPATE-UE 1001
This class builds upon the techniques of the recording studio and the techniques of producing recorded music begun in Engineering the Record I, IIand Producing the Record Side A and B and will explore advanced techniques used in surround and immersive sound recording and mixing. Today, surround and immersive audio can be found in all areas of popular entertainment: music, film, television, streaming, games, etc. By using the multichannel studio facilities of the Clive Davis Institute of Recorded Music, students will further learn to record and mix in surround and immersive audio formats. Assigned work will take place in Studio 1 and Studio 4.
Hollywood in your palm. That is what this combination of lectures, screenings, demonstrations and practical production workshop will offer to the students in this course. There will be several professional guests making presentations and Q&A sessions from the mobile phone filmmaking industry. In addition to the historical and critical overview of the emergence and exponential growth of global cell phone cinema, students will shoot all footage on cell phones and download them for computerized editing. The final project will be under three minute shorts. Projects will include all genres of film and television: news, mini-documentaries, animation, music videos and narrative shorts. Completed student projects will be suitable to be posted on the Internet and entered into domestic and international mobile phone film festivals. For example, two minute long improvisations of Bollywood Style Music Videos shot on Cell Phones by the students have been projected at the Tribeca Cinemas as part of the New York Indian Film Festival. It is suggested but not compulsory that students bring to the class a cell phone capable of recording video.
2D Art and Animation for Games is a 1-semester, 4-credit class that builds fundamental skills around the design and production of art assets for games. Through a series of individual design assignments, critiques, and exercises, students will explore concepts like art direction, color theory, animation principles, and UI design while building a working knowledge of prominent industry tools.
In this course, we will explore how to create narratives that leverage our lesser used senses like touch, taste and smell as well as lesser-known ones like space, time, balance and scale. We will dig into the history of experiential storytelling, starting from immersive theater and Smell-O-vision to cutting-edge haptics and mind-bending illusions of proprioception. To help center this back in practical applications, we will also explore how this evolving art is commonly used in exhibition design, experiential marketing and brick and mortar retail. The class will be a healthy mixture of game theory as well as experienced based learning (meaning there will be a couple field trips and multisensory VR projects to explore). A basic knowledge of game engines is ideal but not mandatory because we will be using predesigned templates in Unreal engine to be experienced and manipulated in real-time through virtual reality hardware.
To be a VR creator, it’s not enough to learn the hard skills—it’s also our responsibility to prime ourselves for the human impact of our work. As a means to design VR that is both enjoyable and accountable, this class proposes we borrow design principles from Hedonomics, a branch of ergonomic science that facilitates pleasurable human-technology interaction. Through the Hedonomic Pyramid, we’re able to section our thinking off into regions (Safety, Function, Usability, Pleasure and Individuation) and map out industry-tested VR design guidance for each. The result is a hierarchical checklist of proven principles, specifications and practices—that promote a culture of inclusive and holistic design—built to serve as a quickstart guide to designing accountable VR interfaces and systems. This class, divided into units that represent each level of the Hedonomic pyramid, will unpack both technical and conceptual strategies for creating VR, from visual interface fidelity to avoiding locomotion cybersickness to designing safer social VR spaces.
Throughout history, musicians have channeled their creativity into outrageous fashion statements and invented personas: think MF DOOM, Sun Ra, Ghostface Killah, Daft Punk, Leikeli47 and Rammellzee. By embracing their alter egos in extreme and outlandish ways, artists have found their authentic creative voices. This course will introduce participants to the art of masquerade using their resourcefulness to create costumes from found materials, and performance as an exploration in creative expression using new media and technology. Students will be introduced to ideas surrounding abstract storytelling, experimental audio video production, and A/V performance using a combination of technical and hands-on approaches.
This course will serve as an incubator to imagine a speculative product advertisement in the year 2030. In films like Blade Runner, or Her adverts fill the world and become an important aspect of exposition for the film. And in the real world, works such as Alisha Wormlsey, Alexandra Bell, and Hank Willis Thomas begin to re-imagine advertisements as an art practice in society today. Our work will begin to speculate on near-future objects in which topics such as communication, energy storage, transportation can begin to be re-imagined in the next industrial revolution. Using 3D tools, students will gain experience in speculative design thinking, industrial design modeling, product lighting, and custom post-production methods. The final project will be a product advert that will be designed to promote a speculative design entirely made from 100% Biodegradable plastics. The course will look at the ready-made objects all around us as a launching pad. We will be starting with modeling an object in detail. Using Moi 3D, Maya, Render Engine TBD, After Effects, and premiere over the course of the semester. I will go through some of the latest tools within the VFX industry and support this course with a series of artists who have re-imaged the role of cultural production. The final will be an advertisement poster and animation.
If social change begins in the imagination, how then can creators better envision and render the more just and beautiful worlds we want to make? Storytelling has the power to be an alchemical force for revolutionary change. Together, we seek to interrogate and apply interactive storytelling as a technology we can deploy in service of our collective liberation. In this course, we pair a study of story as liberatory praxis with a hands-on grounding in emerging tech tools that allow viewers/players to take an active role. Interactive storytelling technology in video, audio, and text powerfully situates viewers inside constructed narrative worlds. Creators in these emerging media gain the capacity to design choices and respond accordingly, propelling imagination toward agency and enhancing empathic connections between viewers/players and characters. What will it mean to use these tools to tell deeper stories that ask urgent questions about how we want to live in the world? “Part of being a revolutionary is creating a vision that is more humane. That is more fun, too. That is more loving. It’s really working to create something beautiful.” —Assata Shakur
Project-based development studio incorporating dramaturgy techniques, user/audience planning, and social/contextual awareness. You bring in a project. We explore how to make it more engaging through paying close attention to medium, context, and details. Students bring existing project ideas and we investigate various methods and ways to make and direct experience within the infinite combinations of contexts of the present moment together. This particular studio is just as appropriate for projects in the areas of interactive art, programming, physical computing, XR as it is in the areas of performance, sculpture, and sound walks (everything). Everything you make is time-based the moment a user interacts with it (even a painting). All time-based work can be thought of in terms of how a user is led (or not led) through it. I call this directing. We will apply various techniques of story-telling and world-building equally to seemingly “non-narrative” projects as we do to traditional-narratively structured projects. No matter what you are working on, I believe that you are a maker / director of experience. You are making / directing with intent (whether you know it or not). Your user (audience, tester, public, patron) brings with them the entirety of their life’s experience. Your intent cannot possibly meet every user’s lived experience. It is your job as the maker / director to draw a circle that encompasses both. This is the studio’s lens. Students will be directed to make using placeholders instead of waiting for perfection to manifest. Step one will tell you what step two is. We will pay rigorous attention to detail, while holding close to the notion that art-making thrives in adaptability. We will critique using various, structured, co-facilitated methods. Let’s pay close attention. Let’s learn through failure. Let’s listen to what we are making. And let’s listen to each other. Here we go.
Live streaming is so seamlessly embedded into our online experience. We lay in bed, on our phones watching hearts flicker across the screen as the person we’re watching greets all of the competing messages in the chat, asking for birthday shout outs and follow-backs. While the ability to live stream feels more accessible than ever, it feels very tied to corporate structures, branding and self promotion. How can we push the concept of a live stream in a new direction and rethink what a live stream can be?
So much of life is circumstance, being in the right place at the right moment in time. Can we shape a life or at least a few brief moments of one by designing a space for it to inhabit? In this course, we will treat space as a time-based medium and ask how experiential spaces can generate narratives that are lived rather than told. We will look to a broad range of storytelling traditions to interrogate four so-called “space-narrative” forms: Wandering The Desert, Processions, Circles and Territories. Through play, discussion and technical and conceptual workshops, we will ask and attempt to answer some of the following questions: What constitutes a space? How do we experience a space over time? How does space shape our experience of time? How can space shape both a personal and collective narrative? We will build spaces with lighting, projection, sound and physical objects. Class time will be split between group improvisation exercises, playing-testing and critiquing projects. The class will culminate in a showing of work at the end of the semester.
The course focus is on the design and creation of digital musical instruments. Music in performance is the primary subject of this class. We approach questions such as “What is performance?” “What makes a musical interface intuitive and emotionally immediate?” and “How do we create meaningful correlations between performance gestures and their musical consequences?” Over the semester, we look at many examples of current work by creators of musical interfaces, and discuss a wide range of issues facing technology-enabled performance – such as novice versus virtuoso performers, discrete versus continuous data control, the importance of haptic responsiveness as well as the relationship between musical performance and visual display. Extensive readings and case studies provide background for class discussions on the theory and practice of designing gestural controllers for musical performance. Students design and prototype a musical instrument – a complete system encompassing musical controller, algorithm for mapping input to sound, and the sound output itself. A technical framework for prototyping performance controllers is made available. Students focus on musical composition and improvisation techniques as they prepare their prototypes for live performance. The class culminates in a musical performance where students (or invited musicians) will demonstrate their instruments. Prerequisites: H79.2233 (Introduction to Computational Media) and H79.2301 (Physical Computing). Prerequisite: ICM / ICM: Media (ITPG-GT 2233 / ITPG-GT 2048) & Intro to Phys. Comp. (ITPG-GT 2301)
This practical, hands-on course will explore questions such as: How can we design engaging, creative learning experiences that are relevant to the cultural goals and needs of today’s youth in China, while laying the foundation for creative learning for the workforce of tomorrow? What are engaging, effective creative learning resources, and how are they best implemented in Chinese learning settings? How can we take advantage of young people’s near ubiquitous love of the arts to facilitate creative learning?’ In this course, students will work in teams to design digital learning resources and experience designs at the intersection of music, coding, arts, and technology. The course will begin with an introduction to emerging trends in learner engagement and design-based research, especially related to web- and mobile-based musical experiences and principles of making music with new media. Innovations in and applications of musical interaction, interactive technologies, user-centered design & engagement, scaffolded learning, creative learning, pedagogies of play and making, and educational entrepreneurship will also be explored. Students will work together in teams and paired with a partner audience of learners and teachers in Shanghai drawn from local and regional international schools, ed-tech startups, and cultural partners. Together they will assess the needs and opportunities of partner students and teachers, and engage in a two-stage iterative, reflective co-design process prototyping custom learning resources and experience designs with their partner end users. At the end of the course, students will present and demo their learning resources as part of a public showcase to an external audience of partners, educators, technologists, musicians, entrepreneurs, and experience designers in Shanghai. Prerequisites: None. Fulfillment: IMA Major Electives; IMB Major Business Elective/Interactive Media Elective; Business and Finance Major Non-Finance Electives; Business and Marketing Major Non-Marketing Electives.
Real-Time Media is a 4-credit class using MaxMSPJitter to survey how real-time and reactive media can be used for art installation and performance. Classes will be a mix of coding labs, surveys and lectures on historical examples of the medium, guest artists talking about their practice, class field trips, and in-class performances and critique. While the primary focus will be on video and sound there will also be attention given to sensors, electronics, web APIs, and more. The class has coding assignments building to solo video and audio performances and final group installation project.
Fairy tales, myths, and stories of magic have always served as a way for both children and adults to make sense of the unpredictabilities of the world around them. How do these stories serve us today? How do new technologies allow us to reinterpret them so that they have new meaning for our times? Through readings, weekly exercises, and a final project, students in this course will explore the historic role and structure of fairy tales as well as the potential contemporary frameworks that allow us to entertain the impossible. Students will work with stories of their choosing however we will examine their implementation through traditional material and book art techniques, as well as projection mapping, 3D and VR (using Unreal Engine.)
This course examines both the practices and the products of adapting narratives from one medium to another. Through case studies of specific adaptations, we will address some of the major formal, industrial, and interpretative questions that transmedia adaptation raises, as creators change characters, stories, settings, and narrative tropes to fit into new stories various, often multiple media: comics, radio, novels, movies, television, games (tabletop and electronic), and more. Theoretical readings will give students concepts and a vocabulary to discuss ways that narrative adaptations use and re-purpose their “source” texts. Students will write prompted response papers, an analytical essay, and an annotated bibliography; in collaboration with classmates, student teams will first propose and then develop transmedia narratives of their own. Prerequisite: Writing as Inquiry Fulfillment: IMA/IMB elective.
Science fiction, in literature, graphic novels, motion pictures, video games, and other mass media, has been a valuable form of sociocultural expression for generations. We use it to express our concerns for humanity, as well as our relationship to technology, one another, and our forms of governance. In the first half of this course we will learn about the field of future studies as we examine a body of works across various media which explore a popular theme in science fiction, such as time travel, space exploration, robots, dystopia and utopia, or pandemic. In the second half of this course we will produce a collective work of transmedia science fiction that reflects on the past, address concerns about the present day, and forecasts possible future scenarios. The theme of this semester will be pandemic. Prereq: None. Fulfillment: IMA/IMB elective.
This course will focus on designing, creating, and performing with self-built electro-acoustic music systems to explore the limits of human musical expression. Over the semester, students are asked to research examples of contemporary work by creators of musical interfaces and discuss a wide range of issues facing technology in the performing arts. Readings and case studies will provide background for class discussions on the theory and practice of designing gestural controllers for musical performance. Students will invent and prototype a complete system encompassing musical control, mapping input to sound, and the creation of sound itself. Interaction Lab is a prerequisite, however, prior performing experience is not required. The performance discipline is inherently interdisciplinary and collaborative, so an open mind to working with others is imperative. The class will culminate in a performance where students will play their instruments live as well as a formal presentation of the students’ works at the NIME2021 conference hosted by NYU Shanghai. Prerequisite: Interaction Lab Fulfillment: IMA/IMB elective.
This course focuses on electronic music synthesizer techniques. Concepts in the synthesis of music, including generation of sound, voltage control, and treatment of sound and tape techniques. Included is a short synopsis of the history and literature of analog electronic music. Students complete laboratory tasks and compositions on vintage synthesizer modules and create one or more final projects that demonstrate(s) the application of these concepts.
Introduction for teachers, composers, and performers to explore potentials of computer music synthesis. Basic concepts of music synthesis presented through the use of a microcomputer, keyboard, and appropriate software. System may be used as a real-time performance instrument or as a studio composition instrument. Educators may explore potentials for classroom application.
Analytical and theoretical concepts required grasping the aesthetic development of electronic and computer music compositions. The course emphasizes analysis and historical understanding of techniques of production and compositional ideas.
Digital recording technology & production techniques are explained & demonstrated. Lecture topics engage analog to digital conversion, digital to analog conversion, digital signal theory & filter design, digital audio effects & mixing. Studio lab assignments are performed outside of class reinforcing weekly lecture topics.
Through weekly performing, readings and discussions, students study the conceptualization and production of live electronics ensemble performance. Students create new works, improvisations, and rehearse compositions from a growing repertoire of electronic ensemble compositions, to be performed at a semester’s end public concert.
This course is about experimental game design. Design in this context pertains to every aspect of the game, and these can be broadly characterized as the game system, control, visuals, audio, and resulting theme. We will explore these aspects through the creation of a few very focused game prototypes using a variety of contemporary game engines and frameworks, high-level programming languages, and physical materials. This will allow us to obtain a better understanding of what makes games appealing, and how game mechanics, systems, and a variety of player experiences can be designed and iteratively improved by means of rapid prototyping and play-testing. The course combines the technology, design, and philosophy in support of game creation, as well as the real-world implementation and design challenges faced by practicing game designers. Students will learn design guidelines and principles by which games can be conceived, prototyped, and fully developed within a one-semester course, and will create a game from start to finish. The course is a lot of (team)work, but it’s also a lot of fun. Programming skills are helpful, but not a hard requirement. Artistic skills, or a willingness to learn them are a plus. | Prerequisite: (Graduate Standing AND CS-GY 6533) for SoE students OR (OART-UT 1600 and OART-UT 1605) for Game Center MFA students OR instructor permission.
This course will combine a history of video art and experimental film with practical training in the use of live video performance art technology. Students will explore new ways to create and edit films and videos using VJ software, projections, and multi-channel video surfaces. Workshops will demonstrate concepts and software that can be integrated into the creative process of video performance art and video art installations. COURSE OBJECTIVES At the completion of this course, the student will be able to: Draw inspiration from the recent history of incredible video and multi-media artists. Develop an understanding of audio and visual hardware used by VJ’s. Use live VJ software to manipulate digital media in real time to create Video Performance Art. Use Projection Mapping techniques to project video art onto 3D surfaces. Create original video performance art, video installations, and other performance pieces. Utilize skills to make video art in the professional market.
Introduction to Virtual Production is a class composed of lectures, discussions, screenings, exercises, group critiques and presentations. The course is designed to expose students to the fundamental principles of storytelling through Virtual Production including writing, directing, cinematography, performance, editing, art direction, and technical direction. The course will explore emerging techniques utilizing software and technology. How do you tell a story in this new form of collaboration? How can students apply what is learned to their own creative work? History and theory of Virtual Production will be studied and used to inspire personal and creative work in order to better understand how story through Virtual Production can successfully be expressed and most effectively reach its audience.
We have all played and enjoyed games, but how do people actually design and develop them? How to describe a game from a professional standpoint? What are the basic elements and structure in video game development? How do game designers create an interactive experience for the player? What about prototyping and iterating in development? This course explores these questions and others through playing, analyzing and making games over 14 weeks. Students will understand game not only as an entertaining production and business model but a form of interactive media impacting current life and future. Students will be introduced to game design concepts, emphasizing the development: paper and digital prototyping, develop iteration, interactive narratives design and embedment, object-oriented programming, 2D/3D game art design, sound effects composition and user testing. For the course project, students will work in teams and create games in multiple projects, from board game focusing on gameplay prototype to digital playable experience with creative game art designs. This course leverages Unity, a game engine that uses C# based programming language. Basic knowledge of any programming language will come in handy. Prerequisite: Creative Coding Lab, Interaction Lab or equivalent programming experience. Fulfillment: IMA Major Electives; IMB Major Interactive Media Elective.
From the history of visual music and abstract film to the contemporary notion of live cinema, this course will be an exploration of the synesthetic relationship between sound and visuals in a real-time performance setting. Dating back as far as the 18th century, systems have been invented to produce images alongside music linking the two through formalized arrangements. Current media technologies make developing such systems both more approachable and more expansive in their scope. Through readings, viewings, and case studies students will gain an understanding of the history and theory of live audiovisuals. During the course students will team up to develop and master a real-time audiovisual system of their own invention. The class will culminate in a show in which they will present their work through a live performance.
With the rise of mobile computing platforms such as smart phones and tablets, location has suddenly become a key element in the production and consumption of media. In this online course, designed for NYU Shanghai Interactive Media Arts majors studying abroad, students will be encouraged to simultaneously explore their unique study away site, as well as to consume, research, critique, and create location-based media for mobile devices. Students will be introduced to GPS (Global Positioning System) technologies through activities such as geocaching and GPS drawing. We will next investigate geocoding, geotagging, and geofencing through the application of JavaScript mapping platforms CartoDB and Google Maps. Students will then explore an emerging technology known as Bluetooth Beacons, which can be used to create custom positioning systems and to facilitate location awareness in mobile devices. Students will be asked to then produce, as a final project, a game that engages participants in a location or locations, as well as in locative media in any number of forms. Note: This is an online course featuring both synchronous and asynchronous learning opportunities. Registration is limited to IMA Majors studying at NYU’s global sites other than New York or Abu Dhabi.
The class will teach how to architect and lead a virtual production by creating a dialogue between the Producer, Director, and Cinematographer in filmmaking with the Technical Producer and Director in creative technology. The class will cover an overview of all of the technical skills required to produce a remote virtual production through the lens of a project manager making administrative and creative decisions. This class will culminate in a real-time 3D project exploring motion capture and virtual production that will adapt a pre-existing cinematic work with the class themes in mind.
Today’s internet, made up of mostly text documents and two-dimensional images and videos, is the result of historical limitations in bandwidth, graphics processing and input devices. These limitations have made the internet a place where the mind goes, but the body cannot follow. Recent advances in motion capture devices, graphics processing, machine learning, bandwidth and browsers, however, are paving the way for the body to find its place online. This course will explore embodied interactions in the browser and across networks. Specifically, we’ll explore TensorFlow.js models like PoseNet and BodyPix, and Microsoft Kinect in p5.js and Three.js. Assignments will consider designing engaging embodied experiences for individual and social interactions online. Experience with Node, HTML and JavaScript is helpful but not required. ICM level programming experience is required. The course will have weekly assignments that explore embodied interaction online. Assignments will begin with exploring single points of interaction (i.e. one mouse or one joint), and progress to considering full bodies and multiple bodies in one browser. Students will have a 2-3-week final project with which they will delve more deeply into the subject matter in one piece of work. Students will have readings/watchings focused on embodied and networked user experience. Some influential works that will likely be assigned/discussed are Laurie Anderson’s “Habeas Corpus,” Todd Rose’s “The End of Average,” and Myron Krueger’s “Artificial Reality.” The course examples will be taught in Javascript using web technologies/frameworks. However, students are welcome to work in their preferred medium.
Juxtaposed to traditional comics, Experiential Comics combines emergent tech, unconventional comic book art/structure, and game engines to offer users a more immersive, continuous storyworld experience. Challenging the status quo of classic and contemporary digital comics, students will explore new technologies/world-building techniques better suited to craft innovative comic book narratives and formats –worthy of the Fourth Industrial Revolution. Students will ingest a brief history of classic and digital comics formats, collaborate with comic book artists to design engrossing characters, engage in world-building sessions, play with Unity/Unreal engines to generate avatars/ virtual environments, work with actors in motion capture/volumetric capture studios, learn the latest iteration of the Experiential Comics format, and share their unique expressions of Experiential Comics in a final presentation. Throughout a 7-week period, the course will be divided into 7 themes 1) The Disconnection of Digital Comics 2) Classic and Unconventional Comics Continuity 3) Marvel vs DC vs Insert Your Universe Here 4) Fourth Industrial Revolution Technologies 5) Capture & Creation 6) Infinite Engagement and Unlocking Immersive Format 7) Experiential Comics Presentations. Each weekly class will be divided into two halves 1) Exploration of Theme/Discussion 2) Process, Practices, & Play. This course requires CL: Hypercinema or equivalent experience.
This course is designed to provide an overview of visual storytelling in the newsroom. We will explore a variety of narrative formats and design principles, learn about reporting techniques for visual stories, touch on the best practices and ethics of journalism and work on collaborative exercises and assignments.
Masks have been used around the world since antiquity for ceremonial and practical purposes, as devices for protection, disguise, entertainment and bodily transformation, made to be worn or displayed. Sociologist Erving Goffman wrote about the everyday life as a masked theatrical performance. The performative aspect of our lives today is ever so present in our use of social media, where we present a curated version ourselves for the immediate visual consumption of others. In our `Selfies`, we can assume a multitude of identities and characters. Recent tools and platforms have evolved social media portraiture to an art form and have created new opportunities for artists to create and distribute interactive augmentations, forming new relationships between artists and viewers. This class explores the developing language of social media portraiture enhanced by Augmented Reality. Students will: – review masks in art history, leading up to today – ideate, design and develop an interactive mask (AKA effects/lenses/filters) – learn to use the Meta Spark software to create AR effects. This course requires CL: Hypercinema or equivalent experience.
How can light, sound and design transform the human experience within a given space? How can psycho-geography be manipulated through audio-visual techniques? In what ways have and will technology allow spaces for sonic entertainment to be more immersive and experimental? Through an exploration of audio-visual techniques (i.e. VJing, MIDI-ing devices, sound synthesis, projection mapping, experiments with spatial sonic composition) along with discussions on how counterculture movements have used music and design as a vehicle for political dissent and community building, students will be invited to imagine new club spaces for social contexts beyond pure aesthetics. Assignments will include the development of different forms of interactive spaces for expression. Ableton (and free DAWs), MaxMSP, Isadora, and Unity will be used within this course.
Time-based art, performance – and theater most specifically – should be perfect manipulators of experience. Many creators of time-based art look for the “universal” in content and overlook what we all have in common in form: brains. What happens in the minds of all truly happens (what happens in the lobby also truly happens). How can we use art to make our brains experience the same things? What behind-the-scenes work can we employ to manipulate experience. Film scoring works on us in ways we don’t perceive in the moment. Can we pay closer attention to this when making work? Sometimes what we have for lunch effects our experience of art more than the hours of labor that went into the most minuscule of decisions when making that art. Is empathy always ethical? Show don’t tell. How does the art take place in the room in front of us? How can public performance not be awkward? How do we experience a performance rather than watch it.
he virtual expansion of screens began during the 1960’s with the exploration of head-mounted displays. Since the 60’s, virtual reality has been explored in a multi-disciplinary context including philosophy, design, arts, behavioral therapy. Baudrillard, with his publication of Simulacra and Simulation (1981), declared that human experience is being replaced by a simulation of reality (HyperReality). His theories brought the dystopian narrative of the virtual to mainstream pop-culture, as seen in films such as The Lawnmower Man and The Matrix . Contrary to Baudrillard, Canadian VR Pioneer Char Davies brings a more positive perspective to Virtual Reality, “facilitating a temporary release from our haitial perceptions and culturally biased assumptions about being in the world, to enable us, however momentarily, to perceive ourselves and the world us freshly.” Throughout the class, the friction between Baudrillard and Davies will create the foundation of our exploration of Virtual Reality, where we will use room scale headsets and game engines to create meaningful “temporal experiences” exploring themes from behavioral sciences to narrative storytelling. We will be exploring ● existing VR projects, popular culture references and theory. ● concepts such as sense of embodiment (SoE), social VR design, and interactive storytelling techniques. ● methods for designing, modeling and rigging avatars for VR. ● live and pre-recorded animation. ● spatial audio techniques such as ambisonic sounds engines. ● packaging and distributing applications for social VR. This is a production class, along with a theoretical foundation, in which we will prototype projects with networking, inverse kinematics, raycasting and face tracking technologies to explore questions such as “how does the viewer become part of the experience?” and “how does the real space relate to the virtual worlds we design?” In the second half of the class, students will work in groups to build a final social VR project based on their exploration of the above framework.
As game and interaction designers we create systems and choices that can either prey upon our psychological foibles or help us avoid decision pitfalls. It is our responsibility to understand how we decide, to consider the ethics of the systems we create and to practice designing systems in a purposeful manner. Game Design & The Psychology of Choice will provide interaction and game designers with an understanding of the factors that influence behavior and decision-making by looking at the intertwining of cognitive psychology and economics through the development of behavioral economics. These disciplines study behavior on the individual and group level, often revealing some of the why behind the rules of thumb and folk wisdom that game designers come to intuitively. But understanding the why—why we fall into decision traps; why certain tradeoffs tax our brain more than others; why we are overconfident about our abilities; why certain decisions make us uncomfortable—allows us to more purposefully apply our design craft, both in and out of games. Finally, as a class, we will take what we learn about how we think and create series of game experiences based around key cognitive science concepts. Assignments may include: •Mod a cognitive science experiment into a game or experience •Analyze and present a game through the lens of cognitive science and behavioral economics •Create game or experience based around a particular insight from cognitive science or behavioral economics
“Lies. Hoaxes. Conspiracies. Rumors. Propaganda. Fake news is an age-old phenomenon—but the internet is making targeted misinformation cheap and scalable. That is affecting politics, public opinion, and the everyday experience of the internet. In this 6-week class, we will explore the cutting edge of “fake news” by engaging in ethical research and fabrication. Participants will manufacture and observe a controlled “fake news” event. We will experiment with command-line tools for doctoring video, neural nets and deepfakes to fabricate reality, Twitter bots, behavioral psychology, and the dark underbelly of the ad economy.”
Whether it’s through photo realistic scans found in current-gen video games or the cartoonish and low-fi aesthetic of Bitmoji there is no limit to ways in which the body and the self are represented in digital spaces. This 2 credit class will look at how avatars have been historically used in the realm of art, commerce, and entertainment and utilize existing avatar creation tools to develop projects that examine identity, body politics, and contemporary performance. In class we will cover the basics of Unreal Engine, photogrammetry, 3D scanning, and model rigging although students will be encouraged to use existing skill sets and creative thinking to complete some of the smaller week-by-week assignments. The class will culminate with a short performance, small installation or single/multi-channel video piece using one or more of the techniques covered in class. This can be a solo project or a group project. In this class students will: – Explore how avatars can be utilized in your creative practice – Gain an introductory understanding of Unreal Engine, photogrammetry, model rigging, and 3D scanning. – Learn how to recontextualize digital spaces for the purposes of art, installation, and performance. – Broaden your thinking of what performance can be, both in a physical setting and digital setting. – Think critically about how physical bodies inhabit digital spaces and how the hardware and software we use reinforces the acceptance and value of certain kinds of bodies.
Until recently 3D sound was a novelty reserved for special uses and reaching a limited audience, no medium in popular culture has been as inherently dependent upon spatial audio as virtual reality. The widespread and standardized implementation of surround sound in film brought cinema to a new level of immersion, but is limited to theatrical exhibition and home theater systems. Today a considerable amount of content is consumed on mobile devices and laptops which excludes the cinematic experience of spatial sound. With the current rise of cinematic VR and the blurring line between gaming and experiential VR, spatial audio is no longer just an added bonus, but rather a necessity in designing immersive VR experiences. In this course we will explore the emerging field of 3D sound design and for both 360 video and game engine-built VR using a digital audio workstation, Unity, and 3D audio plugins.
This course investigates aesthetic and technical aspects of sound for video games and interactive 3-D environments. Artistic implications of the technology are also explored from the perspective of the electronic composer and performer. Students will work with a game engine to create an immersive interactive environment. Additional topics include: Csound, Java and other relevant technologies. Completion of a final project, class presentation, as well as several weekly assignments is required.
Many of us have played and enjoyed games. Have you ever wondered how people actually design and develop them? Can a games as a profession and practice be described? What are the basic theories, structures and skills involved in game development? How do game designers create compelling interactive experiences for the player? How do they respond to feedback, prototype and improve these experiences? This course explores these questions and others through playing, analyzing and making games over 14 weeks. Students will understand games not only as entertainment, production and business models, but as a form of mass interactive media and culture. Students will be introduced to game design concepts, emphasizing all stages of game development: paper and digital prototyping, iteration, interactive narrative design, object-oriented game programming, 3D/2D game art creation, sound effects composition and user testing. For the course project, students will work in teams and create games in multiple projects, from simple board games to a digital game with original game art, mechanics and other design elements. This course leverages multiple tools for game prototyping and design, but will focus on Unity, a game engine that uses C# based programming language, for game build-up.
We will explore a range of programming languages, systems, and activities designed to help learners acquire computational skills and become creative problem solvers and project designers, including arts and interactive projects. We will create projects in turtle geometry, animation, and programmable embroidery (Snap!, Turtle Geometry, TurtleArt, and TurtleStitch), and in simulation systems which model complex systems in the life and social sciences in order to acquire a deeper understanding of their underlying phenomena (NetLogo). The course is fundamentally about ideas, and how some powerful ideas from computation can empower a learner to be a better creator and problem solver, acquire a deeper understanding of social and scientific phenomena, and become a self-directed learner. We will identify these ideas and actively engage with the pedagogical theories that underlie embodying them by creating with systems designed for children, beginners, or people coming from disciplines which traditionally had less emphasis on computing-based tools. We will emphasize reflection on our own learning within the course.”
In this course, students will work in teams to design digital learning resources and experience designs at the intersection of music, coding, arts, and technology. The course will begin with an introduction to emerging trends in learner engagement and design-based research, especially related to web- and mobile-based musical experiences and principles of making music with new media. Innovations in and applications of musical creativity, interactive technologies, user-centered design & engagement, scaffolded learning, creative learning, pedagogies of play and making, and educational entrepreneurship in Chinese contexts will also be explored. The market for creative educational experiences in afterschool settings for youth in China is exploding. For-profit educational service companies are competing and searching for differentiated, learning experiences in music, coding, and creative project based learning that will attract high-paying parents looking for the best supplemental education for their children. This practical, hands-on course will explore questions such as: How can we design engaging, creative learning experiences that are relevant to the cultural goals and needs of today’s youth in China, while laying the foundation for creative learning for the workforce of tomorrow? What are engaging, effective creative learning resources, and how are they best implemented in Chinese learning settings? How can we take advantage of young people’s near ubiquitous love of music and technology to facilitate creative learning? Students will work together in teams and paired with a partner audience of learners and teachers in Shanghai drawn from local and regional international schools (e.g. Alibaba’s Cloud Valley), local afterschool programs (e.g., Music Lab), and cultural partners (e.g., Shanghai Symphony). Together they will assess the needs and opportunities of partner students and teachers, and engage in a two-stage iterative and reflective co-design process prototyping custom learning resources and experience designs with their partner end users. At the end of the course, students will present and demo their learning resources as part of a public showcase to an external audience of partners, educators, technologists, musicians, entrepreneurs, and experience designers in Shanghai. Prerequisites: None.
This course emphasizes on the 3D animation through digital modeling / sculpting techniques, keyframe and blend-shape animation . The course breaks down into 4 stages : 1. basic topology of head model, 2. high-poly sculpting and projection texturing, 3. Keyframe and blend-shapes animation, 4. 3D animation final project. In the final project, students get to choose either lip-sync animation or conceptual piece utilizing the created head models. An overview of digital editing / compositing and sound design will also be introduced to assist with students’ final project at the end of the semester. Prerequisite: None. Fulfillment: IMA/IMB elective.
Sound is all around us. The way we perceive or experience these sounds are largely dependent upon their environments, whether artificially constructed or naturally present. In this studio-based course, students will learn about the development of sound art through readings and listenings by artists, musicians and designers who investigate our sonic environment through sound sculptures, multi-channel immersive installations, soundscapes, audio tours, podcasts and field recordings. The course will begin with an introduction into the physics of sound with time for deep listening exercises. We will read selected texts and listen to pieces by those working in the field of Acoustic Ecology, an interdisciplinary field that employs ethnographic practices to create sound studies or art. We will look at artists who employ narrative techniques to engage the audience. We will study musicians such as Alvin Lucier and John Cage and the history of experimental music that takes into consideration the physical space its recorded or played in. There will be weekly exercises that will help develop the student’s spatial awareness of sound and music. We will take listening and recording trips into the field to understand the acoustic urban environment. We will use different types of microphones such as hydrophones and binaural mics. Students will learn how to build their own contact microphones. Students will have the opportunity to create works for multi-channel speakers. The final project can take on any form within the realm of sound art–multimedia, narrative, non-narrative, music, installation.
This course will cover basic camera use as well as the more sophisticated skills of image editing. Developing sensitivity to the aesthetics of image making through the use of the camera’s technical controls and composition are the central goals of the class. The course will provide a background in the history, theory, and contemporary issues of photography through lectures and visits to museums and galleries. By the end of the semester students will have the know-how to make images that convey their aesthetic and conceptual ideas effectively.
This unique course introduces students to innovative and cutting-edge technologies sound, video, and interfaces that are changing the way music is performed, produced and received. Music and creative technologies have shifted in the last years from preset-focused black-box devices to open and hackable hard- and software. Examples are MaxMSP (Ableton), the Kinect Motion sensor, VR Platforms or open source music instruments like Korg’s Mono series, little bits or bastl. This shift enables artists today to understand the inner workings of instruments better and engage a very different working process: these days, devices can more easily be created and manipulated, forming future tools and creating a rich variety of different media. The course consists of both a theoretical and a hands-on part, and has a workshop component. No special knowledge like programming or electronic skills is presupposed. As this course is intended for students from different disciplines, the content will flexibly be adapted to the level of knowledge of the students, especially for students with little or no technical background.
Can you listen to Michael Jackson’s “Thriller” without envisioning the zombie transformation? What about Beyonce’s “Single Ladies” without seeing the accompanying choreography? Both of those songs, along with countless others, have benefited from the groundbreaking visuals that have accompanied them. From the Classic Rock films of the 1960’s to the MTV revolution of the 1980’s and 1990’s to the innovations of YouTube and Virtual Reality, this class will examine how the convergence of visual and auditory mediums has created some of the most impactful art. We’ll extract the great lessons from the pieces we study and utilize our production skills to create videos, on-stage visuals, and songs of our own. We’ll also investigate how the creation of videos alongside songs has disrupted the marketing and sales fates for the music industry multiple times. The weekly class structure will alternate between one 90min lecture/discussion course and one 90min production course where we will be collaborating on creating new content for each assignment together.
Prerequisite: Photography & Imaging II, or permission of the department. This course considers the creative possibilities of a variety of documentary strategies. The editing of images, their structuring into an essay form, the interpretation of their various meanings, and the impact of the documentary essay on the world are all discussed. Students are assigned a range of problems that explore visual description and interpretation ranging from the photojournalistic to the autobiographical. In addition each student devotes a significant amount of time to producing a single-subject documentary project. Classes are lecture-demonstration with critiques of student work and regular presentations of documentary photographs made throughout history, in different cultures and for different reasons, including the personal and the societal. Each student must have a still camera or a video camera. This course requires a nonrefundable lab fee.
With platforms like Steam and Itch.io making independent games more accessible to the public, we’re starting to see a movement toward physical installations of indie games as well. The New Arcade pays tribute to arcade cabinet designs of the 80’s and 90’s, but infuses them with new interfaces and digitally fabricated components. In this class, students will learn how to use the Unity game engine to design a simple arcade game. They’ll learn about aspects that separate an arcade game from other types of games, and interface their game with different kinds of hardware using microcontrollers. In the second half of the class, students will use Fusion360 to construct a new arcade experience using digital fabrication tools like laser cutters, and CNC machines. The class will culminate in a physical installation that showcases their game in a public gallery. Prerequisites: Physical Computing About Mark Kleback: https://wonderville.nyc
This advanced experiential production course will introduce students to the latest techniques for stereoscopic 360 video, manual 360 video stitching and depth map creation, camera-paired Depthkit volumetric video, and mixed-format photogrammetry. Techniques introduced in the class will presume some working knowledge of more basic forms of these capture methods as well as how to integrate them into Unity. Alongside an intense technical focus, the course will also deconstruct recent groundbreaking experiential works that utilize similar experimental production designs. The format of each class will combine lectures and workshops with the ultimate goal of introducing students to the expectations demanded by professional productions. All of the techniques introduced are being employed by top experiential creators and creative studios. The course will try to show viable paths for students to engage with immersive media pipelines, at an expert level, with an expanded sense of possibility and inspiration. Topics heavily examined throughout the course include: surveillance, race, socioeconomics, carceral capitalism and technoscience, algorithmic bias and oppression, and post-work societal structures.
There is a substantial gap between what AR/VR is capable of today and the future that enthusiasts envision. The hardware is progressing, yet there are lack of design tools and methodologies. Effective augmented and virtual reality games and experience require good storytelling, animation, production and solid graphics. Students will learn a myriad of processes including spatial interface design, volumetric capture, working with spatial audio, porting animations and game programming. This course presumes no prior knowledge and is intended to jump start a career in AR/VR development and interaction design. The goal is to have relevant portfolio projects for entering industry. Students will create content in Unity 3D to agnostically deploy on headsets, mobile phones or tablets. Students will have access to a wide range of available hardware. In the beginning of the course, students will learn to implement the most important AR/VR interaction concepts – concepts such as selection, manipulation, travel, wayfinding, menus and inputting text in Unity3D. In the latter part of the course, students will implement prototypes that will be periodically critiqued by experts at Oculus, Magic Leap, Google, and other top companies. *You do not need to be a programmer. This is intended for students interested in 3D art, interaction design, sound design and game development. All are important for immersive development*
Generative machine learning models open new possibilities for creating images, videos, and text. This class explores the idea of how artists, designers and creators can use machine learning in their own design process. The goal of this class is to learn and understand some common machine learning techniques and use them to generate creative outputs. Students will learn to use pre-trained models, and train their own models in the cloud using Runway. For each week, we will discuss the history, theory, datasets, application of the machine learning models, and build experiments based on the model. In addition to Runway, we will be using JavaScript libraries like the p5.js, ml5.js, and TensorFlow.js, and software like Photoshop, Unity and Figma. Students are expected to have taken ICM (Introduction to Computational Media), or have equivalent programming experience with Python or JavaScript. A list of ML models we will be covering: Image generation: StylanGAN: https://github.com/NVlabs/stylegan BigGAN: https://github.com/ajbrock/BigGAN-PyTorch Style Transfer Fast-style-transfer: https://github.com/lengstrom/fast-style-transfer Arbitrary-Image-Stylization: https://github.com/tensorflow/magenta/tree/master/magenta/models/arbitrary_image_stylization Semantic Image Segmentation/Synthesis Deeplab: https://github.com/tensorflow/models/tree/master/research/deeplab Sapde-coco: https://github.com/NVlabs/SPADE Image-to-Image Translation: pix2pix: https://phillipi.github.io/pix2pix/ pix2pixHD: https://github.com/NVIDIA/pix2pixHD Text Generation LSTM gpt-2: https://github.com/openai/gpt-2
Is augmented reality technology about to enter the mainstream? AR platforms have finally become widely accessible to artists, designers, and technologists thanks to recent advances in mobile performance and a new collection of powerful computer vision techniques. As such, the medium offers rich possibilities for experimentation and a chance to rethink how we experience the intersection of the physical and digital. In this course, students will acquire an understanding of basic concepts and techniques necessary to prototype and build simple AR experiences – with a consideration of not just visual but also aural AR. We’ll supplement practical exercises with an overview of the history of AR, and discuss the ethical, legal, and societal considerations cropping up around this topic. Our tool of choice will be Unity, but we will go over prototyping techniques outside of the platform to speed up the design process. If there is interest, we will cover how to get started building projects in openFrameworks, mobile, or web AR – and discuss why or when you might want to work within other platforms. Even though code samples will be provided, students are highly encouraged to have a basic understanding of Unity or at least have taken an introductory programming course. A working knowledge of Unity can be gained through Unity tutorials (https://unity3d.com/learn/tutorials) or Lynda (https://www.nyu.edu/lynda).
Microinteractions are everywhere, you interact with them on a regular basis. Animated emojis, heart buttons that burst with love when you tap them or password fields that shake when you type in the wrong password – these are the tiny animations that produce delightful moments while providing users with valuable feedback. They can intuitively guide your users without having to explicitly write a set of descriptive rules. They are the soul and character of your interface, and if crafted well they can turn every interaction to a joyful moment while improving the user experience. Using Adobe After Effects and Lottie as our primary tool, we’ll learn how to design, animate and export playful SVG animations as interactive components. We’ll examine the basic principles of UI motion, cover how to create loopable animated states with fluid transitions between them.
Light Emitting Diodes or LEDs are used creatively all around us. They have the ability to emit light at different colors and intensities instantly and from very tiny points. How can we make creative visual works out of these amazing devices? What construction methods can we use to make those works reliable? Big LEDs will cover the process of designing large LED systems. We will cover LED array hardware and how to map pixels from computer generated media onto them. We will go through every major part of the hardware – different styles of LED arrays, drivers and gateways, cables, data protocols, and how to safely power all of them. We will learn to use the pixel mapping softwares Enttec ELM and Madmapper. We will also cover the paperwork needed to furnish a professional LED installation, including drafting riser diagrams, plan, section, and elevation views, creating a bill of materials, and writing instructions for users and installers. This year’s final project will be a site-specific LED sculpture installed in a public space at 375 Jay St. The installations will be able to display student chosen media that can be viewed for one minute. Students will work either in groups or alone and can choose from one of four installation options to present on: – A prepared square section of 2.0mm pitch LED video tiles (approx 256px x 256px, 2’-6” x 2’-6”) – A prepared low-resolution sculpture with diffused linear elements (approx 500px, 2’-6” x 5’-0” overall) – A student conceptualized LED video tile project – A student conceptualized low-resolution project Because of this year’s pandemic, unprecedented changes have come to the professional world of LED installations. As a result, we will be using remote tools such as networked-based cameras, remote desktop applications, and virtual private network connections to watch and operate the final projects. We will spend class time setting these tools up together. The two prepared options for the final project will be installed and maintained by the instructor.
Forget the screen. People want to be part of the action. They don’t want to watch detectives and control superhero avatars. They want to solve the mystery and be the hero. They want to experience it. We see this craving for playful experience in everything from immersive theater to escape rooms to the Tough Mudder to gamified vacation packages. Designing live experiences for large audiences that demand agency offers a distinct set of challenges, from how much choice you give each participant to how many people you can through the experience. We’ll look at examples from pervasive games to amusement parks to immersive theater, examining both the design choices and technology that make the experiences possible. Along the way we’ll create large, playful experiences that put the participant at the center of the action.
For better or worse humanity is heading down the virtual rabbit hole. We’re trading an increasingly hostile natural environment for a socially networked and commercially driven artificial one. Whether it’s the bedrooms of YouTube streaming stars, the augmented Pokestops of Pokemon Go, the breakout rooms of a Zoom meeting, or even the “airspace” of Airbnb; we are witnessing a dramatic transformation of what occupying space means. The socially distanced measures as a result of the COVID-19 pandemic have only accelerated this societal embrace of the virtual. So where are these dramatic spatial paradigm shifts occurring? Who owns and occupies these spaces? Who are the architects and what historical and ethical foundations are they working from? What world do they want to build for humanity and where does the creative individual fit into it? Will it be a walled garden, a role-playing adventure or a tool for creating more worlds? The course will ask students to embrace the role of virtual architect, not in the traditional brick-and-mortar sense of constructing shelter, but in terms of the engagement with the raw concept of space. However this virtual space must be considered and evaluated as a “site,” that is activated and occupied by real people and all the limitations of physical space that they bring with them from the real world. This is the foundation of synthetic architecture; simulated space met with biological perception. This conceptual architecture is free from the confines of physics but host to a whole new set of questions: How do we embrace the human factors of a dimensionless environment? How do we make or encourage meaningful interactions within the limits of current technology? New models of interaction must inform and shape the architecture of virtual space – what does that look like? How can architecture and aesthetics inform the creation of virtual environments and immersive narratives? How do we acutely consider the psychological and social impacts of the worlds we design and what is the metaphorical ground plane to make sense of this virtual world, unbound by physics? About Jonathan Turner: http://www.jonathanwilliamturner.com/about/
Magic windows that allow us to peek into different realities without leaving our physical space, lenses that reveal hidden layers of objects or navigating new universes within the same room. More than ever, mobile devices are getting a human-scale understanding of space and motion allowing us to create more intimate interactions with our surrounding spaces, leveraging them as a canvas to experience other realities. We now have the potential to give life to inanimate objects, tell stories through space, customizing private views of public spaces and recognize places we’ve never been. We’ll question what it means and how can we blend reality exploring themes such as: augmented space and new paradigms in social interaction, public space and privacy; storytelling and navigating the physical space like turning pages in a book; tangible interfaces, mixed objects and animism; Magic windows, x-ray vision, time-machines and impossible universes; Far away so close: telepresence and remote collaboration. The course will survey the past, current and up and coming technologies and experiences in Mixed Reality including environmental augmented reality and interactive projection mapping, handheld devices while fostering a strong user experience perspective on the affordances and constraints of each. We’ll research and discuss the design principles and guidelines for creating mixed reality experiences focusing on the links between real and virtual objects, interaction space and asymmetries between physical and digital worlds, environmental semantics and multimodal and tangible interaction. Technologies explored will be focusing on mobile platforms (phones, tablets) including Vuforia, SLAM, image and object recognition, depth sensing, projection mapping. Unity3D will be the development platform: students must have previous working knowledge of Unity3D and feel comfortable with independently developing using this platform. A working knowledge of Unity3D may be gained by going through the Unity 5 3D Essential Training Lynda Course prior to the course (log in to Lynda for free via https://www.nyu.edu/lynda).
The medium of motion pictures will be will be transformed by virtual reality technologies. But the emerging hybrid form will likely have less to do with the iconic VR headset and immersion, than in the newly possible flow of expression in the other direction, out of the participant. This class looks at the true potential of virtual reality as its mutability, to put ordinary users in the role of director of visual media as they already are in their dreams and fantasies. Democratizing media by breaking it down into discrete more easily remixable parts has historically, from DNA to alphabets to movable type led to an explosion of expression and knowledge creation. Motion pictures, perhaps our most convincing medium, is now undergoing such a transformation thanks to virtual reality technologies. Specifically, we will look at how tools like depth cameras, motion capture, and machine learning can treat a scene as a collection of elements instead of a collection of pixels. We will work with the real-time 3D rendering capabilities available, even in your phone, for the instant contrivance of visual reality using compositing, the transformation of images and models as well as virtual camera moves. The class will also consider how the more conceptual dissection of film by cinema scholars and psychologist into things like plots and character types might be templated into reusable formulas for non-experts to create an emotionally satisfying dramatic arcs.
Code without content gets boring fast. This seven week course will show you how to create stories around which you can weave the technology learned in other classes. When content comes first, interesting problems arise to solve. Participants will get solid grounding in how to tell a visual story using words and images in a traditional format, so then they can take that format and reimagine it in entirely new and unique ways. The first few classes are devoted to getting basic comic skills. The remaining classes will hone and expand these abilities while posing the question: what can be done differently, and how can technology add to what we have created? At the end of the semester you will have a something that sets you apart; – original content AND technological know how. Students will combine words and images, look at each other’s work, look at examples of published works. Reimagine how these stories can be told in new and unique ways. This a demanding course. There is a lot of work involved, they will end up with a lot of original content. During the first half we look at and make traditional comics. Second half we experiment with comic format WHILE honing storytelling skills. Relevant speakers will come in to discuss what they do and how they work etc.
This course teaches the ins and outs of using image processing software with an aim towards some type of real-time use (e.g. a performance or installation). The class looks at ways to manipulate different visual media (time-based, still, vector, and rendered) in real-time to allow students to develop interesting real-time performance systems. While the focus of this class is on using Max for visual work (through a software package called Jitter), it also looks at how to integrate interactive elements (sound, physical interfaces, etc.) into the work. Class time is spent on interface design and software development issues as well. The class explores some interesting capabilities of the software in terms of real-time camera input and tracking, generative graphics systems, and media transcoding. Throughout the class students develop and share ideas on live performance as a medium for visual expression, and learn the software tools necessary to put these ideas into practice in the form of idiosyncratic performance systems. A final presentation in the form of a group performance will be arranged. Students should have some working knowledge of Max/MSP before taking this class.
Wall labels, audio guides and informative maps are just some of the ways galleries and museums convey additional information about an art collection. How can we utilize new interactive mixed reality tools to design and deliver immersive experiences that breathe new life into an exhibit. Augmented and virtual reality are powerful tools for new media production and storytelling, but how can these tools serve to enhance our Wall labels, audio guides and informative maps are just some of the ways galleries and museums convey additional information about an art collection. How can we utilize new interactive mixed reality tools to design and deliver immersive experiences that breathe new life into an exhibit. Augmented and virtual reality are powerful tools for new media production and storytelling, but how can these tools serve to enhance our gallery experience without distracting from the power and importance of a pre-existing collection? This production course seeks to experiment with new ways to experience a museum collection through mixed reality. Topics covered include exhibition installation and curation, mixed reality production in Unity, mobile development for Augmented Reality.
This course will introduce students to the design and development of Virtual Reality experiences. We will examine these increasingly popular means of delivering content and social interactions and identify their unique affordances over existing platforms. Students will be challenged to harness the specific advantages of VR from conception through functional prototype. The class will also cover case studies of effective use of VR in information delivery, as well as social and artistic experiences.
We live in an era of information where the information can be written, accessed, shared, and also eliminated with a single stroke. As a result, the objective “truth” is brought to a question. In the last decade, artists have been experimenting with the fakeness of the truth and the truthfulness of the fake by creating fake documents, staged marriages, an arguably authentic artifact, imaginary advertisements both historical and contemporary. What does it mean to tell the truth in the context of art? How does art cross the boundaries between the real and the fake, truthfulness and misrepresentation? This course will examine social engagement of art and how “truth” is treated, interpreted, and presented. The class will take a field trip to a propaganda museum, have readings and discussions, and analyze artists working with fiction as a medium in art making. Students will work on projects to construct believable reality through object making (3D fabrication) and narrative construction (audiovisual material). Prerequisites: Interaction Lab or Communication Lab or Application Lab
Tangible heritage (site, object, and structure) and intangible heritage (motif, icon, character, textile, wardrobe, music, performance, language and ritual) are unseparated parts of the cultural heritage. The narrative and messaging of cultural heritage can be preserved by moving sequences, motion design and animation. The richness of heritage contents can be further disseminated and known by the dynamic media. This course aims to utilize animation and motion media to depict and preserve the richness of cultural heritage contents. 3D animation and motion graphics techniques will be addressed and applied to the storytelling. Students will be guided to research the Asian cultural heritage contents including the tangible and intangible heritage. They will further explore the visual design and production pipeline of animation. Visiting expert of interactive media design and intangible heritage performance will get involved to share the insights to the students. Prerequisite: None. Fulfillment: IMA/IMB elective.
Virtual Reality and Augmented Reality represent, respectively, visions of “being” somewhere else or augmenting your present environment. These visions are not new, but new technologies have made it possible to produce experiences unlike anything before, particularly through the use of headsets, spatial audio, touch sensors, and custom location-based installations. These new technologies are becoming small, powerful, and inexpensive, and as a result we are witnessing the birth of a powerful new medium, new art form, and new industry – all very quickly. The speed of VR and AR growth has created both opportunity and confusion. “VR / AR Fundamentals” takes a long, deep perspective. We will overview such basic elements as audiovisual resolution and fidelity; spatiality and immersion; other senses such as touch, smell, taste (and even mind); input and interactivity; and live and social. We’ll look at distinctions such as cinema versus games, movies versus models, public versus personal, real world versus fantasy worlds, linear versus interactive, and narrative versus ambient. These elements and distinctions will be presented partially as technical but in an understandable way for general liberal arts students, and will rely heavily on experiencing content and keeping up with current events. That’s the first half-semester. The second half-semester we’ll concentrate on collectively producing a series of timely and relevant projects, all short, entertaining, and useful to others exploring the world of VR / AR. Prerequisites: None Fulfillment: IMA Major Electives; IMB Major Interactive Media Elective.
Board game design is a one-semester course for students who want to dig deeper into table-top games, from design to history to manufacturing. The first half of the course looks at the world of mass market games, which focus heavily on commercials, trends, plastics, licenses, low prices, and casual rules. The second half focuses on hobby games, designed for the dedicated game player, and the different styles of games in that world. The course is hands-on with at least one published game played in every class. There are multiple assignments where students bring these concepts to life through their own designs. Throughout the course, there is a focus on understanding players and designing games for a target audience.
Narrative Design is an advanced game design course where students learn a variety of strategies to bring together game design and storytelling, both in table-top and digital games. Every assignment covers a different challenge when it comes to integrating systems design with storytelling. Students will also learn some of the basics of storytelling, such as character development, dramatic action, generating conflict, and world-building.
Intermediate Game Design builds on the foundation of Introduction to Game Design to help build students’ understanding of how game design works in a practical context. While Introduction to Game Design acquaints students with basic foundational concepts and ideas, Intermediate Game Design puts those ideas into action across four very different kinds of projects. These projects emphasize the professional context of digital game design.
This course introduces students to critical video—the use of documentary, ethnographic, and research-based video to investigate and critique contemporary culture. Students gain a theoretical overview of documentary video, a set of conceptual tools to analyze video, and an introduction to the practice of video production for small and mobile screens. Students apply texts on video’s history, culture, and distribution, as well as the ethical challenges of video production, to their own, research-based video project. No prior experience required.
This course will combine a history of video art and experimental film with practical training in the use of live video performance art technology. Students will explore new ways to create and edit films and videos using VJ software, projections, and multi-channel video surfaces. Workshops will demonstrate concepts and software that can be integrated into the creative process of video performance art and video art installations. COURSE OBJECTIVES At the completion of this course, the student will be able to: Draw inspiration from the recent history of incredible video and multi-media artists. Develop an understanding of audio and visual hardware used by VJ’s. Use live VJ software to manipulate digital media in real time to create Video Performance Art. Use Projection Mapping techniques to project video art onto 3D surfaces. Create original video performance art, video installations, and other performance pieces. Utilize skills to make video art in the professional market.
Introduction to Game Development is a practical course that introduces students to the methods, tools and principles used in developing digital games. Over the course of the semester, students will work alone to create a two digital prototypes or ‘sketches’, before building on them to produce a final polished game, using the lessons learned in the earlier prototypes. This is a hands-on, primarily lab-based course, and so the focus is on learning by doing rather than on reading and discussion.
This course provides an overview of experimental film and video history and theory, centered on hands-on production. The course will include readings, workshops, screenings, discussions, assignments, critiques, and technical instruction around cinema as a radical practice, and the ways in which computing techniques can be leveraged for new forms of expression in the moving image. Students will study and experiment with cutting-edge techniques in cinematic production such as motion capture, 3D (stereo, depth) filmmaking, and interactive / performative cinema. | Prerequisite: DM-UY 2263
This class continues from DM-UY 2153, moving into advanced technological implementations of 2D games. Taking designs from DM-UY 2153 and working in teams, students create a complete game. Students, based on abilities and individual goals, are assigned individually to work in production areas ranging from sprite creation, mapping and level design to engine coding and interaction scripting. Students are responsible for completing their assignments as if they were members of a professional game-development team. | Prerequisite: DM-UY 2153
This course explores sound design, primarily within visual contexts. The course will focus on the use of sound within visual and interactive media, including film, video production, interactive user experience, web design, and gaming. Students will create weekly studio assignments in all of these areas, with an emphasis on developing a strong competence in integrating digital audio techniques into other media. Final projects could include novel sound design developed for film, video, web, applications, or games. | Prerequisite(s): DM-UY 1113 or MPATE-UE 1001
Students will learn advanced techniques of 3D computer animation, along with the theories and principles of motion including motion capture. Students will become comfortable utilizing cameras, lights, dynamics, motion, and effects in an industry-standard software. Comprehensive critiques will be conducted regularly to encourage good design for time-based animation. | Prerequisites: DM-UY 2133
Whether it’s through photo realistic scans found in current-gen video games or the cartoonish and low-fi aesthetic of Bitmoji there is no limit to ways in which the body and the self are represented in digital spaces. This 2-credit class will look at how avatars have been historically used in the realm of art, commerce, and entertainment and utilize existing avatar creation tools to develop projects that examine identity, body politics, and contemporary performance. In class, we will cover the basics of Unreal Engine, photogrammetry, 3D scanning, and model rigging although students will be encouraged to use existing skill sets and creative thinking to complete some of the smaller week-by-week assignments. The class will culminate with a short performance, small installation or single/multi-channel video piece using one or more of the techniques covered in class. This can be a solo project or a group project. In this class students will: – Explore how avatars can be utilized in your creative practice – Gain an introductory understanding of Unreal Engine, photogrammetry, model rigging, and 3D scanning. – Learn how to recontextualize digital spaces for the purposes of art, installation, and performance. – Broaden your thinking of what performance can be, both in a physical setting and digital setting. – Think critically about how physical bodies inhabit digital spaces and how the hardware and software we use reinforces the acceptance and value of certain kinds of bodies.
This course investigates and explores the integration between cultural heritage and digital conservation, specifically towards the objects, deities, sites, and gardens of China. Through the reflection of Chinese cultural heritage from the global perspective of Chinese history, the course raises awareness of heritage conservation and critical heritage studies towards the origins and the transformations of China in the contemporary era. Academic readings and oral presentations revolving around this theme will enhance students’ skills for documentation and restoration by innovative digital techniques in China. Conservator presentations, field trips, gallery visits and art projects enable students to communicate with practitioners in the field and examine the values of Chinese cultural heritage from a global perspective. Prerequisites: None Fulfillment: IMA/IMB elective; GCS Chinese Media, Arts, and Literature.
We live in a world immersed in sound yet we rarely attend to how sound can reflect our social structure or reveal cultural meaning. This course introduces students to acoustic ethnography, soundscape studies and narrative, non-narrative audio storytelling. We will gather and analyze the acoustic environment of China, using recorded sounds to create ethnography through text and sound. Ethnography (literally, “culture-writing”) is both the act of gathering data about culture through observation and interviews as well as the practice of writing analytically about cultural difference. Visual ethnography incorporates the analysis of visual and material aspects of our social environment into creative, multimedia rich projects. With an ethnographic approach to sound, we will document the rich tapestry of sounds around us in the context of the Yangtze River Delta region and think about how this conveys China’s culture, society and history. Through lectures, discussion, readings, listening assignments, field studies and projects, we will re-learn how to listen, observe and record the sounds in our environment. We will study Chinese sound art and Chinese cultural productions in music, film, television and multimedia installation. We will contextualize Chinese sound art against major theoretical approaches to sound including archives and preservation, form versus content, and social studies of science. Students will work collaboratively or individually on a final project that combines sound recording and production, to create an ethnographic analysis of an aspect of social and cultural life in the Yangtze River Delta region. Students will gain experience in gathering ethnographic data and they will transform it into an analytical or creative project integrating sound art and text. Prior knowledge of sound editing and Chinese language is not required. Prerequisite: None
This class examines the idea and practices of ‘cultivation’ in relation to the challenging environment of the 21st century city. Through field trips, readings and discussions, the class explores the concept of cultivation, and how it can be used as a basis for researching the urban ecology of Shanghai, both as a past and future city. The class incorporates a major project in the digital humanities, in which students use the tools of interactive media (audio, video and cartographic technologies) to research, map and narrativize the ways in which architects, designers, artists and thinkers engage with the traditions of cultivation in order to imagine and recreate the future metropolis. Prerequisite: Writing as Inquiry
From the history of visual music and abstract film to the contemporary notion of live cinema, this course will be an exploration of the synesthetic relationship between sound and visuals in a realtime performance setting. Dating back as far as the 18th century, systems have been invented to produce images alongside music linking the two through formalized arrangements. Current media technologies make developing such systems both more approachable and more expansive in their scope. Through readings, viewings, and case studies students will gain an understanding of the history and theory of live audiovisuals. During the course students will team up to develop and master a realtime audiovisual system of their own invention. The class will culminate in a show in which they will present their work through a live performance. Prerequisite: None.
This course will be like the two previous ones (http://ima.nyu.sh/vr-ar-fundamentals/), with the first half-semester as seminar and the second half semester as production. For Fall 2019, production will be something in complement with Zaanheh: A Natural History of Shanghai (https://zaanheh.research.shanghai.nyu.edu/), a new NYU Shanghai interdisciplinary project. Prerequisite: None
In this course, students will work in teams to design digital learning resources and experience designs at the intersection of music, coding, arts, and technology. The course will begin with an introduction to emerging trends in learner engagement and design-based research, especially related to web- and mobile-based musical experiences and principles of making music with new media. Innovations in and applications of musical creativity, interactive technologies, user-centered design & engagement, scaffolded learning, creative learning, pedagogies of play and making, and educational entrepreneurship in Chinese contexts will also be explored. The market for creative educational experiences in afterschool settings for youth in China is exploding. For-profit educational service companies are competing and searching for differentiated, learning experiences in music, coding, and creative project based learning that will attract high-paying parents looking for the best supplemental education for their children. This practical, hands-on course will explore questions such as: How can we design engaging, creative learning experiences that are relevant to the cultural goals and needs of today’s youth in China, while laying the foundation for creative learning for the workforce of tomorrow? What are engaging, effective creative learning resources, and how are they best implemented in Chinese learning settings? How can we take advantage of young people’s near ubiquitous love of music and technology to facilitate creative learning? Students will work together in teams and paired with a partner audience of learners and teachers in Shanghai drawn from local and regional international schools (e.g. Alibaba’s Cloud Valley), local afterschool programs (e.g., Music Lab), and cultural partners (e.g., Shanghai Symphony). Together they will assess the needs and opportunities of partner students and teachers, and engage in a two-stage iterative and reflective co-design process prototyping custom learning resources and experience designs with their partner end users. At the end of the course, students will present and demo their learning resources as part of a public showcase to an external audience of partners, educators, technologists, musicians, entrepreneurs, and experience designers in Shanghai. Prerequisites: None.
Collaborative Arts Lab engages students across disciplines and schools in a layered process of inquiry, dialogue, and creative research in response to different issues of urgency in our community. Each semester, a theme is chosen for the course which serves as both the introduction to the frame for research and the introduction to the class group. This semester, the course will be rooted in performance as a frame to examine what culture is built around the body as subject and explore perceptions of art based on the body and different physicalities. You will work individually and in groups in movement-based exercises, as well as in other artistic mediums, to explore social, historical, and cultural contexts, analyze existing research, develop questions for creative inquiry, and experiment with new ways of thinking about the body as subject and as inspiration for the creative process. This course has no prerequisites and welcomes students from all schools and from all/any creative backgrounds – writers, dancers, actors, musicians, visual artists, filmmakers, designers, photographers, etc. Students will be expected to create works together, and each student will have the opportunity to integrate different creative forms into their class projects.
This is an intensive, hands-on workshop addressing the complex challenges of game design. The premise of the class is that all games, digital and non-digital, share common fundamental principles, and that understanding these principles is an essential part of designing successful games. Learning how to create successful non-digital games provides a solid foundation for the development of digital games. Students will analyze existing digital and non-digital games, taking them apart to understand how they work as interactive systems. A number of non-digital games will be created in order to master the basic design principles that apply to all games regardless of format. This course is subject to a non-refundable department fee, please see the Notes section for more detail.
Hollywood in your palm. That is what this combination of lectures, screenings, demonstrations and practical production workshop will offer to the students in this course. There will be several professional guests making presentations and Q&A sessions from the mobile phone filmmaking industry. In addition to the historical and critical overview of the emergence and exponential growth of global cell phone cinema, students will shoot all footage on cell phones and download them for computerized editing. The final project will be under three minute shorts. Projects will include all genres of film and television: news, mini-documentaries, animation, music videos and narrative shorts. Completed student projects will be suitable to be posted on the Internet and entered into domestic and international mobile phone film festivals. For example, two minute long improvisations of Bollywood Style Music Videos shot on Cell Phones by the students have been projected at the Tribeca Cinemas as part of the New York Indian Film Festival. It is suggested but not compulsory that students bring to the class a cell phone capable of recording video.
Motion graphics can be found in a wide range of media: broadcast, web, animation, and film to name a few. This course will allow students to explore the elements of time and space to convey messages and meaning through type, image, and sound for the screen. Individual creativity will be stressed as well as the understanding and use of industry-standard software for developing motion graphics. | Prerequisite: DM-UY 1123
This course is an orientation to the essential concepts and practices of digital audio. It is a creative and theoretical foundation studio that combines an orientation to sound and listening with fundamental techniques of digital audio production: recording, editing, and mixing. The course covers topic areas such as microphone and field recording techniques, recording studio best practices, audio editing, DAW (digital multi-track) production, and mixing.
This course provides an overview of image making and presentation techniques, from still to moving. Students will also be introduced to experimental image making. This course will cover introductory still and video camera use, as well as how to begin integrating image within media. Students will gain practical and analytical skills through workshops, assignments, critiques, technical instruction, readings, screenings, and discussions.
This course provides an overview of documentary film and video history and theory, centered on hands-on production. The course will include readings, workshops, screenings, discussions, assignments, critiques, and technical instruction around documentary and pseudodocumentary forms. | Prerequisite: DM-UY 2263
In this course, students complete a coordinated sequence of short projects that add up to a finished, live-motion video project. The course strongly emphasizes the relevance of particular tools and techniques to the specific project Concepts are introduced through the screening of historical examples, from 1895 to the present. The course format is modeled on professional standards and workflow for preproduction, production and postproduction. | Prerequisites: DM-UY 2263
How do games tell stories? How can we move beyond the traditional narrative in games? Is there a more holistic approach that embeds the story deeply into the interaction? In this class, students will begin to answer these questions by analyzing games and developing their own experimental narrative games. | Prerequisite: DM-UY 2153
In this course students will build form the skills they learned in 3D Modeling and 3D Animation to produce 3D for Interactive Applications. Projects may be geared to scientific, engineering or entertainment applications according to individual skills and professional aspirations. | Prerequisites: DM-UY 2133
This course explores various image editing techniques and outputs, utilizing various styles of text, including fiction and non-fiction. The assignments foreground how text influences viewers’ perception of images, and how images can enrich a body of writing. Students will use a range of production skills to create work — using After Effects for animation, HTML/CSS for website creation, book and print design, and archival printing methods. All projects are accompanied by readings that provide historical and theoretical grounding to support the concepts explored through practice. An emphasis on refining technical and aesthetic photography skills are central throughout the semester. Prerequisites: DM-UY 2183 or DM-UY 2263
This class introduces the principles of 2D and 3D computer-game design. Students learn the range of game types and understand their conceptual building blocks. Students complete a structured sequence of assignments toward the completion of a new-game design. Students prepare, through a staged sequence of assignments, a fully worked-out design for an original game. Criteria include storyline quality, graphics quality and appropriateness of design to the game concept originality. For games with an educational or instructional purpose, clarity and effectiveness for the target audience is considered. | Prerequisite: DM-UY 1133
Making Webisodes is an intensive 14 week course which combines lectures and workshops in which students create unique and compelling content for the web and then learn how to post that content on the web. Students will explore the basics of film production and online webisode distribution, working with – concept creation – writing – directing – acting – production design – camerawork – sound – editing – online tracking tools and social media – web monetization and advertising. The webisode is an exploding new art form. Web series, embedded ads, 5 second hooks, snapchats, vines and viral videos all present a variety of new media approaches within the entertainment industry, business, lifestyle, and politics. Webisodes are short visual presentations that either entertain us, directly sell us product, indirectly sell us product, or shock and engage our perspective, as in political propaganda videos. Lectures provide students with an overview of the emerging web series industry, concentrating on how the webisode is used to hook the audience, generate hits, and drive customers to websites and/or online advertising. Workshops then employ practical exercises to help the students conceive and create their own unique webisode, which can be narrative or non-narrative, fiction or non-fiction, experimental or satire, personal or political. The goal is to use the resources at hand and create instant media – webisodes. As the students produce their webisodes, they will learn by doing and they will be provided with practical knowledge of the art, craft, and commerce of webisodes.
Students conceive, produce, direct, and edit a short film exploring the Paris experience with smartphone technology. A survey of cellphone cinema history leads to the study of visual storytelling principles and techniques, which students apply through practical exercises. Choosing among available short film genres (experimental, documentary, portrait, essay, fiction), students are trained through every stage of the movie making process: pitching the idea, scripting and storyboarding, shooting, and editing. Each student finishes the course with a facility in smartphone video technology as well as a coherent film record of his or her particular vision of Paris.