Computation and Data (Old Structure)

AI for Creatives (Topics in Media Art)

Tyler Peppel | Syllabus | IMNY-UT 281 | Mon 7:00pm to 8:30pm in 370 Jay St, Room 410> Wed 7:00pm to 8:30pm in 370 Jay St, Room 410 Meetings:7-First Half
2 Point, All, Computation and Data (Old Structure), Elective, Evening, Fall 2025, Monday, Wednesday
Last updated: April 4, 2025
Show Course Description

A survey and hands-on workshop on AI augmentation of creative communication.

We will get hands-on with AI tools for photography, design, art, writing, UX, illustration, and video.
We will produce, critique, and exhibit creative work that tests the creative capabilities of these new AI tools.

Questions we will consider:
How does AI influence the creative process?
How does this new human/machine collaboration challenge my role as a creator?
Who is creating these new tools and how do they work?
What is the longer-term impact of AI technologies on the creative professions?
Our goals are to understand AI technology, use it to produce creative work, and use what we learn to influence the larger conversation about creativity and AI.

Will AI diminish or enhance human creativity? It’s up to us.

Three.js for Makers (Topics in Media Art)

Brian Ho | Syllabus | IMNY-UT 281 | Tues 7:00pm to 8:30pm in 370 Jay St, Room 410> Thur 7:00pm to 8:30pm in 370 Jay St, Room 410 Meetings:7-Second Half
2 Point, All, Computation and Data (Old Structure), Elective, Evening, Fall 2025, IMA Major Elective, Spring 2024, Spring 2025, Thursday, Tuesday
Last updated: April 4, 2025
Show Course Description

“In this increasingly online world, the internet has proven to be a powerful tool that can connect us with one another, host meaningful experiences, and provoke critical thinking. In this class, students will have an opportunity to learn about breaking out of the 2D web page and the fundamentals of working with 3D on the web.

This course hopes to introduce new avenues for creative expression and experimentation via the web and promote learning practical web development skills through experiential learning. Students will use Three.js to create dynamic and immersive web-based experiences that push the boundaries of what is possible online.

The course is intended for technologists who have no programming or computer science background but are interested in 3D exploration on the web. Nothing more than a basic understanding and familiarity with CSS, HTML, and Javascript is required.”

The Code of Music

Luisa Pereira | Syllabus | IMNY-UT 222 | Mon 12:20pm to 3:20pm in 370 Jay St, Room 409 Meetings:14
4 Point, All, Computation and Data (Old Structure), Elective, Fall 2025, In-Person, Midday, Monday, Programming and Data (New Structure)
Last updated: April 4, 2025
Show Course Description

In this course, students learn how to create musical systems –pieces that incorporate randomness, interact with their listeners, or evolve over time, in the browser.

We will start by creating audiovisual instruments and sample-based interactive songs, as students review their p5.js skills and are introduced to the Tone.js music library. Then, we will turn to a structured exploration of the elements of music, focusing on rhythm, melody, timbre, and harmony. For each, we will hold listening sessions, represent and manipulate the element in code, and interact with it via a range of existing interfaces. Students will explore the possibilities that computation and interactivity open up by designing and implementing a series of interactive studies.

The last few weeks of the semester will be dedicated to introducing algorithmic composition techniques such as Markov Chains and Neural Networks. During this time, students will also develop their final project: an interactive/generative musical piece that builds on their previous classwork.

Throughout the course, students are encouraged to bring in their musical tastes and interests into the classroom. This class is a good fit for students who are interested in:
– Creating interactive music pieces and digital instruments.
– Deepening their understanding of how music works. All musically-curious students are welcome: previous experience with music and audio will be useful, but is not required.
– Continuing to develop coding skills. Creative Coding or equivalent programming experience is required.

About Luisa Hors: www.luisapereira.net/

Prerequisite: Creative Computing (IMNY-UT 101)

Networked Media

Sam Heckle | IMNY-UT 223 | Tues 12:20pm to 1:50pm in 370 Jay St, Room 412> Thur 12:20pm to 1:50pm in 370 Jay St, Room 412 Meetings:14
4 Point, All, Computation and Data (Old Structure), Elective, Fall 2025, In-Person, Midday, Programming and Data (New Structure), Thursday, Tuesday
Last updated: April 4, 2025
Show Course Description

The network is a fundamental medium for interactivity. It makes possible our interaction with machines, data, and, most importantly, other people. Though the base interaction it supports is simple, a client sends a request to a server, which replies; an incredible variety of systems can be and have been built on top of it. An equally impressive body of media theory has also arisen around its use.

This hybrid theory and technology course will be 50% project driven technical work and 50% theory and discussion. The technical work will utilize JavaScript as both a client and server side programming language to build creative systems on the web. Technical topics will include server and client web frameworks, such as Express, HTML, CSS, templating, and databases. The theory portion of the course will include reading and discussion of past and current media theory texts that relate to the networks of today.

**** it is HIGHLY recommended you take Front End Web Development (or have equivalent front end web development experience) to get the most out of this course. We will be going over fundamentals of HTML/CSS but it would be useful to have prior knowledge ***

Real-Time Media

Carrie Sijia Wang | Syllabus | IMNY-UT 285 | Tues 3:40pm to 5:10pm in 370 Jay St, Room 407> Thur 3:40pm to 5:10pm in 370 Jay St, Room 407 Meetings:14
4 Point, Afternoon, All, Art and Design (Old Structure), Computation and Data (Old Structure), Elective, Fall 2025, Media and Entertainment (Old Structure), Thursday, Tuesday
Last updated: April 4, 2025
Show Course Description

This course focuses on designing, developing and delivering real-time, performative work using audio and video elements. The class will have an emphasis on using MaxMSPJitter and other tools to create performative experiences that dynamically combine interactive elements such as video, sound, and code, allow for the unfolding of engaging narratives, and generate compelling visuals in real time.

We will look at various examples of both multimedia performances and installations, explore how we can apply the technologies we have learned to design real-time systems, and discuss methods we can use to make our work more engaging.

The class is three-fold and divided into tech tutorials, discussions of existing examples, and in-class performances.

Introduction to Machine Learning for the Arts

Yining Shi | IMNY-UT 224 | Fri 5:20pm to 8:20pm in 370 Jay St, Room 408 Meetings:14
4 Point, All, Computation and Data (Old Structure), Elective, Evening, Fall 2025, Friday, Programming and Data (New Structure)
Last updated: April 4, 2025
Show Course Description

An introductory course designed to provide students with hands-on experience developing creative coding projects with machine learning. The history, theory, and application of machine learning algorithms and related datasets are explored in a laboratory context of experimentation and discussion. Examples and exercises will be demonstrated in JavaScript using the p5.js, ml5.js, and TensorFlow.js libraries. In addition, students will learn to work with open-source generative models including text generation models and image generation models. Principles of data collection and ethics are introduced. Weekly assignments, team and independent projects, and project reports are required.

Front-End Web

Lenin Compres | IMNY-UT 228 | Thur 5:20pm to 8:20pm in 370 Jay St, Room 412 Meetings:14
4 Point, All, Computation and Data (Old Structure), Elective, Evening, Fall 2025, Thursday
Last updated: April 4, 2025
Show Course Description

This course will provide a foundation for understanding modern web development with a focus on front end technologies and accessing public data. The forms and uses of these technologies are explored in a laboratory context of experimentation and discussion. This studio stresses interactivity, usability, and the quality and appropriateness of look and feel.
Students will create two web applications, including one that leverages public APIs and Javascript libraries. The goal of the course is for students to learn how to think holistically about an application, both by designing a clear user experience and understanding the algorithmic steps required to build it. Assignments are arranged in sequence to enable the production of a website of high quality in design and engineering.

Mobile Application Development (Topics in Computation and Data)

John Henry Thompson | Syllabus | IMNY-UT 220 | Fri 09:00am to 12:00pm in 370 Jay St, Room 408 Meetings:14
4 Point, All, Computation and Data (Old Structure), Elective, Fall 2025, Friday, Morning
Last updated: April 4, 2025
Show Course Description

One of the most transformative consumer products in history, the iPhone remains the standard bearer for great design and user experience. With the latest versions of iOS and iPhone, Apple puts depth sensing and augmented reality in our pockets. How do we take advantage of this incredible platform to produce our own compelling experiences?

This course will be a hands-on workshop where we explore the world beyond generic apps and push the boundaries of what’s possible on iOS hardware. Each week, you’ll be asked to complete a programming exercise meant to foster your understanding of iOS application development. We’ll leverage existing open source libraries to quickly build out your app with features such as real time communication and cloud storage.

We aim to create distributed instruments for computed expression.

Full-time access to an iOS device and a Mac laptop computer running the latest operating system and development tools are required.

Prereq: Some programming experience (such as ICM) and willingness to learn Apple’s Swift programming language.

Prerequisite: Creative Computing (IMNY-UT 101)

Immersivity Beyond the Display

MK Skitka | Syllabus | IMNY-UT.281 | Mon 7:00pm to 8:30pm in 370 Jay Street Room 450> Wed 7:00pm to 8:30pm in 370 Jay Street Room 450 Meetings:
2 Point, Computation and Data (Old Structure), Elective, New, Spring 2025
Last updated: December 5, 2024
Show Course Description

In recent years, “immersive digital experiences” have gained widespread popularity, from commercial exhibits like Van Gogh: The Immersive Experience to museums adapting spaces for digital art and brands leveraging interactive installations for promotion. But are these experiences truly immersive, and how can we evaluate them?

Immersivity Beyond the Display delves into the design of visual, interactive installations. Each week, students will modify existing sketches in p5.js to explore immersive design principles while building on foundational programming skills. “Demo days” will provide opportunities to test and reflect on each other’s prototypes. We will focus on narrative development, embodiment, social and emotional engagement and interaction mechanics to frame our exploration.  Weekly exercises will be enriched with relevant readings and discussions. By the end of the class, students will have developed their own model for evaluating immersivity in interactive digital experiences.

Prerequisite: Students should have completed Creative Computing or possess equivalent coding experience with p5.js and JavaScript.

After Creative Coding (Topics in Media Art)

MK Skitka | Syllabus | IMNY-UT.281 | Mon 7:00pm to 8:30pm in 370 Jay Street Room 450> Wed 7:00pm to 8:30pm in 370 Jay Street Room 450 Meetings:
2 Point, Computation and Data (Old Structure), Elective, Fall 2024, New, Spring 2025
Last updated: November 5, 2024
Show Course Description

This seven-week course challenges students to extend their knowledge of creative computing into the exciting realm of embodied interfaces. Building upon the fundamentals taught in Creative Computing, this course will explore the design and implementation of interactive systems that integrate the body as a controller of technology.

Going beyond the limitations of traditional mouse, keyboard, and screen interactions, we will investigate how movement, gesture, and physical interaction can be used to create engaging and meaningful user experiences. Through a combination of lectures and hands-on workshops, students will gain a deeper understanding of:

Sensor Technologies and Physical Computing: Working with sensors (e.g., accelerometers, gyroscopes) to capture and interpret human movement and environmental data.

Movement Analysis and Interpretation: Techniques for analyzing movement patterns, extracting meaningful features, and mapping them to computational processes.

Performance and Embodiment in Technology: Investigating how technology can facilitate new forms of expression and performance.

The course will culminate in one proof of concept project utilizing one of the techniques explored in class.

Prerequisites: Completion of Creative Computing or equivalent experience with Arduino and Javascript p5.

Topics in Media Art: After Creative Coding

MK Skitka | Syllabus | IMNY-UT.281 | Mon 7:00pm to 8:30pm in 370 Jay Street Room 450> Wed 7:00pm to 8:30pm in 370 Jay Street Room 450 Meetings:
2 Point, Computation and Data (Old Structure), Elective, Fall 2024, New, Spring 2025
Last updated: October 11, 2024
Show Course Description

This seven-week course challenges students to extend their knowledge of creative computing into the exciting realm of embodied interfaces. Building upon the fundamentals taught in Creative Computing, this course will explore the design and implementation of interactive systems that integrate the body as a controller of technology.

Going beyond the limitations of traditional mouse, keyboard, and screen interactions, we will investigate how movement, gesture, and physical interaction can be used to create engaging and meaningful user experiences. Through a combination of lectures and hands-on workshops, students will gain a deeper understanding of:

Sensor Technologies and Physical Computing: Working with sensors (e.g., accelerometers, gyroscopes) to capture and interpret human movement and environmental data.

Movement Analysis and Interpretation: Techniques for analyzing movement patterns, extracting meaningful features, and mapping them to computational processes.

Performance and Embodiment in Technology: Investigating how technology can facilitate new forms of expression and performance.

The course will culminate in one proof of concept project utilizing one of the techniques explored in class.

Prerequisites: Completion of Creative Computing or equivalent experience with Arduino and Javascript p5.

Topics in Computation and Data: Mobile Application Development

John Henry Thompson | Syllabus | IMNY-UT.220 | TBD Meetings:14
4 Point, Computation and Data (Old Structure), Elective, Fall 2024, Spring 2025
Last updated: March 12, 2024
Show Course Description

One of the most transformative consumer products in history, the iPhone remains the standard bearer for great design and user experience. With the latest versions of iOS and iPhone, Apple puts depth sensing and augmented reality in our pockets. How do we take advantage of this incredible platform to produce our own compelling experiences?

This course will be a hands-on workshop where we explore the world beyond generic apps and push the boundaries of what’s possible on iOS hardware. Each week, you’ll be asked to complete a programming exercise meant to foster your understanding of iOS application development. We’ll leverage existing open source libraries to quickly build out your app with features such as real time communication and cloud storage.

We aim to create distributed instruments for computed expression.

Full-time access to an iOS device and a Mac laptop computer running the latest operating system and development tools are required.

Prereq: Some programming experience (such as ICM) and willingness to learn Apple’s Swift programming language.

Prerequisite: Creative Computing (IMNY-UT 101)

Topics in Media Arts: AI for Creatives

Tyler Peppel | Syllabus | IMNY-UT.281 | TBD Meetings:7-Second Half
2 Point, Computation and Data (Old Structure), Elective, Fall 2024, Spring 2025
Last updated: March 12, 2024
Show Course Description

A survey and hands-on workshop on AI augmentation of creative communication.

We will get hands-on with AI tools for photography, design, art, writing, UX, illustration, and video.
We will produce, critique, and exhibit creative work that tests the creative capabilities of these new AI tools.

Questions we will consider:
How does AI influence the creative process?
How does this new human/machine collaboration challenge my role as a creator?
Who is creating these new tools and how do they work?
What is the longer-term impact of AI technologies on the creative professions?
Our goals are to understand AI technology, use it to produce creative work, and use what we learn to influence the larger conversation about creativity and AI.

Will AI diminish or enhance human creativity? It’s up to us.

Topics in Media Art: Shared Minds

Dan O'Sullivan | Syllabus | IMNY-UT.260 | Wed 12:20pm to 3:20pm in Meetings:14
4 Point, Computation and Data (Old Structure), Elective, Fall 2024, IMA Major Elective, New, Spring 2024, Spring 2025
Last updated: October 20, 2023
Show Course Description

What capabilities does computational media have for depicting and conveying the experience of our minds? In using the new possibilities of machine learning networks to create media, what should we take or leave from cinema, social media and virtual reality?

In this course we will start out by turning inward to reflect on how our mind transcends time and space and how artificial neural networks might better capture the multidimensional space of our thought. We then turn to using cloud networking and databases to share our thinking with other people across time and space. Finally we need to flatten everything back into 4D interfaces that, while being stuck in time and space, can reach our embodied, emotional and experiential ways of understanding of the world.

The class will operate at a conceptual level, inviting students’ empirical, psychological and philosophical investigations of the nature of their experience and how to convey it with art and story. It will ask students to look critically at existing computational media’s tendencies to bore, misinform, divide or inflame its users.

But this is also very much a coding class where students will prototype their own ideas for new forms of media first with machine learning models like Stable Diffusion using Huggingface APIs or Colab notebooks, and then with networking and databases using Firebase or P5 Live Media, and finally with 3D graphics using the threejs library. Students can substitute other coding tools but game engines will not work for this class. The coding is in javascript, with touches of python, and is a natural sequel to Creative Computing.

Topics in Computation and Data: Nature of Code

Lenin Compres | Syllabus | IMNY-UT.220 | Thur 5:20pm to 8:20pm in Meetings:14
4 Point, Computation and Data (Old Structure), Elective, Fall 2024, IMA Major Elective, New, Programming and Data (New Structure), Spring 2024, Spring 2025
Last updated: October 20, 2023
Show Course Description

Can we capture the unpredictable evolutionary and emergent properties of nature in software? Can understanding the mathematical principles behind our physical world help us to create digital worlds? This class focuses on the programming strategies and techniques behind computer simulations of natural systems. We explore topics ranging from basic mathematics and physics concepts to more advanced simulations of complex systems. Subjects covered include physics simulation, trigonometry, self-organization, genetic algorithms, and neural networks. Examples are demonstrated in JavaScript using p5.js.

Prerequisites: Creative Computing

Instructor Daniel Shiffman Website: https://natureofcode.com/

Topics in Media Art: Generative Art with the Unity Game Engine

Dave Stein | Syllabus | IMNY-UT.281 | Tues 10:40am to 12:10pm in > Thur 10:40am to 12:10pm in Meetings:7-Second Half
2 Point, Computation and Data (Old Structure), Elective, Fall 2024, IMA Major Elective, Media and Entertainment (Old Structure), New, Spring 2024, Spring 2025
Last updated: October 20, 2023
Show Course Description

This course will provide an overview of important topics of generative art. On a weekly basis we will cover a new topic, review examples of work within this topic and discuss their influence in generative artworks as well as in a broader art context.

In addition, we cover the fundamental concepts of the C# programming language and its application within the Unity game engine. C# is a widely used, very fast and efficient programming language and can perform significantly faster than P5 and Processing. As such, creating generative art projects using Unity and C# will make our projects faster with higher definition and larger detail than a typical Javascript sketch.

This course is designed for students who want to continue their creative coding practice and are interested in more advanced coding techniques while building their knowledge of C# and Unity. Students should have a solid understanding of programming concepts such as arrays, classes and objects and be comfortable with creative coding (such as with P5).