Computation and Data (Old Structure)

Recursive Art (Topics in Media Art) +

Shuang Cai | Syllabus | IMNY-UT.281 |
2 Point, All, Art and Design (Old Structure), Computation and Data (Old Structure), Design and Fabrication (New Structure), Elective, Evening, Monday, New, Spring 2025, Wednesday, YES

Last updated: November 15, 2024

This course description is a concise summary and embodiment of the content, objectives, and structure of an educational course. The creation of this course description was initiated from the need to compile the information for the instructors, students, and administration staff. By writing this course description, the instructor is practicing one type of recursive creation process where the creation is a direct representation of the process of creation. This course description also aims to demonstrate how recursion isn’t confined to programming alone; it’s a fundamental idea that can be found in diverse fields, including this very course creation process.

In this course, you will look at theories around recursive philosophy, recursion as an algorithm, recursion as a programming philosophy (function-oriented programming), and various recursive works. You will think really hard alone and learn to communicate your thinking with others. You will do and make puzzles and reflect the method in your own creations through a series of small experiments.

Why recursion:

it challenges traditional problem-solving approaches. In situations where there are numerous methods to achieve comparable outcomes, while thinking about ethical practice within any creative endeavor, the course aims to delve into the underlying meaning and purpose of the work undertaken. Recursion, in this context, embodies a form of process-oriented thinking. By exploring recursive processes, you are encouraged to question established norms, fostering a deeper understanding of the motivations driving their creative endeavors.

Immersivity Beyond the Display +

MK Skitka | Syllabus | IMNY-UT.281 |
2 Point, All, Computation and Data (Old Structure), Elective, Evening, Monday, New, Spring 2025, Wednesday, YES

Last updated: November 15, 2024

In recent years, “immersive digital experiences” have gained widespread popularity, from commercial exhibits like Van Gogh: The Immersive Experience to museums adapting spaces for digital art and brands leveraging interactive installations for promotion. But are these experiences truly immersive, and how can we evaluate them?

Immersivity Beyond the Display delves into the design of visual, interactive installations. Each week, students will modify existing sketches in p5.js to explore immersive design principles while building on foundational programming skills. “Demo days” will provide opportunities to test and reflect on each other’s prototypes. We will focus on narrative development, embodiment, social and emotional engagement and interaction mechanics to frame our exploration. Weekly exercises will be enriched with relevant readings and discussions. By the end of the class, students will have developed their own model for evaluating immersivity in interactive digital experiences.

Prerequisite: Students should have completed Creative Computing or possess equivalent coding experience with p5.js and JavaScript.

The Code of Music +

Luisa Pereira | Syllabus | IMNY-UT.222 |
4 Point, All, Computation and Data (Old Structure), Elective, In-Person, Midday, Monday, New York, Programming and Data (New Structure), Spring 2025, YES

Last updated: November 15, 2024

In this course, students learn how to create musical systems –pieces that incorporate randomness, interact with their listeners, or evolve over time, in the browser.

We will start by creating audiovisual instruments and sample-based interactive songs, as students review their p5.js skills and are introduced to the Tone.js music library. Then, we will turn to a structured exploration of the elements of music, focusing on rhythm, melody, timbre, and harmony. For each, we will hold listening sessions, represent and manipulate the element in code, and interact with it via a range of existing interfaces. Students will explore the possibilities that computation and interactivity open up by designing and implementing a series of interactive studies.

The last few weeks of the semester will be dedicated to introducing algorithmic composition techniques such as Markov Chains and Neural Networks. During this time, students will also develop their final project: an interactive/generative musical piece that builds on their previous classwork.

Throughout the course, students are encouraged to bring in their musical tastes and interests into the classroom. This class is a good fit for students who are interested in:
– Creating interactive music pieces and digital instruments.
– Deepening their understanding of how music works. All musically-curious students are welcome: previous experience with music and audio will be useful, but is not required.
– Continuing to develop coding skills. Creative Coding or equivalent programming experience is required.

About Luisa Hors: https://www.luisapereira.net/

Prerequisite: Creative Computing (IMNY-UT 101)

Networked Media +

Sam Heckle | IMNY-UT.223 |
4 Point, All, Computation and Data (Old Structure), Elective, In-Person, Midday, New York, Programming and Data (New Structure), Spring 2025, Thursday, Tuesday, YES

Last updated: November 15, 2024

The network is a fundamental medium for interactivity. It makes possible our interaction with machines, data, and, most importantly, other people. Though the base interaction it supports is simple, a client sends a request to a server, which replies; an incredible variety of systems can be and have been built on top of it. An equally impressive body of media theory has also arisen around its use.

This hybrid theory and technology course will be 50% project driven technical work and 50% theory and discussion. The technical work will utilize JavaScript as both a client and server side programming language to build creative systems on the web. Technical topics will include server and client web frameworks, such as Express, HTML, CSS, templating, and databases. The theory portion of the course will include reading and discussion of past and current media theory texts that relate to the networks of today.

**** it is HIGHLY recommended you take Front End Web Development (or have equivalent front end web development experience) to get the most out of this course. We will be going over fundamentals of HTML/CSS but it would be useful to have prior knowledge ***

After Creative Coding (Topics in Media Art) +

MK Skitka | Syllabus | IMNY-UT.281 |
2 Point, Computation and Data (Old Structure), Elective, Fall 2024, New, Spring 2025, YES

Last updated: November 5, 2024

This seven-week course challenges students to extend their knowledge of creative computing into the exciting realm of embodied interfaces. Building upon the fundamentals taught in Creative Computing, this course will explore the design and implementation of interactive systems that integrate the body as a controller of technology.

Going beyond the limitations of traditional mouse, keyboard, and screen interactions, we will investigate how movement, gesture, and physical interaction can be used to create engaging and meaningful user experiences. Through a combination of lectures and hands-on workshops, students will gain a deeper understanding of:

Sensor Technologies and Physical Computing: Working with sensors (e.g., accelerometers, gyroscopes) to capture and interpret human movement and environmental data.

Movement Analysis and Interpretation: Techniques for analyzing movement patterns, extracting meaningful features, and mapping them to computational processes.

Performance and Embodiment in Technology: Investigating how technology can facilitate new forms of expression and performance.

The course will culminate in one proof of concept project utilizing one of the techniques explored in class.

Prerequisites: Completion of Creative Computing or equivalent experience with Arduino and Javascript p5.

Topics in Media Art: Recursive Art +

Shuang Cai | Syllabus | IMNY-UT.281 |
2 Point, Art and Design (Old Structure), Computation and Data (Old Structure), Design and Fabrication (New Structure), Elective, Fall 2024, New, Spring 2025, YES

Last updated: October 11, 2024

This course description is a concise summary and embodiment of the content, objectives, and structure of an educational course. The creation of this course description was initiated from the need to compile the information for the instructors, students, and administration staff. By writing this course description, the instructor is practicing one type of recursive creation process where the creation is a direct representation of the process of creation. This course description also aims to demonstrate how recursion isn’t confined to programming alone; it’s a fundamental idea that can be found in diverse fields, including this very course creation process.

In this course, you will look at theories around recursive philosophy, recursion as an algorithm, recursion as a programming philosophy (function-oriented programming), and various recursive works. You will think really hard alone and learn to communicate your thinking with others. You will do and make puzzles and reflect the method in your own creations through a series of small experiments.

Why recursion:

it challenges traditional problem-solving approaches. In situations where there are numerous methods to achieve comparable outcomes, while thinking about ethical practice within any creative endeavor, the course aims to delve into the underlying meaning and purpose of the work undertaken. Recursion, in this context, embodies a form of process-oriented thinking. By exploring recursive processes, you are encouraged to question established norms, fostering a deeper understanding of the motivations driving their creative endeavors.

Topics in Media Art: After Creative Coding +

MK Skitka | Syllabus | IMNY-UT.281 |
2 Point, Computation and Data (Old Structure), Elective, Fall 2024, New, Spring 2025, YES

Last updated: October 11, 2024

This seven-week course challenges students to extend their knowledge of creative computing into the exciting realm of embodied interfaces. Building upon the fundamentals taught in Creative Computing, this course will explore the design and implementation of interactive systems that integrate the body as a controller of technology.

Going beyond the limitations of traditional mouse, keyboard, and screen interactions, we will investigate how movement, gesture, and physical interaction can be used to create engaging and meaningful user experiences. Through a combination of lectures and hands-on workshops, students will gain a deeper understanding of:

Sensor Technologies and Physical Computing: Working with sensors (e.g., accelerometers, gyroscopes) to capture and interpret human movement and environmental data.

Movement Analysis and Interpretation: Techniques for analyzing movement patterns, extracting meaningful features, and mapping them to computational processes.

Performance and Embodiment in Technology: Investigating how technology can facilitate new forms of expression and performance.

The course will culminate in one proof of concept project utilizing one of the techniques explored in class.

Prerequisites: Completion of Creative Computing or equivalent experience with Arduino and Javascript p5.

Real-Time Media +

Carrie Sijia Wang | Syllabus | IMNY-UT.285 |
4 Point, 9.75 Ready to Schedule (Do not edit) YES, Art and Design (Old Structure), Computation and Data (Old Structure), Elective, Fall 2024, Media and Entertainment (Old Structure)

Last updated: March 12, 2024

This course focuses on designing, developing and delivering real-time, performative work using audio and video elements. The class will have an emphasis on using MaxMSPJitter and other tools to create performative experiences that dynamically combine interactive elements such as video, sound, and code, allow for the unfolding of engaging narratives, and generate compelling visuals in real time.

We will look at various examples of both multimedia performances and installations, explore how we can apply the technologies we have learned to design real-time systems, and discuss methods we can use to make our work more engaging.

The class is three-fold and divided into tech tutorials, discussions of existing examples, and in-class performances.

Introduction to Machine Learning for the Arts +

Daniel Shiffman | IMNY-UT.224 |
4 Point, 9.75 Ready to Schedule (Do not edit) YES, Computation and Data (Old Structure), Elective, Fall 2024, Programming and Data (New Structure)

Last updated: March 12, 2024

Prerequisite: Creative Computing (IMNY-UT 101) OR equivalent coursework.

An introductory course designed to provide students with hands-on experience developing creative coding projects with machine learning. The history, theory, and application of machine learning algorithms and related datasets are explored in a laboratory context of experimentation and discussion. Examples and exercises will be demonstrated in JavaScript using the p5.js, ml5.js, and TensorFlow.js libraries. In addition, students will learn to work with open source pre-trained models in the cloud using Runway. Principles of data collection and ethics are introduced. Weekly assignments, team and independent projects, and project reports are required.

Front-End Web +

Lenin Compres | IMNY-UT.228 |
4 Point, 9.75 Ready to Schedule (Do not edit) YES, Computation and Data (Old Structure), Elective, Fall 2024

Last updated: March 12, 2024

This course will provide a foundation for understanding modern web development with a focus on front end technologies and accessing public data. The forms and uses of these technologies are explored in a laboratory context of experimentation and discussion. This studio stresses interactivity, usability, and the quality and appropriateness of look and feel.
Students will create two web applications, including one that leverages public APIs and Javascript libraries. The goal of the course is for students to learn how to think holistically about an application, both by designing a clear user experience and understanding the algorithmic steps required to build it. Assignments are arranged in sequence to enable the production of a website of high quality in design and engineering.

Topics in Media Arts: AI for Creatives +

Tyler Peppel | Syllabus | IMNY-UT.281 |
2 Point, 9.75 Ready to Schedule (Do not edit) YES, Computation and Data (Old Structure), Elective, Fall 2024

Last updated: March 12, 2024

A survey and hands-on workshop on AI augmentation of creative communication.

We will get hands-on with AI tools for photography, design, art, writing, UX, illustration, and video.
We will produce, critique, and exhibit creative work that tests the creative capabilities of these new AI tools.

Questions we will consider:
How does AI influence the creative process?
How does this new human/machine collaboration challenge my role as a creator?
Who is creating these new tools and how do they work?
What is the longer-term impact of AI technologies on the creative professions?
Our goals are to understand AI technology, use it to produce creative work, and use what we learn to influence the larger conversation about creativity and AI.

Will AI diminish or enhance human creativity? It’s up to us.

Topics in Computation and Data: Mobile Application Development +

John Henry Thompson | Syllabus | IMNY-UT.220 |
4 Point, 9.75 Ready to Schedule (Do not edit) YES, Computation and Data (Old Structure), Elective, Fall 2024

Last updated: March 12, 2024

One of the most transformative consumer products in history, the iPhone remains the standard bearer for great design and user experience. With the latest versions of iOS and iPhone, Apple puts depth sensing and augmented reality in our pockets. How do we take advantage of this incredible platform to produce our own compelling experiences?

This course will be a hands-on workshop where we explore the world beyond generic apps and push the boundaries of what’s possible on iOS hardware. Each week, you’ll be asked to complete a programming exercise meant to foster your understanding of iOS application development. We’ll leverage existing open source libraries to quickly build out your app with features such as real time communication and cloud storage.

We aim to create distributed instruments for computed expression.

Full-time access to an iOS device and a Mac laptop computer running the latest operating system and development tools are required.

Prereq: Some programming experience (such as ICM) and willingness to learn Apple’s Swift programming language.

Prerequisite: Creative Computing (IMNY-UT 101)

Topics in Media Art: Shared Minds +

Dan O'Sullivan | Syllabus | IMNY-UT.260 |
4 Point, 9.9 YES, Computation and Data (Old Structure), Elective, Fall 2024, IMA Major Elective, New, Spring 2024

Last updated: October 20, 2023

What capabilities does computational media have for depicting and conveying the experience of our minds? In using the new possibilities of machine learning networks to create media, what should we take or leave from cinema, social media and virtual reality?

In this course we will start out by turning inward to reflect on how our mind transcends time and space and how artificial neural networks might better capture the multidimensional space of our thought. We then turn to using cloud networking and databases to share our thinking with other people across time and space. Finally we need to flatten everything back into 4D interfaces that, while being stuck in time and space, can reach our embodied, emotional and experiential ways of understanding of the world.

The class will operate at a conceptual level, inviting students’ empirical, psychological and philosophical investigations of the nature of their experience and how to convey it with art and story. It will ask students to look critically at existing computational media’s tendencies to bore, misinform, divide or inflame its users.

But this is also very much a coding class where students will prototype their own ideas for new forms of media first with machine learning models like Stable Diffusion using Huggingface APIs or Colab notebooks, and then with networking and databases using Firebase or P5 Live Media, and finally with 3D graphics using the threejs library. Students can substitute other coding tools but game engines will not work for this class. The coding is in javascript, with touches of python, and is a natural sequel to Creative Computing.

Topics in Computation and Data: Nature of Code +

Lenin Compres | Syllabus | IMNY-UT.220 |
4 Point, 9.9 YES, Computation and Data (Old Structure), Elective, Fall 2024, IMA Major Elective, New, Programming and Data (New Structure), Spring 2024

Last updated: October 20, 2023

Can we capture the unpredictable evolutionary and emergent properties of nature in software? Can understanding the mathematical principles behind our physical world help us to create digital worlds? This class focuses on the programming strategies and techniques behind computer simulations of natural systems. We explore topics ranging from basic mathematics and physics concepts to more advanced simulations of complex systems. Subjects covered include physics simulation, trigonometry, self-organization, genetic algorithms, and neural networks. Examples are demonstrated in JavaScript using p5.js.

Prerequisites: Creative Computing

Instructor Daniel Shiffman Website: https://natureofcode.com/

Topics in Media Art: Three.js for Makers +

Brian Ho | Syllabus | IMNY-UT.281 |
2 Point, 9.9 YES, Computation and Data (Old Structure), Elective, Fall 2024, IMA Major Elective, New, Spring 2024

Last updated: October 20, 2023

“In this increasingly online world, the internet has proven to be a powerful tool that can connect us with one another, host meaningful experiences, and provoke critical thinking. In this class, students will have an opportunity to learn about breaking out of the 2D web page and the fundamentals of working with 3D on the web.

This course hopes to introduce new avenues for creative expression and experimentation via the web and promote learning practical web development skills through experiential learning. Students will use Three.js to create dynamic and immersive web-based experiences that push the boundaries of what is possible online.

The course is intended for technologists who have no programming or computer science background but are interested in 3D exploration on the web. Nothing more than a basic understanding and familiarity with CSS, HTML, and Javascript is required.”

Topics in Media Art: Generative Art with the Unity Game Engine +

Dave Stein | Syllabus | IMNY-UT.281 |
2 Point, 9.9 YES, Computation and Data (Old Structure), Elective, Fall 2024, IMA Major Elective, Media and Entertainment (Old Structure), New, Spring 2024

Last updated: October 20, 2023

This course will provide an overview of important topics of generative art. On a weekly basis we will cover a new topic, review examples of work within this topic and discuss their influence in generative artworks as well as in a broader art context.

In addition, we cover the fundamental concepts of the C# programming language and its application within the Unity game engine. C# is a widely used, very fast and efficient programming language and can perform significantly faster than P5 and Processing. As such, creating generative art projects using Unity and C# will make our projects faster with higher definition and larger detail than a typical Javascript sketch.

This course is designed for students who want to continue their creative coding practice and are interested in more advanced coding techniques while building their knowledge of C# and Unity. Students should have a solid understanding of programming concepts such as arrays, classes and objects and be comfortable with creative coding (such as with P5).