An introductory course designed to provide students with hands-on experience developing creative coding projects with machine learning. The history, theory, and application of machine learning algorithms and related datasets are explored in a laboratory context of experimentation and discussion. Examples and exercises will be demonstrated in JavaScript using the p5.js, ml5.js, and TensorFlow.js libraries. In addition, students will learn to work with open source pre-trained models in the cloud using Runway. Principles of data collection and ethics are introduced. Weekly assignments, team and independent projects, and project reports are required.
3D environments and objects are powerful prototyping tools. This class will introduce the basics of 3D modeling techniques in Rhino and students will learn to create assets for prototyping and 3D printing. The class will take an industrial design approach to design and build with specifications and materials in mind. Students will learn to think, plan, design, and produce well thought out objects to fit their specific needs. (examples: motor mounts, enclosures, wearables etc.)
Whether it’s through photo realistic scans found in current-gen video games or the cartoonish and low-fi aesthetic of Bitmoji there is no limit to ways in which the body and the self are represented in digital spaces. This 2-credit class will look at how avatars have been historically used in the realm of art, commerce, and entertainment and utilize existing avatar creation tools to develop projects that examine identity, body politics, and contemporary performance. In class, we will cover the basics of Unreal Engine, photogrammetry, 3D scanning, and model rigging although students will be encouraged to use existing skill sets and creative thinking to complete some of the smaller week-by-week assignments. The class will culminate with a short performance, small installation or single/multi-channel video piece using one or more of the techniques covered in class. This can be a solo project or a group project. In this class students will: – Explore how avatars can be utilized in your creative practice – Gain an introductory understanding of Unreal Engine, photogrammetry, model rigging, and 3D scanning. – Learn how to recontextualize digital spaces for the purposes of art, installation, and performance. – Broaden your thinking of what performance can be, both in a physical setting and digital setting. – Think critically about how physical bodies inhabit digital spaces and how the hardware and software we use reinforces the acceptance and value of certain kinds of bodies.
This course combines two powerful areas of technology that will enable you to leap from being just a user of technology to becoming a creator with it: Physical Computing and Programming. The course begins with Physical Computing, which allows you to break free from both the limitations of mouse, keyboard & monitor interfaces and stationary locations at home or the office. We begin by exploring the expressive capabilities of the human body and how we experience our physical environment. The platform for the class is a microcontroller (Arduino brand), a very small inexpensive single-chip computer that can be embedded anywhere and sense and make things happen in the physical world. The core technical concepts include digital, analog and serial input and output. The second portion of the course focuses on fundamentals of computer programming (variables, conditionals, iteration, functions & objects) as well as more advanced techniques such as data parsing, image processing, networking, computer vision. The Javascript ‘p5’ programming environment is the primary vehicle. P5 is more oriented towards visual displays on desktops, laptops, tablets or smartphones but can also connect back to the physical sensor & actuators from the first part of the class. The course is designed for computer programming novices but the project-centered pedagogy will allow more experienced programmers the opportunity to go further with their project ideas and collaborate with other students. What can computation add to human communication? You will gain a deeper understanding of the possibilities of computation–– possibilities that will augment and enhance the perspectives, abilities and knowledge you bring from your field of study (e.g. art, design, humanities, sciences, engineering). At first it may feel foreign, as foreign as learning a new language or way of thinking. But soon, once you get some basic skills under your belt, you’ll be able to make projects that reflect your own interests and passions.
Physical Computing is an approach to learning how humans communicate through computers that starts by considering how humans express themselves physically. In this course, we take the human body as a given, and attempt to design computing applications within the limits of its expression. To realize this goal, you’ll learn how a computer converts the changes in energy given off by our bodies (in the form of sound, light, motion, and other forms) into changing electronic signals that it can read and interpret. You’ll learn about the sensors that do this, and about simple computers called microcontrollers that read sensors and convert their output into data. In the other direction you will learn how to actual physical things in the world with devices like speakers, lights and motors. Finally, you’ll learn how microcontrollers communicate with other computers. To learn this, you’ll watch people and build devices. You will spend a lot of time building circuits, soldering, writing programs, building structures to hold sensors and controls, and figuring out how best to make all of these things relate to a person’s body.
How does someone become famous on the internet? What does it take to capture our digital attention? While movie stars, rock gods, and other mainstream A-listers struggle to find their place in a sea of emerging technologies and platforms, a new swarm of micro celebrities and influencers has coasted into the cultural space they once filled. Riding a wave of viral content and memes, the newly-famous rule an internet where anyone can have adoring fans… for a price. They are nimble, niche, obnoxious, empowering, and sometimes disturbing. This class explores what happens when fame is freed from the traditional intermediaries of print, television, and radio, when social media provides everyone with the tools to be their own marketing studio and PR department. It examines the transformation of celebrity, from a 19th century sales gimmick to the formidable cultural, social, and technological force it is today. Students will study a wide array of fame-related topics, from the privacy effects of trolling to the class implications of selfies. And we will engage in practices and exercises that produce real-world instances of celebrity in case we, too, wish to join the ranks of the internet famous.
An introductory course designed to provide students with hands-on experience using various technologies including time based media, video production, digital imaging, audio, video and animation. The forms and uses of new communications technologies are explored in a laboratory context of experimentation and discussion. The technologies are examined as tools that can be employed in a variety of situations and experiences. Principles of interpersonal communications, media theory, and human factors are introduced. Weekly assignments, team and independent projects, and project reports are required.
This course will explore the fundamentals of new media scholarship. Together, we will review and engage with different theories of emerging media in its social, cultural, political, and historical contexts. Students will be able to research, think and write critically about some of the central debates in media studies, including new media forms and aesthetics, issues of gender, race, and labor, platforms, infrastructure and various emerging paradigms. Classes consist of theoretical readings, media example discussion, and writing workshops. Prerequisite: WAI (or co-requisite). Fulfillment: IMA Major Foundations/Elective; IMB Major Emerging Media Foundation/Elective.
Re-make: make (something) again or differently. In this class students will investigate why China became the world’s largest importer of waste. They will study local communities in China, how they manage their waste, and explore innovative ways to transform discarded materials or products around us into something new and precious in areas such as art, graphic and industrial design, architecture, fashion, textiles, etc,. Through research and development, students will learn how traditional techniques and new technologies among the sustainable design philosophy can be utilized as powerful tools for addressing social and environmental problems.
Machine Learning for New Interfaces is an introductory course with the goal of teaching machine learning concepts in an approachable way to students with no prior knowledge. We will explore diverse and experimental methods in Machine Learning such as classification, recognition, movement prediction and image style translation. By the end of the course, students will be able to create their own interfaces or applications for the web. They will be able to apply fundamental concepts of Machine Learning, recognize Machine Learning models in the world and make Machine Learning projects applicable to everyday life. Prerequisite: Creative Coding Lab or equivalent programming experience Fulfillment: IMA/IMB elective.
Data is at the heart of the increasing role technology has in our lives. Data collection and algorithmic processing are not only central to recent technical breakthroughs such as in AI and automation but have created new economic paradigms where data equals value and shape political approaches to power and control. Decisions based on algorithms affect society at large whether it’s changing the way we transport and distribute goods, or influencing the things we buy, the news we read or even the people we date. The *world* that algorithms *see* is data. For the average person, however, data is seldom more than an abstract idea. So what exactly is data? How is value extracted from it? And why should we care? How can we ethically balance the positive uses of data-driven systems with the threats they pose to discriminate and infringe basic human rights? This class seeks to untangle some of these issues practically and theoretically. Each week will include a lecture introducing contemporary theorists, artists, groups, and in-class discussions or exercises. Potentially there will be a guest speaker, too. Topic sections may include surveillance and privacy, data journalism and activism or automation and machine bias. What we cover will be complemented by reading and research assignments. The other half of the week is a programming lab in which you will learn the fundamentals of web-based data visualization using JavaScript. Programming assignments will allow you to further practice what we learn. Throughout the semester, you will work on three main visualization projects that are inspired by the theoretical subjects that we cover. The form of these projects will usually be a website. Successful projects feature data visualizations that are both playful as well as effective in conveying information and a reflection that links the practical work to the theoretical learnings. Prerequisite: Interaction Lab, Communications Lab or Application Lab
Artificial Intelligence Arts is an intermediate class that broadly explores issues in the applications of AI to arts and creativity. This class looks at generative Machine Learning algorithms for creation of new media, arts and design. In addition to covering the technical advances, the class also addresses the ethical concerns ranging from the use of data set, the necessarily of AI generative capacity to our proper attitudes towards AI aesthetics and creativity. Students will apply a practical and conceptual understanding of AI both as technology and artistic medium to their creative practices.
This interdisciplinary project-based class focuses on the design, development, and use of technology that increases the quality of life of individuals of disabilities. Students will be introduced to various assistive technology and strategies, including no-tech and low-tech as well as software and online-based practices. This class features lectures, discussions, guest lectures, field trips, and project presentations by students. Software programming, physical computing, machine learning, and 3D fabrication will be introduced for developing an assistive device. Field trips of local facilities will be scheduled during the semester. They provide an off-campus real-world learning experience as well as an opportunity for students to interact with users of assistive technology in the local community. Students will participate in a team-based design project that identifies challenges for an individual of disabilities and create an innovative and useful assistive device to meet their needs. Prerequisite: Interaction Lab
From the history of visual music and abstract film to the contemporary notion of live cinema, this course will be an exploration of the synesthetic relationship between sound and visuals in a realtime performance setting. Dating back as far as the 18th century, systems have been invented to produce images alongside music linking the two through formalized arrangements. Current media technologies make developing such systems both more approachable and more expansive in their scope. Through readings, viewings, and case studies students will gain an understanding of the history and theory of live audiovisuals. During the course students will team up to develop and master a realtime audiovisual system of their own invention. The class will culminate in a show in which they will present their work through a live performance. Prerequisite: None.
This course investigates and explores the integration between cultural heritage and digital conservation, specifically towards the objects, deities, sites, and gardens of China. Through the reflection of Chinese cultural heritage from the global perspective of Chinese history, the course raises awareness of heritage conservation and critical heritage studies towards the origins and the transformations of China in the contemporary era. Academic readings and oral presentations revolving around this theme will enhance students’ skills for documentation and restoration by innovative digital techniques in China. Conservator presentations, field trips, gallery visits and art projects enable students to communicate with practitioners in the field and examine the values of Chinese cultural heritage from a global perspective. Prerequisites: None Fulfillment: IMA/IMB elective; GCS Chinese Media, Arts, and Literature.
This class examines the idea and practices of ‘cultivation’ in relation to the challenging environment of the 21st century city. Through field trips, readings and discussions, the class explores the concept of cultivation, and how it can be used as a basis for researching the urban ecology of Shanghai, both as a past and future city. The class incorporates a major project in the digital humanities, in which students use the tools of interactive media (audio, video and cartographic technologies) to research, map and narrativize the ways in which architects, designers, artists and thinkers engage with the traditions of cultivation in order to imagine and recreate the future metropolis. Prerequisite: Writing as Inquiry
We live in a world immersed in sound yet we rarely attend to how sound can reflect our social structure or reveal cultural meaning. This course introduces students to acoustic ethnography, soundscape studies and narrative, non-narrative audio storytelling. We will gather and analyze the acoustic environment of China, using recorded sounds to create ethnography through text and sound. Ethnography (literally, “culture-writing”) is both the act of gathering data about culture through observation and interviews as well as the practice of writing analytically about cultural difference. Visual ethnography incorporates the analysis of visual and material aspects of our social environment into creative, multimedia rich projects. With an ethnographic approach to sound, we will document the rich tapestry of sounds around us in the context of the Yangtze River Delta region and think about how this conveys China’s culture, society and history. Through lectures, discussion, readings, listening assignments, field studies and projects, we will re-learn how to listen, observe and record the sounds in our environment. We will study Chinese sound art and Chinese cultural productions in music, film, television and multimedia installation. We will contextualize Chinese sound art against major theoretical approaches to sound including archives and preservation, form versus content, and social studies of science. Students will work collaboratively or individually on a final project that combines sound recording and production, to create an ethnographic analysis of an aspect of social and cultural life in the Yangtze River Delta region. Students will gain experience in gathering ethnographic data and they will transform it into an analytical or creative project integrating sound art and text. Prior knowledge of sound editing and Chinese language is not required. Prerequisite: None
Since the beginning of civilization people have fantasized about intelligent machines sensing and acting autonomously. In this course we will discover what robots are, learn how to design them, and use simple tools to build them. Students will use open source hardware to explore sensors and electronics, as well as design and build robot bodies and actuators through a variety of digital fabrication technologies. Using a set of community developed tools, students will become familiar with concepts such as mechatronics, inverse kinematics, domotics and machine learning. No previous programming or electronics experience is necessary, however students will be guided through a series of design challenges that their robots should be able to accomplish. With an emphasis on experimentation, peer learning, and teamwork, the objective of this course is to share in the excitement of robotics by enabling students to make their own creations. By the end of the course, students will present a short research paper and documentation about their robotic explorations. Co-requisite or Prerequisite: Interaction Lab or Creative Coding Lab. Fulfillment: CORE ED; IMA Majors Electives; IMB Major Interactive Media Elective.
Programming Design Systems is a course focused on the intersection between graphic design and code. Class time is divided between design topics like form, color, grid systems, and typography, and more computational topics like randomization, repetition, transformation and generative form. The students work to write software that abstract design theories into the code, and show the work in class for design critique. Weekly readings include relevant writings from the history of graphic design, articles from the history of computation, and everything in between. The class aims not only to teach the students how to create designs via code, but also to have something interesting to say about it. The course is based on the Programming Design Systems book, and more background info can be found in the book’s introduction. Prerequisite:Communications Lab