What is the relationship between American musician and poet, Gill Scott-Heron and cybernetics? Heron’s “The Revolution Will Not be Televised” was created in hopes to wake-up 1970’s America from complicity in societal oppression to then realize the revolution begins in your mind and something created through active participation, not passive media consumption. While, iterated forms of cybernetics also challenge the notion that “technology [or society] is self-correcting”, through fostering design thinking and systems theory through a meta-scale analysis of computational practices, essentially promoting being an observer of systems through conversation with process and feedback loops. Both ideologies center intention and holistic design within innovation — concepts vital to the equitable well-being of society in a technocratic era. In this course, students will explore the origins and history of the internet and HCI technologies as stemming from hippie communes, counterculture and warfare; survey historical and current social relations in the U.S.; study how these social relations have been replicated and embedded into everyday technologies; and consider potential futures in tandem with rapid technological advancements. We will study the works of pioneering designers and thought-leaders from Norbert Wiener, Stewart Brand, and Victor Papanek to Alondra Nelson and Ruha Benjamin. Phenomena discussed throughout the course range from the power of Black Twitter, the social media revolution of Egypt’s Tahrir Square, to cryptocurrency being used in bail reform. After in-depth historical analysis of societal structures, geo-political dissent, and algorithmic patterns, students will engage with futurism and speculative practices, techno-culture manifestos, and design and systems theory to develop a variety of research-driven projects. We will conclude with using both second-wave cybernetics as a design framework, and an anthropological lens to imagine, innovate and design more equitable technocratic futures. Topics heavily examined throughout the course include: surveillance, race, socioeconomics, carceral capitalism and technoscience, algorithmic bias and oppression, and post-work societal structures.
As demographics in consumer markets and the global labor forces shift rapidly, diverse, equitable and inclusive (DEI) designers are necessary in all facets of business, from product and service design, to organizational and business design. In this multimedia and interactive Intro to Design for Diversity™(D4D), students will be provided with critical thinking skills to begin viewing diversity, equity and inclusion as design processes necessary for the future wellbeing of humans. D4D is a design framework that marries design thinking with diversity, equity and inclusion best practices and frameworks to illuminate cultural and racial biases. D4D is not only a means to harm reduction, but also a way-finding tool for better business. Students will be provided with historical context and case studies of racially biased practices that are now prevalent in biased algorithms in tech, as well as the lineage of culturally biased media narratives and how that plays a central part in today’s design practices, as well as their negative impacts. Students will be provided with a shared language and the outline of the D4D framework’s 5 Key Areas to enable them to design away from dominant culture, and instead, design for all people.
Tangible interfaces are interfaces that you touch. You control them with your hands, feet, and other body parts. Their shape, feel, and arrangement provide feedback. This is where interaction design meets industrial design. In this class, you’ll design, program, and build devices with tangible controls in order to better understand how humans understand and control technical systems through our sense of touch. We’ll discuss physical interaction concepts such as expressive interfaces and utilitarian ones, real-time control vs. delayed control, and implicit vs. explicit interactions. You’ll learn programming and electronic techniques to sense state change, thresholds, peaks, and other signs of user action. You’ll also learn how to design, shop for, and construct housings for the devices you build. On the electronics side, the primary tools will be the microcontroller and common tangible controls: pushbuttons, switches, rotary encoders, rotary and slide potentiometers, force sensors and touch sensors. The class will also cover on-device feedback through LEDs, speakers, and force-feedback actuators. On the fabrication side, you’ll work with the tools of the shop and XXX CAD program. You’ll design and build four projects in the course of the semester. Projects will be designed (and parts specified). Projects will build on the skills learned in Intro to Physical Computing and Intro to Fabrication. Prerequisites: Intro to Physical Computing and Intro to Computational Media, or a working knowledge of microcontroller programming in Arduino; Intro to Fabrication or basic knowledge of laser cutter.
In this course, we’ll investigate two closely related ideas: public data and data publics. We’ll learn how to access and represent data from an assortment of existing public data sources, how to liberate currently obfuscated data sets, and how to create our own useful/whimsical/critical APIs from scratch. We’ll also investigate the act of putting data into public space – through sculpture, projections, performance, and participatory interventions. Particular attention will be paid to methods which bring data back to communities from which it was collected, and to tactics which build grassroots literacies, respect local ways of knowing and reinforce capabilities for stewardship and sovereignty.
Inspired by the Japanese art of Chindōgu, this class will introduce a playful and whimsical approach to learn industrial design. In this 14-week studio format class, students will develop gadgets, inventions, and electronic devices that present absurd solutions to problems, while learning concepts and techniques of design ideation, prototyping, model making, CMF (color, material, and finishes), and manufacturing. This is a production heavy four-credit course, where students will learn about industrial design and tangible interactions. Prerequisite: Intro to Phys. Comp. (ITPG-GT 2301)
This class will introduce the basic skills and resources required to pursue a career as an artist. Students will learn the day to day tasks of working artists, such as writing critically about their own work, drafting grant proposals, and planning the business administration of their studios. They will also learn how to balance commercial and experimental projects, collaboration and community work, and teaching and studio practice. They will engage in the critical text about ethical dilemmas of working with art institutions, corporations and academia. By the end of class, students will write a personal statement, update an art portfolio, and apply for an artist residency. While the class will focus on contemporary art and artists, students who are interested in other disciplines may benefit from learning about entrepreneurship, authorship and cultural industry.
There is a substantial gap between what AR/VR is capable of today and the future that enthusiasts envision. The hardware is progressing, yet there are lack of design tools and methodologies. Effective augmented and virtual reality games and experience require good storytelling, animation, production and solid graphics. Students will learn a myriad of processes including spatial interface design, volumetric capture, working with spatial audio, porting animations and game programming. This course presumes no prior knowledge and is intended to jump start a career in AR/VR development and interaction design. The goal is to have relevant portfolio projects for entering industry. Students will create content in Unity 3D to agnostically deploy on headsets, mobile phones or tablets. Students will have access to a wide range of available hardware. In the beginning of the course, students will learn to implement the most important AR/VR interaction concepts – concepts such as selection, manipulation, travel, wayfinding, menus and inputting text in Unity3D. In the latter part of the course, students will implement prototypes that will be periodically critiqued by experts at Oculus, Magic Leap, Google, and other top companies. *You do not need to be a programmer. This is intended for students interested in 3D art, interaction design, sound design and game development. All are important for immersive development*
Generative machine learning models open new possibilities for creating images, videos, and text. This class explores the idea of how artists, designers and creators can use machine learning in their own design process. The goal of this class is to learn and understand some common machine learning techniques and use them to generate creative outputs. Students will learn to use pre-trained models, and train their own models in the cloud using Runway. For each week, we will discuss the history, theory, datasets, application of the machine learning models, and build experiments based on the model. In addition to Runway, we will be using JavaScript libraries like the p5.js, ml5.js, and TensorFlow.js, and software like Photoshop, Unity and Figma. Students are expected to have taken ICM (Introduction to Computational Media), or have equivalent programming experience with Python or JavaScript. A list of ML models we will be covering: Image generation: StylanGAN: https://github.com/NVlabs/stylegan BigGAN: https://github.com/ajbrock/BigGAN-PyTorch Style Transfer Fast-style-transfer: https://github.com/lengstrom/fast-style-transfer Arbitrary-Image-Stylization: https://github.com/tensorflow/magenta/tree/master/magenta/models/arbitrary_image_stylization Semantic Image Segmentation/Synthesis Deeplab: https://github.com/tensorflow/models/tree/master/research/deeplab Sapde-coco: https://github.com/NVlabs/SPADE Image-to-Image Translation: pix2pix: https://phillipi.github.io/pix2pix/ pix2pixHD: https://github.com/NVIDIA/pix2pixHD Text Generation LSTM gpt-2: https://github.com/openai/gpt-2
With Machine Learning models are getting smaller, and microcontrollers are getting more computing power, Machine Learning is moving towards edge devices. This class explores the idea of how machine learning algorithms can be used on microcontrollers along with sensor data to build Physical Computing projects. In this class, we will learn about TensorFlow Lite, a library that allows you to run machine learning algorithms on microcontrollers. We will talk about common machine learning algorithms and techniques and apply them to build hands-on interactive projects that enrich our daily lives. Students will learn to use pre-trained models, and re-train the models with sensor data. We are going to talk about Image Classification, Transfer Learning, Gesture and Speech Detection. For each topic, we will first discuss its history, theory, datasets, and applications, and then build simple experiments based on the topic. Prospective students are expected to have taken Introduction to Physical Computing and Introduction to Computational Media course, or have equivalent programming experience with Arduino and JavaScript.
The ways in which we communicate has changed radically in the last 100 years. As the communication systems we use have increased in complexity, so has the effort it takes to understand how they work. Most of us use protocols like LTE, HTTP, TCP/IP, and BLE every day. We take them for granted, almost like we do the laws of nature. But there are more than the laws of physics, more than techniques of engineering, embedded in the design and implementation of our protocols of communication. To understand their role in our lives, we need to look into the societal and economic contexts in which they came to be. In this class, we will examine communication protocols using Raspberry Pi’s, Arduinos, Software Defined Radios, and other connected devices. We will look closer at organizations like iSOC, ICANN and IEEE to better understand how protocol designs are implemented and standardized. Through readings, research and hands-on work we will build an understanding of how these protocols work, how their designs incorporate the physical, technical, cultural, corporate and political assumptions of the actors behind them. In the first half of the class readings and assignments will help familiarize students with some of the different protocols we rely on every day. In the second half, students will work in groups to investigate a communications protocol and consider its impact from a technical, societal and environmental perspective. Final projects will communicate their findings in whatever form students deem appropriate – explanatory blog posts, physical or digital installations, or even videos and podcasts.
Is augmented reality technology about to enter the mainstream? AR platforms have finally become widely accessible to artists, designers, and technologists thanks to recent advances in mobile performance and a new collection of powerful computer vision techniques. As such, the medium offers rich possibilities for experimentation and a chance to rethink how we experience the intersection of the physical and digital. In this course, students will acquire an understanding of basic concepts and techniques necessary to prototype and build simple AR experiences – with a consideration of not just visual but also aural AR. We’ll supplement practical exercises with an overview of the history of AR, and discuss the ethical, legal, and societal considerations cropping up around this topic. Our tool of choice will be Unity, but we will go over prototyping techniques outside of the platform to speed up the design process. If there is interest, we will cover how to get started building projects in openFrameworks, mobile, or web AR – and discuss why or when you might want to work within other platforms. Even though code samples will be provided, students are highly encouraged to have a basic understanding of Unity or at least have taken an introductory programming course. A working knowledge of Unity can be gained through Unity tutorials (https://unity3d.com/learn/tutorials) or Lynda (https://www.nyu.edu/lynda).
Introduction to Critical Mapping and Open Source Geospatial Web Analysis and Visualization is an introduction to critical perspectives in cartography and geospatial information systems and web technologies. This course will introduce students to the foundations of geographic data analysis and visualization, grounding practical studio based exercises and projects with critical readings and theory. Students taking this course will gain an appreciation for geographic thinking, learn to ask geographic questions, and apply basic methodologies to “make sense” of geographic data. The course will be conducted with open source geographic information systems and web technologies. Students are not expected to enter the class with knowledge of these tools, but will be expected to learn and apply them through studio exercises and project briefings.
Over 7 weeks students in this course will explore different game mechanics, puzzle mechanics, group dynamics, and narrative structures and work in groups to design and build a room sized escape game. We will explore how to design immersive and participatory experiences through play and problem solving. Students will construct weekly puzzles and narratives and in the final week build and operate an “escape room” experience. Prerequisites: Physical Computing and ICM. Comfort with fabrication strongly encouraged.
Autonomous Artificial Artists (AAA) is a class to explore ways of making artworks “autonomous.” In this context, “autonomy” brings together three independent but related criteria: 1) artificial intelligence being a primary determinant in an artwork’s aesthetics 2) autonomous software principles culled from peer-to-peer network design, blockchain and decentralization technology, serverless and federated machine learning, cryptoeconomics, and agent-based multiplayer simulation. 3) crowd-sourced art where mass, unbounded cooperation of many participants creates novel artworks which represent the “hive mind” or collective input. The goal of this class is to learn a little bit about each of these seemingly disparate fields, and see how they may interact in interesting new ways. The idea of autonomous artworks is very new, and is being actively discussed by a small group of interdisciplinary researchers and artists since 2016/2017. Although the topic is highly experimental, it is nevertheless based on concrete technologies, making simultaneous use of several techniques which are under active development and have potentially far-reaching ramifications well outside the domain of art. The time is ripe for people within more design-oriented fields to begin thinking about how they might be used in a broader context. The class has both a theoretical component (learning about each of the individual technologies and their interplay) as well as a practical component: training and deploying generative models on computational environments that are as close to decentralized or autonomous as possible. In addition, we will explore prior notions of crowd-sourced or mass-collaborative art, touching on older principles and strategies such as Oulipo, exquisite corpse, and crowd-sourced computational artworks like Electric Sheep, Exhausting a Crowd, and others.
This class is a project development studio for interactive music projects —that is, pieces of music that are not linear, but rather offer multiple dimensions for listeners to explore (on their phones in a crowded subway, at an abandoned factory in Palermo, back on their couches after a long day, at a classical concert hall). Students will take a project from concept to execution over several iterations, applying Interaction Design principles and techniques. During the first half of the semester, they will gather aural and visual references, compose graphic notations, and create interactive studies to explore specific elements of their composition. This work will lead to the implementation of the midterm project: a functional, high-fidelity prototype. For their final projects, students will evaluate their midterm pieces from the perspectives of music, visual design and interaction design, and refine them to produce an expressive piece of interactive music. ICM or equivalent experience is required. Some experience in making or producing music will be useful, but is not required.
Stereo (2-speaker) sound is the default way we produce and distribute most audio. This class challenges the stereophonic-centricity of digital sound and instead focuses on the context of listening, interfacing and interacting with audio beyond 2 speakers. We will take a novel approach to spatialization by interfacing web technologies (Javascript, Web Audio, WebRTC) with multichannel audio to create room-scale interactive music and sonic spaces, and then make our findings publicly available through musical artifacts, open source tools, and documentation. We will explore conventional and unconventional loudspeaker arrangements ranging from 3 to 40 channels and how distribution and context affects music. Together, we will participate in a semester-long project examining the affordances of spatial and multichannel sound, designing and build open source tools to interface Tone.js, Higher-Order Ambisonics (HOA) and other multichannel techniques. Students will perform/install their final on the 40-channel speaker array at Dave&Gabe’s studio in Bushwick. Topics include 3D sound, open source, space and listening. Prerequisites: willingness to listen critically, some programming experience. No formal music training required.
This class will apply a product and service design lens to the rapidly evolving role of technology in politics, government services, and human rights, with a dual focus on the use of technology to advance the public good, and the threats that technology can pose to various aspects of civil society. On the one hand, there has been an explosion of technology tools that aim to promote healthy and peaceful democracies, provide more efficient government services, and promote human rights. This course will survey those efforts and the product design methodologies that guide them. On the other hand, 2018 has been a reckoning year for technology all over the world. The post-mortem of the 2016 US Presidential election shined a light on technology’s role in a promulgating a deeply polarized electorate, enabling election-meddling from foreign actors, and profiligating fake news, while also raising deep concerns about data privacy and security for everyday citizens. Meanwhile, new technologies like AI and Blockchain are poised to have deeply transformative effects on multiple aspects of society – from policing to education to financial services. This course will also explore the risks various technologies can pose to privacy, safety, liberty, and well-being. The course will feature guest speakers from leading practitioners in the space. Potentially including the ACLU, Higher Ground Labs, Elucd (YC17), NYU Better Policing Initiative, Microsoft, and more.
In this course, students will learn the process of developing products that address user needs. Students will go through the process of identifying a user need, developing a product prototype, evaluating the product with the target user, and outlining the next development steps. Topics will include such elements as need finding, archetype development, user journey maps, ideation, prototyping, user evaluation and validation.
How do you process data from connected devices? This class examines how to build systems to collect, process, store, and visualize data from connected devices. The class will review and discuss real world IoT systems using case studies and actual projects. We will build system using Arduino hardware and open source software. We will discuss how to IoT systems are built on commercial cloud infrastructure. Students will learn about IoT devices and the data pipelines for processing data. They will build an Arduino based device to send and receive data over WiFi via MQTT. Students will write code to move data from MQTT into a database. Students will learn how to query the database and present data as tabular data and graphs. To gain an understanding of an entire IoT system from device to application, we will start at a high level and then drill into each of the pieces — we will: * Discuss sensor hardware and wireless options (WiFi, Cellular, LoRaWAN, LTE-M, etc) for moving data to the server * Discuss transport options MQTT, CoAP, AMQP, HTTPS, etc. * Examine SQL, NoSQL, and Time Series Database * Look at tools and techniques for querying and visualizing data. Prerequisite: * Introduction to Physical Computing * Introduction to Computational Media (suggested) The class will be a mix of lecture, discussion, and building IoT systems. Real world examples and case studies will be used to demonstrate how IoT can be built.
he course title, “Veillance” is a reference to the root of “surveillance” and “sousveillance”, watching from above and watching from below respectively. As digital media becomes a greater part of our everyday lives, it is important to understand the new forms of surveillance that it enables as well as to harness these capabilities and perhaps to create systems of sousveillance. Through the course, we’ll critically examine technologies that have become integral part of our lives; the technologies that drive the internet, the capabilities of web browsers, mobile phones, and the emerging class of networked devices such as IP cameras and Amazon’s Alexa and Google Home. Through the course of this examination, we’ll look at how this technology works, the policies that govern their use, what their benefits are, and finally look at what we are giving up by embracing them. This will be a hands-on endeavor; we’ll develop software to illustrate their use and misuse as well as looking for ways that we can harness their “veillance” capabilities for creative, educational, and possibly subversive purposes. Of course, in this process, we’ll pay particular attention to ethical and moral concerns. The course will have weekly reading and technical assignments, encompasses group work, and will culminate final projects.