Intangible Interaction

Spring 2020 @NYU ITP
Thursday, 12:10pm-2:40pm
ITP Course InfoClass Overview

Instructor: Yeseul Song | | @yeseulsong_office hours link

Class GitHub: code examples from the class will be uploaded here.



There will be three production assignments over the semester, along with occasional lab homework and reading. For the details of the production assignments, see the Assignments page.


Document your ideas, sketches, process, and final outcome with texts, images, gifs, and videos on your blog.
Please complete your blog post by every Wednesday 9PM to allow the instructor and classmates enough time to read your post before the class.
You are expected to read at least 2 of your classmates’ blog posts every week.


On-time Class Participation 20%
Documentation 20%
Class Assignments 30%
Final Project 30%


Weekly Schedule 

Weekly schedules are subject to change as we go, depending on the class dynamics, students interests and progress.

Click each week’s title to access the session pages. Each week’s page will open before/after each class.

January 30

Week 1. What is Intangible Interaction?

Lecture/Discussion: Intro

In-class Activity: Implicit Interaction Framework

Project Announced: Redesigning Interaction (Project #1)



  • Do the Part I of the Redesigning Interaction (Project #1) and blog post about your 1) observations along with analysis of the interaction and 2) research about the current system works technically.
  • Based on the lecture, discussion and readings, write a blog post that address your answers to any of these questions: What is your definition of intangible interaction? What is your favorite project that involves intangible interaction? Have you seen any projects that trick your senses with sensory illusions?  What is the relationship among intangible interaction, multi sensory experience and human perception?
  • Complete the class survey, which includes the submission of your blog link to be added to the class website. 
February 6

Week 2. Proximity Sensing: Sensor Survey & Programming

Discussion: Sensing and perception (based on the previous week’s reading and blog posts)

Show & Tell: Redesigning Interaction (Project #1’s Part I: What’s your observations and research?)

Surveying and choosing proximity sensors for applications
Physical Computing Review. digital & analog input/output, I2C communication for sensors

Lab: Using a beam breaker sensor with attachInterrupt() and serial communication for sound output


February 13

Week 3. Project #1 Presentations & ADPS9960 & Environment Sensors (1) 

Class Presentations : Redesigning Interaction (Project #1). Each team will have 7 min each including presentation and feedback.

Lecture: Environment sensors (1) – rain and wind with weather station, wind sensor

Reading sensor values from ADPS9960 (proximity, light, RGB, and gesture sensor). ADPS9960 will be distributed in class.

Project Announced: Curious Cube (Project #2)


If you want to go deeper into this topic, read below also:


  • Brainstorm up to three ideas for Curious Cube (Project #2) and post the sketches on the blog. You’ll present your ideas in the next class. Take advantage of the two assignments below to spark/refine your ideas:
  • 1. Find three sensors that can be useful in implementing intangible interaction from these resources: SensorWikiSensor Reports from Tom Igoe’s Sensor Workshop class, Sparkfun’s sensor section, Adafruit’s sensor section
  • 2. Make a simple project with ADPS9960. Use this as an opportunity to test or refine your ideas for Curious Cube. You can use any of the sensor’s capabilities: light intensity, color, proximity, and gesture sensing. Some examples/inspirations: Abra-Cadabra, Wavepad, Gesture Controlled Maze, Color Detector for Blind People. We’ll be doing value optimizations in the next class so it’ll be good to get familiar with this sensor. However, if you have a particular sensor you want to work with for the Curious Cube, you can play with that instead.
February 20

Week 4. Environmental Sensors (2) & Sensor Calibration

Show&Tell: Your ideas for Curious Cube

Discussion: Gestures in different cultures (based on the previous week’s readings and your experiences)

Lecture: Environmental Sensors (2) – temperature, humidity, pressure, air quality

Lab: (automatically) Calibrating sensor readings from ADPS9960


  • Narrow down your idea for Curious Cube (Project #2), research relevant sensors/technologies, and blog post the system diagram and BOM. Start prototyping if you have more time.
  • Complete and review the in-class lab: calibrating ambient light value on APDS9960 to control a servo (code here). If you can, make a simple application based on the lab, or apply the calibration technique to another sensor and output.
  • Watch Project Planning Video
February 27

Week 5. Extending Capability of Sensors 

Show & Tell: Curious Cube (Project #2) project planning

Light pipes: Concept, related research, applications and design of light pipes

Lab: Smoothing values with ADPS9960.


  • Keep working on Curious Cube (Project #2): modify the design based on feedback if needed and work on the implementation.
  • Review the in-class lab.


  • Book chapters from Learning from Las Vegas
  • WSJ articles on new Moma’s space design and curation (PDF will be added)
  • The Trouble with Table 101 (Podcast)
March 5

Week 6. Sensing with Signals

Lecture/Lab: Using Bluetooth and Wifi for sensing

Discussion: Intangible interaction in exhibition or spatial design, based on reading


  • Keep working on Curious Cube (Project #2): implementation, play testing
March 12

Week 7. Curious Cube (Project #2) Presentations


Assignment: Individual Field Trip

  • Visit at least one of the venues from the list and blog post about your experience and thoughts. A list that has art/design exhibitions and institutions related to our class will be announced later.
March 19
🌸Spring Break🌸

March 26

Week 8. Body/Skeleton Tracking with Depth Cameras

Lab: Object location tracking with Kinect1, Skeleton tracking with Kinect2, Introduce other depth tracking cameras

Lecture/Discussion: Intangible interaction in art making (based on the previous week’s readings and individual field trip)

Project Announced: Final Project (Project #3)


  • Brainstorm ideas for Final Project (Project #3)
  • Make a simple application with depth sensing
April 2

Week 9. Sensing with Cameras

Lab: Intro to OpenCV, A brief intro to PoseNet

Lecture: Intangible feedback


  • Prepare a proposal for Final Project (Project #3)
  • Make a simple application with either depth sensing or camera vision
April 9

Week 10. Class Presentations (Final Project Proposal)



  • Work on Final Project (Project #3)
April 16

Week 11. Guest Talk

A case study on multi-sensory inclusive design in a museum environment
AND/OR Conceptualization and development of Programmable Air (Details will be announced later)


  • Work on Final Project (Project #3)
April 23

Week 12. User Testing for Intangible Interaction

Lecture: User testing methods (Surveying, observation, evaluation metrics, eye tracking, a/b testing, focus group, etc.)

In-class activity: User testing session design for final project


  • Work on Final Project (Project #3): prepare for the user testing in the next class.
April 30

Week 13. Final Project User Testing & Workshop


  • Reflect the user testing results and keep working on Final Project (Project #3)
May 7

Week 14. Final Project Class Presentations



Plagiarism is presenting someone else’s work as though it were your own. More specifically, plagiarism is to present as your own: A sequence of words quoted without quotation marks from another writer or a paraphrased passage from another writer’s work or facts, ideas or images composed by someone else.


The core of the educational experience at the Tisch School of the Arts is the creation of original academic and artistic work by students for the critical review of faculty members.  It is therefore of the utmost importance that students at all times provide their instructors with an accurate sense of their current abilities and knowledge in order to receive appropriate constructive criticism and advice.  Any attempt to evade that essential, transparent transaction between instructor and student through plagiarism or cheating is educationally self-defeating and a grave violation of Tisch School of the Arts community standards. For all the details on plagiarism, please refer to page 10 of the Tisch School of the Arts, Policies and Procedures Handbook, which can be found online at:


Please feel free to make suggestions to your instructor about ways in which this class could become more accessible to you.  Academic accommodations are available for students with documented disabilities. Please contact the Moses Center for Students with Disabilities at 212 998-4980 for further information.


Your health and safety are a priority at NYU. If you experience any health or mental health issues during this course, we encourage you to utilize the support services of the 24/7 NYU Wellness Exchange 212-443-9999. Also, all students who may require an academic accommodation due to a qualified disability, physical or mental, please register with the Moses Center 212-998-4980. Please let your instructor know if you need help connecting to these resources.


Laptops will be an essential part of the course and may be used in class during workshops and for taking notes in lecture. Laptops must be closed during class discussions and student presentations.  Phone use in class is strictly prohibited unless directly related to a presentation of your own work or if you are asked to do so as part of the curriculum.


Tisch School of the Arts to dedicated to providing its students with a learning environment that is rigorous, respectful, supportive and nurturing so that they can engage in the free exchange of ideas and commit themselves fully to the study of their discipline. To that end Tisch is committed to enforcing University policies prohibiting all forms of sexual misconduct as well as discrimination on the basis of sex and gender.  Detailed information regarding these policies and the resources that are available to students through the Title IX office can be found by using the following link: Title IX at NYU.