Intangible Interaction

Spring 2020 @NYU ITP
Thursday, 12:10pm-2:40pm
ITP Course InfoClass Overview

Instructor: Yeseul Song | | @yeseulsong_office hours link

Class GitHub: code examples from the class will be uploaded here.



There will be three production assignments over the semester, along with occasional lab homework and reading. For the details of the production assignments, see the Assignments page.


Document your ideas, sketches, process, and final outcome with texts, images, gifs, and videos on your blog.
Please complete your blog post by every Wednesday 9PM to allow the instructor and classmates enough time to read your post before the class.
You are expected to read at least 2 of your classmates’ blog posts every week.


On-time Class Participation 20%
Documentation 20%
Class Assignments 30%
Final Project 30%


Weekly Schedule 

Weekly schedules are subject to change as we go, depending on the class dynamics, students interests and progress.

Click each week’s title to access the session pages. Each week’s page will open before/after each class.

January 30

Week 1. What is Intangible Interaction?

Lecture/Discussion: Intro

In-class Activity: Implicit Interaction Framework

Project Announced: Redesigning Interaction (Project #1)



  • Do the Part I of the Redesigning Interaction (Project #1) and blog post about your 1) observations along with analysis of the interaction and 2) research about the current system works technically.
  • Based on the lecture, discussion and readings, write a blog post that address your answers to any of these questions: What is your definition of intangible interaction? What is your favorite project that involves intangible interaction? Have you seen any projects that trick your senses with sensory illusions?  What is the relationship among intangible interaction, multi sensory experience and human perception?
  • Complete the class survey, which includes the submission of your blog link to be added to the class website. 
February 6

Week 2. Proximity Sensing: Sensor Survey & Programming

Discussion: Sensing and perception (based on the previous week’s reading and blog posts)

Show & Tell: Redesigning Interaction (Project #1’s Part I: What’s your observations and research?)

Surveying and choosing proximity sensors for applications
Physical Computing Review. digital & analog input/output, I2C communication for sensors

Lab: Using a beam breaker sensor with attachInterrupt() and serial communication for sound output


February 13

Week 3. Project #1 Presentations & ADPS9960 & Environment Sensors (1) 

Class Presentations : Redesigning Interaction (Project #1). Each team will have 7 min each including presentation and feedback.

Lecture: Environment sensors (1) – rain and wind with weather station, wind sensor

Reading sensor values from ADPS9960 (proximity, light, RGB, and gesture sensor). ADPS9960 will be distributed in class.

Project Announced: Curious Cube (Project #2)


If you want to go deeper into this topic, read below also:


  • Brainstorm up to three ideas for Curious Cube (Project #2) and post the sketches on the blog. You’ll present your ideas in the next class. Take advantage of the two assignments below to spark/refine your ideas:
  • 1. Find three sensors that can be useful in implementing intangible interaction from these resources: SensorWikiSensor Reports from Tom Igoe’s Sensor Workshop class, Sparkfun’s sensor section, Adafruit’s sensor section
  • 2. Make a simple project with ADPS9960. Use this as an opportunity to test or refine your ideas for Curious Cube. You can use any of the sensor’s capabilities: light intensity, color, proximity, and gesture sensing. Some examples/inspirations: Abra-Cadabra, Wavepad, Gesture Controlled Maze, Color Detector for Blind People. We’ll be doing value optimizations in the next class so it’ll be good to get familiar with this sensor. However, if you have a particular sensor you want to work with for the Curious Cube, you can play with that instead.
February 20

Week 4. Environmental Sensors (2) & Sensor Calibration

Show&Tell: Your ideas for Curious Cube

Discussion: Gestures in different cultures (based on the previous week’s readings and your experiences)

Lecture: Environmental Sensors (2) – temperature, humidity, pressure, air quality

Lab: (automatically) Calibrating sensor readings from ADPS9960


  • Narrow down your idea for Curious Cube (Project #2), research relevant sensors/technologies, and blog post the system diagram and BOM. Start prototyping if you have more time.
  • Complete and review the in-class lab: calibrating ambient light value on APDS9960 to control a servo (code here). If you can, make a simple application based on the lab, or apply the calibration technique to another sensor and output.
  • Watch Project Planning Video
February 27

Week 5. Extending Capability of Sensors 

Show & Tell: Curious Cube (Project #2) project planning

Light pipes: Concept, related research, applications and design of light pipes

Lab: Smoothing values with ADPS9960.


  • Keep working on the Curious Cube (Project #2): Create physical prototypes, implement the core technical parts, and iterate. Make a blog post about the process.
  • Review the in-class lab.


March 5

Week 6. Sensing with Signals

Discussion: Intangible interaction in exhibition or spatial design, based on the readings

Lecture: Using Bluetooth and Wifi for sensing

Lab: ArduinoBLE


  • Keep working on Curious Cube (Project #2): implementation, play testing. You’ll present in class next week.
March 12

Week 7. Curious Cube (Project #2) Presentations

Each team will have 9 min for present+feedback.



  • Document your Curious Cube project on your blog, if you haven’t. Your documentation includes:
    • Concept, sketch, progress images/videos with descriptions, reflections, final outcome (whether the project is finished or not), what you would like to do differently if you had more time.
  • Tour to the Virtual Intangible Interaction Gallery
    • “Come visit the Virtual Intangible Interaction Gallery. We just opened three virtual exhibitions for you. Each exhibition has a different theme and each features artists or a group of work that demonstrate different aspects of intangible interaction.Pick one of the virtual exhibitions from the gallery and take a tour. That said, pick one of the exhibitions you’re interested in, take a look at the listed work, and research the artist/work (e.g., read about the artist, read about work, research how the technical aspects are implemented…).Make a brief blog post about your tour to the exhibition. How did you feel at the show? What is your favorite work if you had? How is intangible interaction used in pieces? If you were the artist, what would you have done differently?”
  • Individual Field Trip: Visit at least one of the venues from the list and blog post about your experience and thoughts. A list that has art/design exhibitions and institutions related to our class will be announced soon. I’m still in the process of coming up with alternatives under this corona situation because most museums are closed. (This has been replaced with Tour to the Virtual Intangible Interaction Gallery above, due to the closures of all cultural venues.)
March 19
🌸Spring Break🌸

March 26

Week 8. Sensing with Cameras (1): Intro


Discussion: Virtual Gallery Tour

Lecture/Demo: Sensing with Cameras (Project Sharing by Guest M.H. Rahmani)

Lab: Controlling physical object with object tracking from camera (Object Tracking with Processing, Serial Communication from Processing to Arduino)

Project Announced: Final Project (Project #3)


April 2

Week 9. Sensing with Cameras (2): Using PoseNet, Teachable Machine (and more) with Physical Output

Lecture, Demo, Lab: More on computer vision and using them for physical computing. Computer vision tools and libraries that use machine learning. PoseNet, Teachable Machine and more. 


  • Prepare a proposal for Final Project (Project #3) to share during the next class
  • Make a simple application based on the labs/resources from the week (see more in the Week 9 class notes linked from the week’s title). If relevant, try to use it for an ideation/prototyping opportunity for your final project idea.


April 9

Week 10. Class Presentations (Final Project Proposal), Grant Writing Tips

Lecture: Grant Writing Tips

Presentation/Discussion: Final Project Ideas

Work on Final Project (Project #3). Identify the installation venue, context, and audience interaction of your project. Make prototypes–the prototypes can be either physical or digital. If digital, utilize prototyping tools (see Resources > Prototyping Tools).

April 16

Week 11. HTTP Request, Getting Weather Data from Database

Presentation/Discussion: Final Project Ideas (the rest 5 people are presenting this week)

Lecture/Lab : HTTP Request (POST, GET) from Arduino, Getting Weather Data from Database over Wifi, Using Weather/Pollution API from Arduino


  • Work on Final Project (Project #3): Make prototypes and iterate. Make a system diagram of the project and itemized budget including BOM. Enrich your draft with notes and ideas. For details of the proposal requirements, see the assignments page.
  • Make a simple application with http request / API. Feel free to play with example sketches that I linked at the end of the Week11 page.


April 23

Week 12. User Research, Guest Talk

Lecture: User research as a testing tool and research tool

Guest Talk: Eozin Che (Lead Creative Technologist at American Museum of Natural History Museum)


  • Work on Final Project (Project #3)–be prepared to share the first draft of your proposal during the next class. In the next class, we will make suggestions on each other’s drafts. I emailed the template and a sample for the writing.
  • Make sure you have prototypes of your project to be included in the proposal as videos.
  • Please submit a link to your first draft on this form by 8pm Wednesday, April 29.
April 30

Week 13. Final Project Consultation / Cross Editing of Proposals

Lecture: User research, intangible interaction, and research based art

Discussion: Recovery from Early Blindness – A Case Study


  • Keep working on Final Project (Project #3) – prototype, proposal
  • Conduct user testing if applicable
May 7

Week 14. Final Project Class Presentations



Plagiarism is presenting someone else’s work as though it were your own. More specifically, plagiarism is to present as your own: A sequence of words quoted without quotation marks from another writer or a paraphrased passage from another writer’s work or facts, ideas or images composed by someone else.


The core of the educational experience at the Tisch School of the Arts is the creation of original academic and artistic work by students for the critical review of faculty members.  It is therefore of the utmost importance that students at all times provide their instructors with an accurate sense of their current abilities and knowledge in order to receive appropriate constructive criticism and advice.  Any attempt to evade that essential, transparent transaction between instructor and student through plagiarism or cheating is educationally self-defeating and a grave violation of Tisch School of the Arts community standards. For all the details on plagiarism, please refer to page 10 of the Tisch School of the Arts, Policies and Procedures Handbook, which can be found online at:


Please feel free to make suggestions to your instructor about ways in which this class could become more accessible to you.  Academic accommodations are available for students with documented disabilities. Please contact the Moses Center for Students with Disabilities at 212 998-4980 for further information.


Your health and safety are a priority at NYU. If you experience any health or mental health issues during this course, we encourage you to utilize the support services of the 24/7 NYU Wellness Exchange 212-443-9999. Also, all students who may require an academic accommodation due to a qualified disability, physical or mental, please register with the Moses Center 212-998-4980. Please let your instructor know if you need help connecting to these resources.


Laptops will be an essential part of the course and may be used in class during workshops and for taking notes in lecture. Laptops must be closed during class discussions and student presentations.  Phone use in class is strictly prohibited unless directly related to a presentation of your own work or if you are asked to do so as part of the curriculum.


Tisch School of the Arts to dedicated to providing its students with a learning environment that is rigorous, respectful, supportive and nurturing so that they can engage in the free exchange of ideas and commit themselves fully to the study of their discipline. To that end Tisch is committed to enforcing University policies prohibiting all forms of sexual misconduct as well as discrimination on the basis of sex and gender.  Detailed information regarding these policies and the resources that are available to students through the Title IX office can be found by using the following link: Title IX at NYU.