Peiqi SU | ps2409@nyu.edu | www.supeiqi.com

Paper
The Beauty of Communication: A Real-Time Audiovisual Interface for Conversation. Under review for 13th International Conference on New Interfaces for Musical Expression (NIME)

Contents

Concept

This project is aimed at visualizing the communication experience between people with different cultural backgrounds. From my 3-month experiences in ITP, people can always have a nice chat when sitting at the table, no matter how different their countries, ages, colors, languages and even cultures. Also cultures always influence the way of communication. For example, easterners speak often tactfully and mildly, like tea; westerns speak always directly and excitingly, like coffee. It would be a pleasure to see the process of the conversation and the fusion of cultures.

Here is a video to quickly introduce the idea, user scenarios, and system diagrams:

Play Testings

Here is a video of play testing. // Nov.27 2012

Pictures

surroundings

sketch on the computer screen

running for a half an hour

projection on a table covered with a white paper

left: projection on a white paper | right: projection on a black table

System Diagrams

This is an animation generally showing how the system works:

My bread board is like this: (Diagram made with Fritzing - download)

The whole installation is hidden under the desk:

The biggest problem I encountered was to analyze the frequency of sound captured by 2 electret microphones. How to do FFT in Arduino board? Please see my blog here: [Pcom] Visualize Sound Frequency Captured By 2 Microphones. You will find the great FFT library and Timer library for Arduino. For how to conect the electret microphone, please see my blog: [note] Electret Condenser Microphone
This video shows how well my voice was analyzed:

Bills of Materials

User Scenarios

When people come to the table, they will see the surface of the table appearing a pattern as flowing water, which is projected by a projector from the ceiling. They can choose a cup of tea or coffee, and then sit down and chat normally. The hidden microphones will collect the volume and frequency of each person, and corresponding animations, generated by Arduino and Processing, will show on the table. Also specific volume and frequency, such as laughter, will trigger animals like a group of fish or a flock of birds. If no one speaks, water flow will be displayed, since it is still a kind of communication even without speaking. Finally, when people leave, all graphics will fade.

Here is a sketch of user scenarios:

Brainstorming

visual inspiration came form these trees - left: Piet Cornelies Mondrian | right: Guanzhong

Coffee and tea are normal in a conversation, and they also could be symbols representing different culture.

ideas on whiteboard

ideas on sketch book

References

Thanks to Tom, Katherine, Daniel, Eric, Todd and everyone who helped me! Thank you!

Processing inspired by:

Libraries for Processing and Arduino:

Circuits and Others: