Parsing our Silent Language is a computer program that parses body language in real time utilizing machine learning and skeletal tracking.

https://itp.nyu.edu/thesis2016/project/kat-sullivan

Description

Parsing the Silent Language is software that analyzes nonverbal language using a Kinect and the C++ framework, Cinder. Coming from a movement background, I was curious why sentiment analysis placed all emphasis on our words when our body language can contradict or add depth to our words. I started this process by first manually, and then algorithmically, categorizing my own body language. However, I was inadvertently programming my own movement tendencies and as a result, the algorithms became biased to my own body. To address the issue, I enlisted the help of several graduate acting students and recorded their skeletal data while they acted out various emotions. I then used this data to train a classifier using machine learning, which now analyzes body language in real time.

Classes

Thesis