Distance is a key point of my project. Trigger sound by the distance changing between people, to bring them temporary closeness is the basis for the ultimate goal of my entire project. I currently read the distance in two approaches. One is Posenet, the other is in the AR environment, which is very strong in expressing the soundscape and relating the audience from the virtual world to the real world.

  • pose net

The results of the Posenet test last Wednesday gave me a big strike. Thought it has advantages in reading the distance between people on the screen. My sketch was so sensitive that sometimes there are too many sounds triggered, which caused my computer to be overloaded.  The experiment went not well, I didn’t get what I need. The code needs to be reconstructed. The model still needs to be improved and testing.

  • Kinect in unity

I headed on to using Kinect in Unity. Unity got a lot of errors with AR scene-setting, which makes it impossible for me to continue to promote the AR part. Though I’ve asked Craig for technical support. Before I got the help, I have turn to use Ar software to a prototype.（The default objects in this scene don’t make any sense. I’m just testing the sound.）

Last Thursday, I had a meeting with Xingchen, a professor in NYU Shanghai IMA who is familiar with Unity. He believes that the current AR technology only reads the position of the mobile phone and the ground. In terms of detecting the distance in a limited space, he suggested at least four Kinects from different directions should be needed. I mentioned that Piere suggested using GPS to try it out. He said he felt that it required WebSocket, this was too much for me to handle. But if I could set the sound on a virtual object, that would be much easier for me. I appreciate this meeting, such great tech research for me. With the limited technical support, I think the idea of trigger sounds in AR scenes by people’s distance changing in the physical world is not feasible.

  • users testing

I shared my concerns with Andrew in User Testing last week. For the user testing, I have to focus on the current content. My team members and Andrew gave a lot of creative suggestions.

what I learned from the feedback is I could switch my focus from random people to a limited group of people, to set this project as an ice-breaking exercise. Another point, however, is that I need to give more advice about the lines，and more time for the audiences to respond to the instructions. I think if I can make it system finally flexible to respond to the distance, this issue will figure out.

I think Andrew gave me a brilliant suggestion on this point，it is setting more instructions to let people feel this is a paradox play, like a game “everything I said is a lie”. In this way, my challenge is I need to like set up with instructions enough to that let the audiences know they’re in a paradox world or a play world where the rules aren’t the same and then slowly let them go. It might feel counterintuitive. I thought when I heard that that would solve my problem in terms of content. Since currently, I was in a state of uncertainty due to technical problems. When I review my concept, I still hope that people have the initiative to get some experience in the project, rather than relying on instructions. Though unity doesn’t treat me well, from my experiment, when I added sound sources to the scene, I got this:

I was excited about the scene of everybody is talking but I’m not involved in any conversation. I was free to pick out what I wanted to hear from the chaos. If I want, I will go closer to the specific source. I think it’s more intuitive and reflects the preferences， also it makes introverts feel comfortable.

Gal showed me this project at my feedback session. With individual microphones, the artist has recorded each singer of the choir of Salisbury Cathedral, working with male voices for bass, baritone, and tenor as well as a child soprano.

Base on the experiment and inspiration. I want to keep iterating on my ideas and revise my commitment. I will move to the physical world where I feel more comfortable going.