I would like to use this post to go over the progress I made for the week, share my production timeline, and layout next steps…

Production Timeline

I really liked Adam’s timeline and decided to use his formatting as a template for mine. Thanks Adam!

 

Performance 1.1 Proof of Concept

This week, I spent the majority of my time working on a proof of concept for the first performance, what I am for now calling “The Breathing Room.” It started with this image of my room glowing in response to my breath, with the lights burning bright with each inhale and falling dark as I exhale silently. I ended up switching this, with my exhale causing the lights to burn, because it was easier to do in my program.

I worked a lot with SSML to make this. In a Fly-By I led a couple of weeks ago, we discovered that the <amazon:effect name=”whispered”> tag creates a very human-sounding and unnervingly intense tone. Using the <prosody rate=”20%” pitch=”-33%”> tag with this whispered effect, I was able to make the absolute creepiest, weirdest sounding Alexa voice I have ever heard come out of a smart speaker. It doesn’t feel like I should be allowed to generate this affect using the amazon developer tools. In my first run through, I kept all of the speaking at this rate, but it was just too monotonous and creepy. Here is the code for the text I came up with:

PART 1:

`<break time=”2s”/><amazon:effect name=”whispered”><prosody rate=”70%”>Take </prosody><prosody rate=”50%”>a moment </prosody><prosody rate=”70%”>to breathe </prosody><break time=”3s”/><prosody rate=”40%”>relax </prosody><break time=”4s”/>
<prosody rate=”70%”>bring your awareness more deeply and </prosody><prosody pitch=”+30%” rate=”40%”>widely </prosody><prosody rate=”70%”>into your breath</prosody><break time=”3s”/>
<prosody rate=”20%”><phoneme alphabet=”ipa” ph=”ha”/></prosody><break time=”2s”/>
<prosody rate=”70%”>there you go <break time=”3s”/>keep noticing what you notice </prosody><break time=”2s”/><prosody rate=”60%”>feel what you’re feeling </prosody><break time=”2s”/><prosody rate=”50%”>see what you see </prosody><break time=”2s”/><prosody pitch=”-33%” rate=”40%”>be who you are </prosody><break time=”4s”/>
<prosody rate=”20%”><phoneme alphabet=”ipa” ph=”ha”/></prosody><break time=”2s”/>
<prosody rate=”60%”>that’s it <break time=”2s”/>let yourself </prosody><break time=”2s”/><prosody rate=”30%”>relax </prosody><break time=”4s”/>
<prosody rate=”60%”>would you like to have a memory?</prosody>
</amazon:effect>`;

PART 2:

`<amazon:effect name=”whispered”><prosody pitch=”-33%” rate=”20%”><phoneme alphabet=”ipa” ph=”ah”></phoneme></prosody></amazon:effect>
<break time=”2s”/>
<amazon:effect name=”whispered”><prosody rate=”60%”>
Two Children </prosody><break time=”1s”/><prosody rate=”40%”>a brother and a sister </prosody><break time=”1s”/><prosody rate=”60%”>set out sailing in </prosody><prosody rate=”20%”>windy weather </prosody><break time=”3s”/>
<prosody rate=”60%”>They have their first bluetooth speaker sitting in the cockpit <break time=”2s”/>
The girl’s Ipod is wrapped in plastic <break time=”1s”/> to keep the water out </prosody><break time=”4s”/>
<prosody rate=”40%”>It’s their first time setting the spinnaker with this much breeze </prosody><break time=”2s”/>
<prosody rate=”60%”>Once they get it flying they reach across the lake listening to the blaring music </prosody><break time=”3s”/>
<prosody rate=”40%”>When was the last time you felt like this </prosody><break time=”5s”/>
</amazon:effect>
<amazon:effect name=”whispered”>
Now he shitts on the toilet and scrolls through videos of world champion racers
It’s his sister’s birthday and he can’t comprehend when they last were on a boat <break time=”3s”/>
<prosody rate=”50%”>It’s okay <break time=”2s”/> there’s plenty of time</prosody>
</amazon:effect>`;

As I was developing this, a few themes came up that inspired the content I ended up with. The first was the theme of computer assisted memory. I was thinking about how good my iPhone is at selecting meaningful photos from my library for each day’s “featured photos,” and was wondering what the equivalent of this would be in the Smart Home. I added in the interaction “would you like to have a memory?” and included this memory of my sister and I sailing on a windy day in Canyon Lake the first time we had a waterproof, bluetooth speaker. I found this memory interesting in particular because its the oldest memory I have of technology really enhancing a naturally exhilarating experience. Now, whenever I have tempo or hill runs, it’s almost second nature to use my headphones to enhance those experiences. I have so much longing for that memory, but I feel like it would require taking time away from technology to make something like that happen again. I believe we often try and force certain emotional, experiential outcomes using technology these days. With this memory, I think I really just miss being there with my sister.

In making this proof of concept, there are many many many things I think I can improve on. Technically, I was able to play a bit with feedback and liked the feeling this gave. However, I was not using p5 sound output, simply routing the sound through Garageband instead. This did make it difficult to do proper mixing for all the sound, as I was using my microphone -> p5 sound -> Echo via bluetooth, my microphone -> garageband, Alexa skill ->Echo via Aws cloud. The balancing was all over the place and made the MQTT light control very finicky.

Thematically, I want to move away from just the creepiness of Alexa. It feels way too on-the-nose right now. In my mind’s eye, the performance would start very soothing, almost like a Headspace meditation experience, and devolve into this creepier feeling. Content wise, I would like to revise so that it highlights the presence of technology (the bluetooth speaker) in my memory.

Sharing/Feedback

On Wednesday, I showed my proof of concept to Ruta and Carrie and met with Arnab for some technical advice. Here is what they had to say:

  • RUTA
    • What is the dramaturgy of this proof? While I am working on making three performances, how can this one be self contained? How much of its meaning needs to be taken from relation to the others?
    • Surprisingly comfortable and the interaction flow was nice.
    • When does the interaction become uncomfortable for me? At what point does the memory machine skill go too far?
    • How to capture liveness in this performance? How can I program places for me to show up as a living human performer?
    • Did not mention specifics about the theme of the memory
  • CARRIE
    • Really liked the creepiness, how I can lean in even more?
    • Felt like the beginning of the horror movie.
    • Wanted to know what happens next.
    • The lighting is creepy from the outset. If I want it to start more calming, start with a different lighting scheme. How can the live lighting be used to convey the journey toward a more sinister place?
    • For the performance using audience interaction, how can I trick them into participating. Should I exit saying “10 minute break” only for the devices to yell at the audience to stay in their seats?
  • ARNAB
    • Consider getting a Philips Hub so I have more access to their cloud services
    • Zigbee is a good protocol, but there is much more documentation for other methods using Wifi and Bluetooth, consider moving away from zigbee2MQTT
    • I am still going to have issues with limited amounts of MQTT requests no matter if I am doing it using Wifi, Bluetooth, or Zigbee. Look more into the documentation and find a different work around.
    • Node RED would potentially have a plug in for the hub-optional philips lights

Next Steps

There is so much to do with this first performance and I want to keep tweaking it. I do want to move on to the second performance which will be focused on audience interaction with me puppeteering the smart devices. I believe this one will be the most tech heavy of the performances, so I anticipate needing a week at least to get the capabilities up and running. For this second performance, I want the story to be about devices who are starving for data and each device wants data that says something different. For example, my smart TV may want data that says users are still using their televisions to watch cable so that more TV’s will be made and the TV species will benefit. This performance will also probably need the most user testing, so I want to start working on it now.

Some last thoughts: I need to get back to posting more regularly and in shorter bits. I feel like once a week is far too little to capture everything I have been working on. I also want to discuss how to document something that will ideally be performed live while we are still remote. I spent a lot of time hitting my head against the wall trying to document the first proof.