On the Fly! Jamming with Code

Photograph by Hayeon Hwang

Over Halloween weekend last year, Dave Currie (ITP ‘22) and Jeeyoon Hyun (ITP ‘22) hosted a 2-day livecoding workshop at ITP that I was fortunate to attend. We installed and played around with TidalCycles, a software that generates music through code in a way that both celebrates and defies repetition.

Since then, I’ve learned that ITP has a massive livecoding presence in the global livecode community, which, admittedly, is very small. I invited Jeeyoon, Dave Stein (ITP ‘23), and Yeseul Song (ITP Faculty) to chat about the highs and lows of livecoding, their rituals, and what they’re looking forward to next. 

Jeeyoon performs as herself and uses TidalCycles, Ableton Live, and Supercollider. Dave performs as Colonel Panix and uses Fox Dot, Supercollider, and other custom tools. Yeseul performs as KoalaTokki with Michael Simpson (ITP ‘18) and uses Fox Dot, Supercollider, openFrameworks, and incorporates physical computing.

JML: Can you talk about your rituals as a performer, either in person in front of a crowd, or at home in your practice?

JH: I think everyone has different rituals. Because I use TidalCycles I usually make sample-based compositions and music. For me, the first step in preparing a performance is gathering and processing samples from Ableton. I often record interesting sounds in my everyday life, for example popping noises from a La Croix soda can or weird noises from the refrigerator. I use my phone to record these and process them in Ableton Live to make them sound more interesting. I also download sounds from the Free Sounds Website.

Then I try connecting them and applying different effects to see if they make sense. If I find an interesting loop or interesting effect that I really like, I add it to a file of all my favorite lines of code, so my performance is based on my favorite lines of code. I think everyone has that favorite piece of code. I mean, you could write everything from scratch but I’m not that type of person.

Also, you should always have collaborators, because when you’re performing you can’t do it by yourself. When I perform, I always do it with my friends, especially from the second year students at ITP. I performed with Dave Currie but there’s also John Bezark who does amazing visuals. We also did jams with Todd Whitney and Ami Mehta. We participated in an official TidalCycles global livestream just an hour after we did a livecode workshop together, and it went well.

I think you always need collaborators when you’re livecoding and I think that is also part of my rituals–connecting with the friends you want to perform with and setting everything up together.

JML: Dave, you performed at Wonderville last Sunday and you were paired with someone for visuals. Have you worked with them before? Was this your first time? How did you collaborate with them on that piece?

DS: I would say it was a light collaboration. We were a little bit in touch a couple days before but we had also performed together in August and we know each other from SFPC (School for Poetic Computation). That partner was Edgardo Avilés-López. He’s an amazing artist and he’s now teaching at SFPC with Zach Lieberman.

For the performance, I gave him some samples of what I planned to do so that he could adjust accordingly. Because these kinds of visuals are really about setting the mood, we talked about what the arc of my music was going to be. For example, it’s going to start off as a drone and then I’ll gradually bring in a beat and end with some samples. So I gave him that description and he probably took his own ideas to make whatever he wanted to do in visuals.

Caption: Dave Stein (ITP ‘23) performing as Colonel Panix and Edgardo Avilés-López performing at Wonderville, Bushwick, NY

+ Photo: Julia Margaret Lu


I have also worked with Nick Monfort who’s a computational poet and teaches at MIT. Yeseul saw our performance, it’s very unique. He raps, and the lyrics are incredibly clever and a little nerdy. He performs with a Commodore 64 which is a home computer from 1982. Nick is very interesting and fun to watch as well. When we performed together we had to practice a couple times because he was rapping to my music, so we had to find the right beats.

Generally in Livecode NY you’ll get randomly matched up with someone. They’re happy to let you come as pairs, but part of Livecode NY is to teach you how to perform. By matching you with different people all the time, you build a broad range of habits. Some habits are “We’re not going to talk until the day of the show;” some are “I need to work with you for a couple hours several weeks in advance.”

JML: What are your rituals before you perform?

DS: I try not to drink too much, I try to get a lot of sleep. I’m usually totally obnoxious to talk to because I’m just too stressed. It’s very very stressful for me. Your reference question, ‘Does live coding bring you joy?’ [chuckles] There’s the release of the adrenaline high but there’s no joy. It’s a challenge and I guess people find joy in climbing mountains and things like that but it’s a huge adrenaline rush. Usually my routines are around calming myself down and trying to think of being in a quiet place.

JML: You also told me that you spend a week preparing for a set.

DS: Usually about two weeks in advance I’ll start playing with ideas, listening to music, thinking of beats or whatever I want to do. About a week before, I start to pull together scraps of what I want it to sound like and then I start playing it. And like Jeeyoon, I keep my favorite code and sometimes I’ll even hide it below the scroll in case I start panicking and want to pull up a line when no one’s looking.

Part of livecoding is that you’re supposed to code from scratch, but one of the best livecode sets, this artist from San Francisco, had everything pre-prepared, which in some circles would be cheating, but it was so so good. He was a music producer and this sounded like an EDM track. He had maybe 400 lines of code, so this would have been totally different if he had to code it live from scratch in 30 minutes.

JML: Yeseul, what about livecoding brings you joy and what are your rituals, either performing at home or in person?

YS: When I perform as KoalaTokki with Michael Simpson we start jamming without saying anything. I usually use the built in sounds within Fox Dot and Supercollider and Michael usually does a lot of sampling. Over time the theme naturally emerges and we begin to think “Okay, I’m going to use this or that pattern, or this or that sound” and then we shape it together. We aren’t planning anything out, it’s more like doing by intuition. It’s like free jazz where the members are listening to the overall sound, following each other’s direction, and taking turns leading things elsewhere. That process was very fun and there is something that only music can do that it really touches your heart, even more than visual arts or cinema to me, so that is really a pure moment of joy.

Usually at clubs if there’s a weird or choppy moment in the DJ set or music it really loses the momentum and people will be cooling down even while dancing. I’m always a little bit afraid of those moments because it’s on the fly. There’s always a moment when the beat is not right or the volume isn’t set right or there’s a weird thing coming on and people are a little bit confused. Those moments are really traumatizing! But I think the more we perform we become stronger in terms of reminding ourselves that “This is live coding, this is a joy, and this is what it is.” And people want to watch those moments.


KoalaTokki (Michael Simpson, Yeseul Song) + Live Coding Performance at ITP Camp 2019 hosted at 721 Broadway, New York, NY. 2019

JML: Jeeyoon and Dave Currie taught us that the livecoding community really embraces epic failures. Have you ever experienced an epic failure when you’ve performed?

JH: Oh of course! [chuckles]

Dave and Yeseul just mentioned being nervous and super stressed before performances and I totally relate to that. I kind of go crazy before a performance, I’m so nervous. Part of it is because of these experiences of failure. I remembered that we were doing a small performance at Wonderville for TNO and there were only a few people in the audience but I was supposed to play like 16 tracks at once and TidalCycles just broke. So something is happening, the music is going on but something is happening and I can’t control it and error messages are just showing up.

When you’re live coding part of the reason that you use prepared code is that you can’t debug when you’re performing. It drives you crazy if you try to debug while you’re doing a performance.

Typically when people sense that there is no change in the music like they will be okay for about a minute but if it gets much longer then people are gonna be like huh? I can feel that happening.

That kind of stuff happens a lot when you’re doing livecode. Right before a performance I feel really terrible; during a performance I feel really panicked and I try to just go into the flow. After the performance I usually feel kind of down because I tend to think “Okay I screwed this up; I should have done this or that” but if you think about it 2 weeks later you’re like “That was wonderful, I want to do it again!” because you just remember the best parts, it motivates you.

JML: I’m glad you put a positive end to that, I was getting worried that people would read this conversation and think “I don’t want to try this, it sounds so stressful!”

JH: No, no! I mean it’s stressful when you’re preparing for a performance but believe me, after you do it you’re going to be so proud of it. Or right after the performance!

DS: I am in total agreement with you. When I’m done I get high from the adrenaline but I’m usually thinking “I could have done this, I could have done that, I didn’t get to this, my levels were off.” But one thing I do notice every time I perform, and this is probably the primary reason why I perform, is that I see myself getting exponentially better after each performance. Significantly better. I see myself coming up with new ideas because building my skill set is behind me; then what I’m looking forward to is new ideas and having the skills to be able to make them happen and that’s the primary reason why I do it.

Caption: Dave Stein (ITP ‘23) performing as Colonel Panix at Wonderville with Ulysses Popple, December 31, 2019 

+ Video: David Stein and Ulysses Popple, Videographer Dylan Mars Greenberg


YS: Yeah I totally agree. Even when I talk about the stressful part, I’m proud that we did it and it’s a really unique experience. It’s a process like David described; “Okay we are doing this, and next time we are trying these out,” so I feel like we are exposing the process of exploration and the growth.

Usually when you go to a concert, an artist has released a new album and there’s a plot already. When you do livecoding it’s about the process itself and the exploration of exposing the under-the-hood code. And in the long term it is exposing the process with the community so that you don’t need to be perfect on stage, you are creating something live and we are enjoying the process together.

Maybe the reason that I’m nervous is that I think when I play music it’s not a very logical thing, it’s intuitive. But I think livecoding has everything: it is very logical but at the same time it’s also musical, though it’s very hard to think on your feet when you have to do this meticulous thing which is code, which is a very private activity, in public.

I find it easier to just forget about the audience and just become part of the code. Not really caring about the surroundings and the crowd and just feeling the rhythm, which I think are sweet spot moments.

JML: Talking about sweet spot moments, when you are performing, what is your thought process behind the way you start and end a set?

DS: Well I’ve learned after many performances that the start has to be easy and has to be able to last a little bit without touching it because my hands are shaking so much.

At the New York Live Arts show, Nick and I got up there and I decided that I was going to code everything from scratch. But my hands were shaking too much to type. So I’m slowly pushing the buttons and Nick is waiting because his plan was to come out with the visuals as soon as the first note hit. But I was typing so slowly and hitting backspace to fix the typos in the first line of code. He whispered “Dave, is everything all right?” It took two minutes to write my first line of code.

So now the way it works is that I have a line pre-typed so I can hit control-Enter. Which you can generally do without too many nerves. The line starts and if I don’t do anything for a minute because I’m typing it’s okay. Then I’ll layer instruments on and sometimes I’ll do a couple different peaks and layer up to something and then come back down and then maybe change the baseline and the rhythm and build back up, come back down and just remove instruments, and that’s how I usually end. Sometimes I’ll even end it the way I started.

Caption: Noodling around in FoxDot 

+ Video: Dave Stein


JH: I usually prepare my set as a file so I have my code and I try to organize them and comment them out with how I’ll start the set or how I’ll wrap up the set, so I just comment stuff a lot, I just hope people don’t see it too obviously.

I usually layer up stuff that I want to show and that’s the first part of the performance. I’ll sometimes silence a track or solo a track and I’ll just kind of slowly fade out by muting every track. I rely on a lot of comments and notes that I’ve written for myself in advance and I have a picture of how it will end. I guess my style is to come up with the structure of the piece and then the live elements of my performance are specific notes that I play, or I’ll change the specific examples that I put in my code where I can change the effects or the composition, and so on.

YS: For me it really depends on which set. We mostly play ambient music for at least 50% of the set and we are also very careful with layering because when there’s a sudden sound change that’s when everything breaks. So we really slowly start with ambient drum beats and  then layer it one by one and make an arc of different compositions but try to build it up very slowly and also bring it down very slowly. At the end we have this ritual of ending with a voiceover of our group name and for some of the samples I recorded my voice, going “KoalaTokki, KoalaTokki” so we’re destroying the song in a fun way and using the voice as sort of an ending signal.

JML: What are some things that you’re looking forward to?

DS: I usually run out of ideas before the end of the set, but I didn’t in the last one, which is interesting. I also build my own tools and I have my own language that will allow me to type music notation and turn that into a live-coded line of code, which I’ve been working on. I really want my stuff to eventually be danceable and move from the ambient space to the danceable space. I think I could do something like that with raw code with the proper library and code behind it. I do spend a bit of time coding my own stuff to make my live coding easier.

YS: What is the tool like?

DS: I created this language when I saw a music teacher filling out a notation software and he wasn’t using his mouse and filled out an entire score by just typing. What is he typing? And how does he do that without a mouse? I thought that it would be really cool to use that for live coding and I found that like there’s actually a shorthand notation language that composers use. So I’m incorporating them into some of my livecoding.

YS: I was looking at the video that we played in October 2019 at Pioneerworks House called ‘Can We Play: Machine Learning Literacy Salon’ that Molmol Kuo (ITP ‘11) curated. The concept is that we’re playing, and the light bulb is listening to and learning from our beat and blinking with corresponding musical patterns while sonifying the patterns. In this process, we used Magenta, a machine learning framework for generative music by Google powered by TensorFlow. So it’s a three person performance.

KoalaTokki (Michael Simpson, Yeseul Song) <Can We Play: Machine Learning Literacy Salon> at Pioneerworks House, Governors Island, NY – (curated by Molmol Kuo)

*An invisible player represented with a light bulb joined Koala Tokki for a live code music performance. The third player listens to what we’re playing and accompanies melodies based on Magenta, a machine learning framework for generative music by Google powered by TensorFlow. Whenever the third player makes a sound, the light bulb is being activated. (2019, Courtesy of Yeseul Song)

I really like having the physical object as a part of the performance because I’m naturally inclined to physical stuff. So I have some ideas about what kind of objects I want to work with or what I want to be able to perform. At one of our performances at Mana Contemporary, we repurposed and live-coded abandoned floppy disk drives to use them as musical instruments to accompany digital sound in the performance. There was one livecoder, probably at Wonderville, who brought a 3D printer and amplified the sound and had cool visuals. I’m interested in further developing work that combines physical computing and livecoding.

Another thing that I’m interested in is using this as more of an education tool. Because I teach, I naturally think about new ways to engage students. Especially when we work on collaborative projects, livecoding is a cool tool for learning about the concept of code. Fox Dot is pretty intuitive to learn and use compared to others and on the basic level, getting things working is not as hairy as others. Learning about the concepts of pattern and rules can lead to some sort of collaborative performance workshop where people can play together.

JH: I’m currently interested in participatory performances and blurring the border between the audience and performers. In Korea before I came to ITP I made music interfaces with a group called mididice and we built websites that create music or games that create music. I’m still interested in this where the audience doesn’t stop at just listening to the music, but can also participate and do something to the music together.

My current project is making a participatory performance with web browsers on the mobile. It’s not related to livecoding now, but when I finish this mobile web browser participatory music project, I’m thinking about building on it and making a way to let the audience participate in livecode. Because livecoding is on the fly you sense that it’s not recorded. Although you might not decipher 100% of the code you can sense that this performer is actually creating the performance in real time. I think the ephemeral quality of the music is what makes livecoding really interesting. It only happens once and even the performer can’t remember exactly how they did it before.

https://player.vimeo.com/video/682325657?h=74377729e5&title=0&byline=0&portrait=0
Caption: Jeeyoon Hyun (ITP ‘22) performing as herself at the NIME presentation in December 2021. The audience is performing with Jeeyoon’s web browser + Video: Julia Margaret Lu


JML: Yeah I know, I always feel like everything needs to be recorded when I’m practicing. You hear something good and then it’s just gone, and you don’t remember the sequence and then you lose it.

DS: I think that’s actually probably one of the most important parts that Jeeyoon just said. The act of it being created in front of you and you can see it being created. In today’s world with electronic dance music you go to see Skrillex and he’s already got every single clip programmed onto his Ableton device and he just pushes a button and stands on the table. You can’t do that in livecoding–or maybe you could–but it still has to be your lines of code. You can see the line of code that built that beat as opposed to Skrillex pressing one button.

YS: Like a cold sandwich versus a warm sandwich that is made in front of you.

DS: Yeah, exactly. [everyone laughs]

JH: The sense of being on the fly is also why the livecode community is really kind of open to failure and embracing failure. You just do stuff again and again and you can deceive everyone of different skill levels because it’s supposed to be on the fly. Failures happen and nobody can be perfect and I like that the most.

Poster design by Dave Stein, 2019. Courtesy of Yeseul Song.