aparrish@itp

Most of the progress this week was made on the score/performance plan. I've made some additions to what I turned in on Wednesday: the game now has a bit more geography and has a mechanism for awarding points. I've been practicing with this score and I'm actually really happy with the results: being in control of all the elements of the game has made it easier to sketch out a consistent performance.

The main problem I'm still struggling with is how to end the piece, since all of the actions I set in motion during the performance are continuous (or at least take a long time to fade out). The stopgap solution is to introduce a master volume control and just fade that shit out when I feel like it. A better strategy might be to send a "terminate" message over OSC when a win/lose condition is reached. I'll think of something.

Posted by Adam Parrish at 11:06 PM | Permalink | Comments (0)

Here's an excerpt of a Frotzophone session with some of the adjustments I've been working on during the week (~2min 192kbps MP3). I incorporated some of the droning noises that I was experimenting with last week, and added some code to eliminate old/boring threads.

I spent a lot of time this week just playing with the ChucK patch and making noise. I'm still not satisfied with the sound—in fact, I think it's getting TOO complex—but at least the patchmore stable than it used to be. I also worked a bit on the visualization applet, which is now almost usable (but still not pretty), and on some general documentation (hopefully we'll see Frotzophone in the Winter show).

Posted by Adam Parrish at 10:02 PM | Permalink | Comments (0)

This week has so far been devoted to experimentation and rehearsal. For "rehearsal" read finding bugs: one problem I identified a few days ago is that my ChucK patch gives out after about 47 concurrent threads (or shreds or whatever). This limitation turned a rather stirring headphone performance of Spider and Web into a stuttering fiasco. There's not much I can do to optimize the code at this point—each thread contains just a single sine oscillator going into an envelope generator—so I'm trying to think of ways to cut down on the number of threads I create, or some way to cull uninteresting threads. One idea is to look for and drop objects past a certain age that have only ever been attached to one parent.

Or I could just make sure that the story file I write and/or perform never plays more objects than my patch can handle.

One major change to the patch this week was that an object's note sequence gradually gets quieter unless some activity occurs that involves that object. This has the benefit of decluttering the sound a little bit, but has the side effect of making the piece time-sensitive: if I don't keep playing at a certain speed, the piece will fade to silence. I like this. I'm enjoying the added sense of dynamics and urgency. A little bit of analog control goes a long way.

I've also been experimenting with a few new audio textures. One idea that seems promising (though I haven't implemented it yet) is to introduce sustained droning sounds for parent objects. As an object has more children attached to it, its drone will increase in volume. (There will probably be some kind of lower limit to the number of attachments necessary.) This fits in pretty well with the mapping I'm already working with, and adds a number of musical possibilities.

Posted by Adam Parrish at 09:46 PM | Permalink | Comments (0)

The above is a screenshot from a very early version of the Frotzophone visualizer. The circles represent Z-Machine objects, and the white lines indicate parent/child relationships between those objects. Gray lines are drawn between any two objects that have been in a parent/child relationship. It's rough, and not very readable, but the basic components are there. (This applet, developed with Processing, receives information from ChucK over OSC.)

Eventually the dots will change color when the object in question is being played. The "root node" will also be hidden, as this is merely a convenience object present only in the Processing code.

Giving the prototype presentation last week was helpful, and I learned a lot about what I'm doing in the process of preparing it. Here's a quicktime file of the presentation, and here is the Z-Machine file of the quickie text adventure that I prepared for the presentation. Play it with Zoom or another interpreter of your choice; you can also download the source code.

Other work this week included a few experiments with volume and timbre. I'm still not 100% happy with the sound of the piece, but I'm wary of anything that will complicate the mapping between data and sound.

Posted by Adam Parrish at 11:54 PM | Permalink | Comments (0)

The above screenshot (click for a larger version) shows the primary components of my piece as they currently stand. The ChucK patch is open on the left; my hacked version of Frotz is in the center; and on the right you can see the ChucK status window, which is showing some of the data being passed from the interpreter to the ChucK patch.

Here is what the Frotzophone currently sounds like (3'31" 192kbps MP3). The game (and gameplay) that generated this sound comes from the very beginning section of Zork I. Appreciate this noise, for it is dearly won: I had to learn how to handle strings in C. Not fun.

How it works

Last week I explained a method of tracking objects in the Z-Machine—specifically, keeping track of when one object is "attached" to another. This happens whenever the player object is attached to a room (i.e., when the player moves to a room) or when an object corresponding to an in-game item is attached to the player (i.e., when the player picks up an item) or when a non-player character does either of these things.

The Z-Machine interpreter sends an OSC message whenever such an attachment takes place, reporting both the parent object and the child object. The ChucK patch checks to see if the parent object has been seen before; if not, it associates that parent object with a note from a pre-determined scale. That note is then (and on subsequent appearances of that parent object) added to a list of notes for that child object. The ChucK patch plays the list of notes for each child object simultaneously and in a loop; the effect is sort of an aural history of how objects have been related to one another.

Challenges

I feel confident that I'm closing in on both the technical and aesthetic aspects of the piece. The mapping of notes to relationships between in-game objects feels right, and I like the way it sounds. I still have a long way to go, though. Here are some pretty significant challenges and issues (just off the top of my head):

(1) I need a more effective way of explaining the piece and how it works. I feel like the underlying mapping is actually kind of intuitive (in a strange way), so I should be able to explain it without resorting to a full-blown exposition of the history of interactive fiction. I like talking about the technical details, but I need to be better prepared to explain it succinctly to less technical audiences.

(2) I think I also need some kind of visual representation of what's going on inside the ChucK patch—for the audience's benefit and my own. Fortunately, the data I'm getting out of the interpreter lends itself to this kind of display. I might be able to work up something real quick in Processing.

(3) The current mapping doesn't leave much room for the piece to grow and breathe. The sound gets almost overwhelmingly intricate after only a dozen or so moves; the different threads are hard to pick apart. I have a few ideas for getting around this—stereo separation (or multichannel audio!) is a good start. I'm also considering having the loops for objects gradually grow quieter and louder in response to how often the object is seen. Ideally, though, I'd like to map the timbre of the sound to the data in some way.

(4) The piece isn't as expressive as I'd like it to be. When I imagine my performance, I feel a little bit hemmed in. I'm considering maybe adding some extra-diagetic commands to the interpreter that would allow me to make changes to the sound of the piece—something to change the tempo, maybe, or to solo/mute loops for particular objects.

Hopefully I'll have made some headway on one or more of these issues by Wednesday.

Posted by Adam Parrish at 11:45 AM | Permalink | Comments (0)

This week was spent deep in research, both in the function and form of interactive fiction. I'm knee-deep in source code and documentation of Inform interpreters (and the Inform 6 compiler), of course, but I also took some time to familiarize myself with some interactive fiction theory—and to actually play some games (consider this a form of rehearsal). There's still no sound coming out of my ChucK patch, but I'm getting close.

Theory

My main bit of breakfast reading is The Z-Machine Standards Document, which defines how implementations of the Z-Machine should behave (this information was reverse engineered from vintage-era Infocom story files and interpreters, and then later used as the basis of Inform). The opcode table is especially elucidating. I think I have a better handle on what information is available to me as the story file is running in the virtual machine.

I've also been skimming the Inform 6 Designer's Manual, which is short on the kind of low-level technical details I need but contains no end of helpful information about what to expect from games designed for the Z-Machine (particularly those programmed in Inform)—and the practice of designing interactive fiction.

Nick Montfort's Twisty Little Passages has proven to be an invaluable source of theoretical insights, and has helped to fill in my knowledge of interactive fiction history. (Further notes on this forthcoming.)

Practice

At this point, I think the key to my instrument is going to be the Z-Machine's object tree. This is the component of the virtual machine most specific to the creation and interpretation of interactive fiction; its presence distinguishes the Z-Machine from its more generic brethren (e.g., the Java virtual machine).

Here's how it works: every Z-Machine game has a number of objects, each of which represents something tangible in the game—everything from items to locations to people to the player himself/herself. These objects are contained in a hierarchy which, by and large, corresponds to physical containment. For example, the child objects of a room are the items contained in that room; the child objects of the player are the items in the player's inventory; the child objects of a box are the items contained within it, etc. Player movement through the map is, in fact, usually represented in code as detaching the player object from one room object and attaching it to the next. These actions are all conveniently performed using Z-Machine opcodes, making them highly visible to my hacked version of Frobnitz.

This is a sufficiently generic method of getting my hands on the data that effectively makes up the "map" of the game; it's also probably enough information to automatically draw a graphical map of the as it's traversed. But the fact that it's generic is also a drawback: for example, because the player object is just one among many, its identity can't be determined simply by looking at the object tree. (Heuristics could be used, such as looking for the object identified with the word "you," but this would fail in some cases.) If I want to focus the operation of my instrument on player movement, I'll either need some way to interactively define this, or else limit the scope of the instrument's functionality only to games specifically written for it. I'd rather not succumb to the latter course of action—I really like the idea of performing Zork.

Posted by Adam Parrish at 11:55 PM | Permalink | Comments (0)

The screenshot above doesn't look all that exciting, but it's a big breakthrough: It shows my hacked version of Frotz sending Z-Code opcodes to ChucK using OSC. (The game being played is Adam Cadre's Photopia.)

Here's a breakdown of what's going on, for those of you whom the description above completely confounds (probably pretty much everybody). Frotz is a Z-Code interpreter; Z-Code is a variety of bytecode designed to run on a Z-Machine, which is a virtual machine (along the lines of the Java JVM) originally developed by Infocom to ease cross-platform deployment of their interactive fiction. (The format has remained a popular with creators of contemporary interactive fiction thanks to Inform, which compiles to Z-Code.) A Z-Machine implementation, like Frotz, looks and works a lot like a microprocessor emulator: it interprets opcodes that affect the state of the virtual machine in some way. My instrument will work by hooking into the Z-Code interpreter and sending OSC messages to ChucK (or some other OSC server that makes noise) based on the current state of the game.

The basics of that process are in place, and I've at least settled on an architecture, which is a big step forward. I'm not making any sound yet, though. Over the next week, I'm going to be reading up some more on Z-Machine internals, and trying to pin down those aspects of the interpreter that will best allow me to realize my initial idea; hopefully, there's some way to get a consistent snapshot of the player's inventory and location in the simulated space.

One central conceptual problem that I've yet to solve: interactive fiction is inherently turn-based, in that the program responds only to a complete unit of user input (e.g., "get lamp"), and is essentially inert until further input is received. This means that my ChucK patch will be receiving a huge chunk of data every time I press "enter," and nothing in between. The challenge is how (whether?) to convert this data chunk into a data stream that produces sound in a manner both aesthetically pleasing and (relatively) transparent to the audience. My current idea is to use a combination of buffering and slow attack times (e.g., buffer x bytes, play a byte from the buffer every 100-500ms, the sound takes 5-10 seconds to reach full volume), but obviously the optimal solution will depend on which data I decide to capture and how I'm capturing it.

Oh yeah, and the working title is now the FROTZOPHONE—aside from being the name of the interpreter I'm hacking, frotz is also a hacker term for "an unspecified physical object, a widget" (as defined under the word "frobnitz" in the Hacker's Dictionary). So frotzophone is appropriate. Dorky, but appropriate.

Posted by Adam Parrish at 12:10 PM | Permalink | Comments (0)

Here's my idea: A text adventure that makes noise. Why? Because I'm interested in maps. Let me explain.

From Somewhere Nearby is Colossal Cave

These two maps (pictured above) are both, after a manner of speaking, mapping the same thing: a cave in Kentucky—the Bedquilt region of Colossal Cave, to be precise. The top map is an overhead view, representing the spatial layout of the cave to scale. It's a map designed to help cavers navigate the space.

The bottom map is more abstract: it breaks the cave into specific rooms, and shows only the connections between them (not necessarily their physical layout). In fact, this map doesn't directly map Bedquilt cave at all; it's actually an excerpt from a schematic represention of a game called Adventure—widely recognized as the first "text adventure" game (play online here). Adventure was originally designed by Will Crowther, himself an avid caver and co-creator of the first map of Bedquilt Cave. Although Adventure has significant game-like elements, at its heart it's essentially a Bedquilt Cave simulator: an intimate recreation of the cave, designed to evoke some of the wonder experienced in its exploration.

Adventure was one of the primary inspirations for Zork (Infocom, 1980), which popularized the text adventure genre. A screenshot from a Zork play session is shown below.

Screenshot from Zork I

While the geography of Zork is fanciful, rather than (mostly) based on reality (as in the case of Adventure), the structure of the game retains a map-like quality. For this reason, some critics (such as Julian Dibbel) have called the Zork games—and text adventures in general—"interactive maps." This may or may not be a fair summation of the entire genre. But at the very least, I find myself drawing maps in the process of playing text adventure games—mainly as a navigational aid, but also because it's fun.

An excerpt from a map that I drew while playing Zork

The interesting thing about these maps is that they show the topography underlying the game. As the game progresses, structures emerge. The most obvious structure is the map itself: the area of explored territory grows, and as it does the connections between nodes becomes apparent. They form branching structures, loops, and rhizomes.

A map of Dungeon, predecessor to the Zork series (warning: spoilers!)

Another equally important structure is the path that the player takes through the map. This path is constrained by the map, but not defined by it: the player must choose which direction to go. Given a map that permits doubling-back, an infinite number of paths are possible. It's even possible to get lost. Movement is further constrained by the game's rules: puzzles that must be solved, obstacles and enemies that can hinder the player's progress, etc. A text adventure, In essence, is a "playable" map.

A "heat map" of Quake III level. Areas where players spend the most time (on average) are drawn with a brighter green color; purple dots indicate where shots were fired. From Orbus Gameworks.

The structures of such games—recursion, repetition, branching—are shared with music, as is the improvisatory nature of "playing" the space. My musical interface is about exploring these analogies. How can the process of exploring and make a map itself "map" onto musical expression? Can you play (in the musical sense) play (in the gameplay sense)?

The performance I imagine consists of the performer (me) playing a text adventure game, using software that has been prepared to react sonically to the state of the game. Possible variables for mapping include percentage of map explored, topology of the player's path, which items have been collected, location of items, etc. The text adventure map itself will be randomly generated, so each performance of the piece will be different. As part of the performance, I'll be drawing a pen-and-paper map of the game as I progress; this might be displayed to the audience (along with a real-time projection of the game in play).

From an aesthetic point of view, this process is appealing to me because it suggests sound with many elements that unfolds over a long period of time. Changes to the music can only be made by entering entire phrases of text, which confines the performer's control of the piece to broader, textural gestures. This is exactly what I'm after.

Continue reading "Concept Presentation: Mapping mapping and playing playing" »

Posted by Adam Parrish at 10:52 AM | Permalink | Comments (0)

Here's a short clip from a performance by Pelt, an Eastern-inspired drone band that includes guitarist Jack Rose. Here are some of the elements of their performance style that I enjoy:

• Small, careful, deliberate movement
• ... but there's still a lot of noise!
• Certain processes (such as the ringing bowls) are set into motion and left to operate on their own
• Gestures and patterns shift in and out of synch, defying a figure/ground distinction but sometimes emerging as a discernable unity (this could be planned or unplanned)

Posted by Adam Parrish at 04:25 PM | Permalink | Comments (0)

I love me some Vibracathedral Orchestra. What I like is the constant battle between foreground and background: your attention is equally served no matter where it turns. I also like the idea of a constant battle to maintain what ends up sounding very static. (See also: Pelt.) Here's an excerpt from a Vibracathedral Orchestra song called "Aeolian Cistern" off of Versatile Arab Chord Chart.

Posted by Adam Parrish at 03:29 PM | Permalink | Comments (0)

Here are some of the more unusual "uses" of music that I picked up on this week.

A demonstration of taste and ethics

This thread on popular community site MetaChat concerns "songs you love" but whose lyrics (or the message contained in the lyrics) you hate. The discussion uses music as a medium for talking about personal aesthetics (what kind of music you like) and personal values (what propositions do you agree with? what are your politics?)—the thread addresses (among other things) abortion, sexism, and race. It's an example of music being used as a way to negotiate personal and group identity.

Territorial music

My walk to the train station each morning lasts about twenty minutes. I listen to music while I walk. I'm not sure why I do this. Maybe to relieve the tedium, but that can't be the whole story—taking a different route or simply trying to be more aware of the environment could also do that. I think it might be a territorial thing: I'm trying to claim that space for me, trying to make it more like home, trying to get it to belong to me. I don't mean the physical space as much as the temporal space. If I listen to music, I feel less like I've wasted those twenty minutes.

(more after the break)

Continue reading "NIME: What I saw music doing" »

Posted by Adam Parrish at 11:38 AM | Permalink | Comments (0)