@GhostfaceHaiku

Posted on May 15, 2013 by mjuzzi

@GhostfaceHaiku is a twitter feed that generates random haiku using Ghostface Killah lyrics. The project is a companion to one of my previous experiments, the Ghostface Remix Project. The inspiration for these projects is Ghost’s bizarre and seemingly random lyrics. For example:

“Lightning rod fever heaters, knock-kneeder Sheeba for hiva
Diva got rocked from the receiver bleeder
Portfolio, looking fancy in the pantry
My man got bigger dimes son, your shit is scampi
Base that, throw what’s in your mouth, don’t waste that
See Ghost lamping in the throne with King Tut hat”

The remix project plays with the idea of randomness by cutting the audio of the vocal track of a song and rearranging it to create something different and perhaps not any less decipherable than the original. This project deals strictly with the text of the lyrics. I wanted to create poems with a defined structure and quickly settled on haiku. I feel like there’s a great juxtaposition between the calm, contemplative tone of traditional haiku and Ghostface’s manically weird songs. Beyond that, just the image of Ghostface delivering a haiku makes me smile. I chose twitter as a medium to publish the poems not just because it’s a convenient way to share, but because it inherently implies a singular human personality behind the content, as if perhaps these really are coming from the man himself.

The Python application makes use of the context free tools from Adam Parrish to generate lines of 5, 7, and 5 syllables. I built out a context free grammar using words from Ghostface lyrics that builds out noun phrases, verb phrases, and combinations thereof. Each word is labelled according to is part of speech, number of syllables, and singular or plural context. I also added some interjections such as “yo” and “a-yo” which help keep the results sounding like they came from Ghostface’s lips. I’m using a cron job to kick off the script once an hour, resulting in a new tweet. The results have been pretty satisfying and I think plausibly Ghostfacian. The lexicon does need to be expanded as there tends to be a lot of repeated words. Long term, I think slowing it down to maybe a tweet a day would also make sense.

Peep the Github Repo

Tweets by @GhostfaceHaiku

Grey Album

Posted on April 10, 2013 by mjuzzi

I revisited my Kimye project, this time using Markov chains with the lyrics of the White Album and Black Album to create lyrics that hopefully land somewhere between The Beatles and Jay-Z.

I played around with both words and characters as the units for the Markov chains, as well as the order of the n-grams. The results weren’t all that interesting but I found some diamonds in the rough. Overall, the results tending to lean heavily towards Jay-Z. The Black Album file is twice as long as the White Album so I decided to feed the White Album into the generator twice, which helped a bit. I think this technique would work better with more input text. Maybe using the entire catalog of each artist would improve things. The sweet spot I settled on was words as a unit, 2-gram, max 500.

Some results:

I’m so tired I’ll have another cigarette
Why don’t we do it in the forhead
I went to school got good grades could behave when i no longer exist

Make your mother sigh
Oh honey pie my position is tragic
Come and show me the answer

Only God can judge me, so I’m gone
With voices out of Earth
Truth be told

Nonsense?

Posted on March 13, 2013 by mjuzzi

I’m calling the poetic form I devised Nonsense?. It’s a pretty simple restraint; no dictionary words allowed. My intention was to present text devoid of any traditional meaning, instead focusing on the phonetic possibilities of the English language. This is a step toward the impossible end goal of completely context-free text.

My program generates poems of between 6 and 20 lines. Each line consists of up to 6 compounds or “words”, which in turn consist of up to 3 fragments. The elements are defined as follows:

Fragment: random sequence of 1 or 2 letters, including at least 1 vowel

Compound: random combination of between 1 and 3 fragments, containing no more than 2 consecutive consonants, and is not a dictionary word

Line: random sequence of between 1 and 6 compounds

Poem: random sequence of between 6 and 20 lines

I wanted the fragments and compounds to emulate phonetics of the English language, which is why I require at least one vowel per fragment and no more than 2 consecutive consonants per compound. These restrictions are imperfect, but serve the purpose of practicality. Similarly the length restrictions on each element aim for text that can practically be consumed and spoken without excessive effort.

Example 1

iwy
noiru yqus aifiw e y dyy
aosi
e
ueu xuuj ufyi
vo guo u tozide piirfa
yvta o yyc va aoi y

Example 2

u
iby y u
odeo uuy e ype
ieco pywopy eyd suma
iheu
ok aek oaaq
oc ekyo e teiw u
icaiz geaol icu omdy saoer o
edmia y ij
o fu oiba ecuuj uoso y
y xy
liad iiak uuas ryruy ouo
qeebu ol ouh yui ew
ayla
oyy alab
y eagez
iy waui iii ytu
ahoow viciu o y izmu
maie o o yh

Example 3

qoa izy uuae
zewa uojo iya aoq
o y maio koif execo ui
o ew
yvyky u u yr uodu oaut
noiir ahio iz eui olugzu
riie
wysaun ixi y u
etuge uho
yuon iperyt egu une qu
ehyn

The goods:

Kimye

Posted on February 27, 2013 by mjuzzi

Our assignment this week was to create cut-up poetry using one or more source texts. I thought it would be fun to play with the words of Kim Kardashian and Kanye West, creating poems that playfully speculate how their child might talk. At first I thought I would use Kanye’s lyrics and excerpts from Kim’s TV transcripts but it was a little bit of apples and oranges going on there. Ultimately I decided to use their tweets, creating a more even starting point. I found some “best of” twitter compilations for each of them online and saved them to text files, which you can view here.

My first approach was to compose poems by pulling random lines, which yielded uninspiring results, especially since each tweet is a line unto itself, allowing for little variation from the original. I also tried composing poems of variable number of lines and length of lines, using randomly selected words. These results were a little more inspired. For example:

was think inappropriate! elevator tattoos
personally close than water
used everyone of hahahahahaaa of butt it sleep tight
drinking me haha though don’t better sweat it for
commercial sometimes do next into tight for
bucks hair long have is point same
in just george be my
on my a julius
is coke in medium they making they
so feeling and be how — full
butt elevator same bugs

Here’s the code for the above:

Another approach was to sort the words, which produced my favorite results. Just taking the words that begin with capital letters and allowing each letter it’s own line yielded the following:

A ALCOHOL Actually, Armenian, Armenian. Ashton At
BEFORE Bahamas, Bieber Boyfriends Brothers. But
Caesar Christ…fuuuuuuck Classical Clooney
Devil Do Don’t
EWW. Every
FUCKING For French Fur
George Gross
Halloween He Hilton Hollywood Hotel How Howie
I I’ll I’m I’ve IN If Imma Is It It’s
Jonas Julius Justin

LOL Leggings Let’s Like
Man…ninjas Man…whatever Mandel Mason My
NO Never Nick
Paris
REALLY Reggie
SMH! Seriously She Sheik Sometimes
TATTOOS THE Taylor. The There They This Train

WAY! We What Why

You

Here’s what that program looks like:

I also made another variation that converted every word to lowercase. I plucked out the lines for ‘K’ ‘I’ ‘M’ ‘Y’ ‘E’ and inserted some line breaks to yield the following:

kazoo keep kids killed kind know
i i’ll i’m i’ve if illuminati imagery imagery!!!
imma in inappropriate!
inner instead internthey into irony! is it it’s
mad made magic make makeup making mandel manninjas
manwhatever marble mason may maybe me me!
mean meant medium met might million misspell monkey monkey!
more moron motto much music my myself
yall year yo you your
elevator email emotional empty end enough
especially ethnicity ever ever!
every everyone everything eww exactly?

First Python Script

Posted on February 13, 2013 by mjuzzi

This week I created a script that takes a text input, strips away all the words of 3 characters or more, and outputs the remaining one and two letter words. The intent was to strip away the “meaningful” words and give a greater magnitude to the less glamourous elements. When read aloud the results sound like basic phonetic building blocks strung together randomly.

Here are some example poems. These are excerpts from larger outputs and I have added line breaks where I see fit.

Example 1

we as by to a to to in or To by
in so on be a of of To an If of my
my by to be it in is if is so of Or is he
so be
Of to in of So of of if to
to to So a of of to be be to
to on To A in of no it In be
it be is to Or be

Example 2

me to my
me me by me
in my to Is my
me my my my do I
am me my
Be as do My
me of my
If I

Example 3

I am
I be of my to in my To
I To to my so so as in to
I of In my on me on my To me of
I to
I do my
I me to my a in my To my my
I a to my on do my to my a in by my by my no
I in am ofis by by by to Do in to by to
I
I to do So
I my in
I my my my me to in
I in my
I on my to at of at
I to my to of
I up of
I of a
I my

Here is the source code:

Hive 240

Posted on December 17, 2012 by mjuzzi

Earlier this semester we conducted a focus group with the residents of 240 Central Park South. Although the residents were overwhelmingly happy about living in the building, one topic that the residents kept coming back to was the problem of noise pollution. The building is located in a busy area of Midtown Manhattan, right on Columbus Circle. There is also a significant construction project going on next door. Chris Egervary, Yin Liu, Alex Olivier, and I worked together on Hive 240, a lighting installation that attempts to convert the annoyance of noise pollution into a delightful lighting display. The display is composed of a series of hexagons that form a hive-like structure. Each hexagon has an LED inside. The LEDs pulse in sequence, with the rate of change tied to the level of noise outside the building. As the noise level increases the lights are more active, and conversely as the noise level drops the lights are less active. In this way the installation also serves as a data visualization.

We installed electret microphones at the front of the building facing Columbus Circle and the rear of the building facing the construction site. Each microphone is connected through an operational amplifier to an Arduino microcontroller. The Arduino program logs the analog value of the microphone input every 10 milliseconds and calculates an average value for each minute. The Arduino transmits that value via an XBee radio which is configured as part of a ZigBee wireless mesh network in the building. This mesh network consists of a local network of radios as well as the XBee Internet Gateway, which provides internet connectivity and allows us to log the data on the iDigi Device Cloud. In addition to data storage, the iDigi site provides statistical and graphical representations of the data over time which allows us to monitor the microphones remotely.

The lighting installation is installed in the lobby of the building and also contains an Arduino/XBee setup that utilizes the XBee Internet Gateway to access the noise data being stored on the iDigi Device Cloud. The Arduino program requests the audio level each minute and uses that value to control a series of 18 LEDs via an LED driver. The LEDs are embedded in a hive, which is made of a foam structure with wood finishing and plexiglass covers. The foam and wood were cut on a CNC machine and the plexi on a laser cutter.

Project sketch:

Project rendering:

Microphone, Operational Amplifier, Arduino, XBee setup. Inside the box:

Outside the box:

Columbus Circle mic:

Rear mic:

iDigi Device Cloud:

CNCing the foam:

CNCing the wood:

Fabricated components:

Installation rear:

Installation front:

Ghostface Remix Project

Posted on December 2, 2012 by mjuzzi

For my final I continued my Ghostface Killah remix project. To recap the idea… Ghost’s lyrics are weird, cryptic, and often non-sensical. For example:

Scientific, my hand kissed it
Robotic let’s think optimistic
You probably missed it, watch me dolly dick it
Scotty Wotty copper-tipped me, big microphone hippie
Hit Poughkepsie crispy chicken verbs throw up a stone richie
Chop the O, sprinkle a little snow inside a Optimo
Swing the John McEnroe, rap rock’n'roll
Tidy Bowl, gun hold pro, Starsky with the gumsole
Hit the rum slow, parole kids, live Rapunzel

I thought it would be fun to break lyrics from one of his songs down into chunks and rearrange them, then see if the new version made any more or less sense than the original. Rather than work with the text, I wanted to use the actual audio so I could end up with a real remix. I chose the track Mighty Healthy off the Supreme Clientele album because it’s short, doesn’t have a hook, and is bangin’.

I wanted the remix to preserve the rhythmic flow of the original as much as possible so I devised a system that breaks the vocal track into audio clips, analyzes each clip, then rearranges the clips in a way that tries to match the original vocal.

In my previous attempt I broke the vocal track into an arbitrary number of clips of equal length. This proved problematic because some clips started or ended right in the middle of a word or phrase. For this version I decided that each time Ghost takes a breath, that marks the end of a clip. I manually cut the track up along these lines, ending up with 64 clips of varying lengths.

Cutting up the clips…

Each of the clips was then run through a Processing sketch that uses FFT to generate a sort of “DNA” for that clip. As Processing iterates through the draw() loop the amplitude of each frequency band is exported to a csv file. I added the amplitude of each band for each data point to create an overall amplitude for that point. Then I assigned a rating of loud (“A”), medium (“B”), or quiet (“C”) to each point based on it’s overall amplitude, then concatenated the ratings of the points in each clip to form the DNA strands as strings. A problem with this approach that I was not able to correct is that the data points are not at consistent time intervals due to Processing’s not quite consistent frame rate, so when I do my comparisons later it’s not exactly apples and apples.

Some of the data…

DNA…

Last week I used Greg Borenstein’s dynamic programming library to compare the DNA of each clip to all the other clips, looking for the longest common subsequence of characters. I swapped each clip with it’s best match, making sure each clip was only used once. That worked well, especially since all the clips were the same length. Now that I had clips of varying lengths that approach seemed to be not ideal. Instead I set out to create complete sequences using every clip and compare the DNA of the sequences to the DNA of the full original vocal track, looking for the best match. The problem is that with 64 clips and over 2840 characters in the DNA, the number of possible sequences is enormous and the work to compare each one is significant. I tried using Dan Shiffman’s genetic algorithm example but couldn’t seem to make it work for me. Instead I created a sketch that generates random sequences and scores them, adding 1 to the score for each character match. Each clip was only allowed once in each sequence, and was not allowed at the same index where it appears in the target sequence. The maximum amount of sequences I was able to successfully run at a time is 100 and even that caused the sketch to choke more often than not. Ultimately I ran 50 random sequences at a time. Therefore I was only able to test a tiny fraction of the possibilities over the course of a few hours.

I took the sequence with the best score and manually re-ordered the audio clips accordingly, layered the instrumental beat underneath and voila!

Remix

Lyrically I think it makes, or doesn’t make, about the same amount of sense as the original. As for the flow, there are times where the rhythm gets a bit awkward but a surprising amount of the track actually sounds locked-in.

For reference, here is the original vocal track.

And the remixed vocal.

Ghostface

Posted on November 19, 2012 by mjuzzi

I decided to give one of my dream projects a go. I worked with an a-capella Ghostface Killah track, dividing the audio into segments, comparing them to each other, and reordering them by swapping the best matches. Since the dynamic programming library isn’t made for audio, I developed a system where I converted the audio levels to text and compared the sequences of characters. I made three Processing sketches; one to analyze the original audio, one to compare the segments, and one to play the segments back in the new order. I also used Excel to work the data into something I could use. The result is pretty lame, but satisfying as a first pass.

Original

Remix

The Process:

1. Use FFT to analyze the audio level at each frequency band. For each frame, export playhead position and levels to csv file.

2. Aggregate frequency band levels to get overall audio “level” for each recorded playhead position.

3. Dumb it down… identify each position as loud(“A”), medium(“B”), quiet(“C”). Here’s a little bit of the spreadsheet.

4. Create segments to work with. In the name of deadlines and sanity I decided that 30 lines of data would comprise each segment. That gave me 26 segments, each a few seconds long. Here’s what that looks like.

Some of the segments start mid-phrase or mid-word so this is not a great way to do it. An improvement to this overall process would be to thoughtfully break up the audio as separate files as the very first step and work from there.

5. Run the dynamic programming sketch and find a partner for each segment to swap with, making sure each segment is included just once. My console output…

For example, as per the above, segment 4 will be played first and segment 0 will be played 4th.

6. Play it back. Cross fingers.

Download the whole thing, including the data.

Computer-Generated Watercolor

Posted on November 14, 2012 by mjuzzi

This week I read a paper about computer-generated watercolor. The system described simulates the look of watercolor painting, which is characterized by edge-darkening, granulation, back runs, separation of pigments, and glazing.

The researchers have developed models that represent the materials(paper, pigment, water) and the components of the the painting. They represent the painting as a series of “washes” which I think loosely correspond to brushstrokes. The wash model simulates the flow of water and pigment above the paper surface, the adsorbtion and desorbtion of pigment onto and off of the paper, and the absorbtion of water through the pores of the paper. The paper model contains height(think peaks, not thickness) and fluid capacity attributes that affect the flow of liquid. The fluid simulation is broken into three layers:

The shallow-water layer simulates the flow of water and pigment above the surface and takes into account the wet-area, water velocity in x and y planes, water pressure, concentration of the pigment, paper height, and the viscosity ad viscous drag of watercolor.
The pigment-deposition layer simulates how pigment is deposited onto and lifted off of the paper and takes into account the density, staining power, and granularity of the pigment.
The capillary layer simulates the absorbtion of water through the pores of the paper and takes into account the water saturation of the paper and fluid-holding capacity of the paper.

The images below communicate the above explanation quite well.

The pseudo code below summarizes the processes of creating a wash.

The process of rendering the pigmented layers is more of a mystery to me. It is based on the Kubelka Munk model. Each pigment is given absoption and scattering coefficients that affect how they absorb and reflect light. Here are some of the synthetic pigments they generated.

The researchers have applied their method to an interactive painting application and automatic image “watercolorization” applications in 2D and 3D. See below.

Lydia

Posted on November 10, 2012 by mjuzzi

Lydia Callis is Mayor Bloomberg’s sign language interpreter and she is amazing. My idea was to give the user an opportunity to become Lydia. I used a Kinect to gather depth images and principal component analysis to determine the orientation of the user’s hands. Based on those orientations I place an image of one of Lydia’s amazing facial expressions on the user’s head as per the chart below. I wanted to use the RGB image instead of points but found it quite difficult to line up the images with all the matrix popping, translating, and z-index action going on. Very frustrating.

Zip

Michael Uzzi

stuff and things.