Matt London

Paper Dolls

Use clothes to communicate with a video model in this virtual closet.

Introduction to Computational Media

Everyone has anxieties about shopping for clothes. A few months ago, my wife shared the way she avoids the stress. "I go shopping with friends. When someone you relate to is there in the trenches, who tells you those pants look good, or saves you from buying a shirt that clashes with everything, it makes shopping way less stressful, and a lot more fun."

Wouldn't it be great, I thought, if there was some way to provide this service to everyone, any time they wanted? Thus, Paper Dolls was born. In this game, the model is always in motion, smiling, ready to wear new outfits you would like to try on. The user can drag from an assortment of shirts, pants, skirts, and dresses, and the model will respond as a friend would -- with compliments, jokes, and playful teasing.

There are many benefits to trying clothes on a video Paper Dolls model, from seeing what certain outfits look like on a real person, to discovering all the funny things the model says. Possible real-world applications include interactive billboards, virtual in-store personal shoppers, and downloadable fashion apps to your smartphone or tablet.

I filmed Paper Dolls in front of a green screen with the Canon 5D. It was programmed in Processing. The completed version will have five tops, four bottoms, five dresses, three pairs of shoes, and a jacket. There are nearly fifty different combinations of outfits, and over 200 lines of dialogue. Amazingly, the shoot only took six hours, which tells me that it would be easy to regularly update the program with new models, outfits, and phrases. The system of interactive video that I developed has incredible potential outside of Paper Dolls. Using the basic engine of the program, you could create interactive training videos, choose-your-own-adventure movies -- the possibilities are endless.

Since the beginning of the semester in ICM, I have been trying to create programs that allow for the progression of various states. One early project emulated old text adventure games. I used key presses to print various strings, which told an interactive story where the user decides the outcome.

For my midterm, I did a similar project, but incorporated photos and level counters. In this game, you have a conversation with a virtual person who asks questions, and based on your responses, the virtual person's mood improves or sours.

With Paper Dolls, I went another step forward. I used video instead of photos, and a spectrum of choices instead of the binary choices in my midterm. No two games of Paper Dolls are the same.

Anyone who shops for clothes, or wants a fun shopping experience trying on clothes with a playful friend.

And, candidly... people who like playing dress-up with cute models.

User Scenario
The user approaches the screen with Paper Dolls and it is already running. The user can pick right up or hit "Reset" to start the game over. The model tells the user that she can pick any clothing she wants and try it on her. The user can drag shirts, skirts, pants, or dresses on or off the model, to create dozens of different combinations. Instantly, the model's outfit changes, and she talks about what she's wearing. The user can then tap the model to hear more informative and funny things about the outfit.

Paper Dolls was filmed on the Canon 5D, with post production in Final Cut 7 and After Effects. The game was programmed in Processing. I used HashMaps to load nearly 400 video clips, and to direct the program to instantaneously switch from one video clip to another.

Currently, the user uses the mouse to click and drag clothing on and off the model, but in my final version, I am going to use OSC to create a touch interface.

I learned that if your code is sound, adding new data is relatively easy. Once my HashMaps and associated functions were in place, adding new articles of clothing was a breeze -- you just drag in the video files and PImage of the clothing object, and copy/paste a few lines of code.

I also learned about Big O notation. The hardest part of using video in Paper Dolls, rather than a computer generated avatar, is the sheer number of clips you need to film. The model was limited in what she could wear. There are only 20 articles of clothing, but that means almost 400 clips, and much more than 20 combinations of those articles. I worked around this by having some outfits that only worked as a set, top and bottom, and also having unlockable pieces of clothing, like shoes and the jacket, which only work on certain outfits.