Story Interface + Illustrations

Big updates. This project kind of took over my life this past week since it’s bringing together a lot of elements I’ve been working on this semester.  So I’m really running with this and I think I’ll work on it long after ITP.

I started in a place where I knew Quill paintings/animations could be used as story elements. All semester I’ve been trying to figure out how to “paint” in AR, and it arguably took me too long to think of illustrating subjects that interacted with the environment. That pandoras box is now swung open and I’ve been living in VR for a few days cranking out illustration/animations.

To quickly recap, I’m painting illustrative animations in Quill, recording them (with a green screen), converting that video to transparent looping video (like a gif), and placing those videos in AR.

I was stumped for awhile on what to draw and for a while thought I would choose a specific story to build assets around. I started with the basics and created some basic background scenes (a mountain, a forest, some waves) and I realized each of these environments could be built out – instead of forcing a specific narrative, I would give users the option to add assets centered around the same theme to make their own story. The (tentative) themes are forest/woods, space, prehistoric, ocean, and city.

What really came as a breakthrough was structuring the project so that users could “drag and drop” which ever illustration they wanted to in AR. Before this I would spend hours in Unity trying to line up the illustrations in the right spot, build the app, test, and go back and make changes. Now those changes are live! (although the user control needs a lot of work).

I feel like I’m exponentially learning what works and what doesn’t as I’m building up familiarity in how to animate with this program (it’s extremely tedious). A few things I’ve noticed works well are subjects that “melt” into the ground or illustrations that have a similar optical illusion so it appears as if they’re integrated in AR (for example the Narwhale). So I’m still defining my illustrative style in this and building up each scene.

(On a side note – my workflow changed since this documentation so the line drawings will be more vivid, thicker and more noticeable. I used a particle shader that removed all black from a video, and that took some of the line weight away as well – now I’m recapturing these illustrations with green screen and removing it beforehand in After Effects.)

This upcoming week I’m building up the assets for each scene and making the user interface much better. I’d ideally like users to be able to scale and rotate the illustrations (and eventually have some capturing/sharing option).