— Directing Virtual Reality

Chris & Penelope’s Post

Read More

Janet Murray’s work is tremendously well delivered piece of writing. I have a feeling I will be rereading this book more than one time. I would like to share some initial thoughts about ideas presented in a book.

In one of the chapters Ms. Murray mentions the importance for the interactive piece of art to have an in-built desire. I might understand it slightly different, but I find this statement truthful for any work of art. In a way, the desire of the art work is its own pleasure form itself which creates this endless flow of energy capable of hypnotizing the viewer, hooking the spectator, inducing the state of trance  for a second or eternity. That’s how I felt when I saw Vincent Gogh’s “The starry night” and Andrew Wyeth “Wind from the sea”. It is something you cant put into words, yet every time I remember this paintings I feel it all- the unspoken, full of desire. Technology is perceived (at least by me) as something very precise and structural, yet when author mentioned the ability of the artificial intelligence to contain this innate energy, the engine that pushes, engages, takes in viewer I thought that we indeed live in the age of breathtaking progress. I got so inspired: maybe the next “starry night” will be a VR experience that reimagines sky, humans and the world as we know it-leaving us in a land so familiar yet so unexplored. To be continued…

Read More

Sunset (Alex/Penelope) – great location, time-lapse worked amazingly well.

Brooklyn Bridge (Dimos/Lauren)- location has a sense of opportunity. Anything can happen on the bridge. I felt engaged and experienced anticipation.

Ice-skating (Ashley/Oriana/Chris) – very dynamic and unexpected. The piece made me think more about the possibilities of 360 moving camera.

Trapped (Swapna) – the location was perfect for 360 video. Music really made the whole experience so rich and satisfying.

2017: Alice’s Odyssey (Lauren/Swapna) – very engaging and evocative piece. The locations and production design enriched the story and left me with a feeling of something magical.

Enter the void (Oriana/Mary) – the dreamy state was achieved, the performance was believable and overall atmosphere left with feeling of altered state. Location was amazing.


Read More

moodboard: http://pin.it/nXMz03Q


schedule: https://docs.google.com/a/stern.nyu.edu/document/d/14Yo-ZzBorE19BAYfBfJCSJ0IogulThnIJ8KtjLFUASQ/edit?usp=sharing

Read More

We recreated the scene from Hannah and Her Sisters, focusing on trying to create a VR camera headset that puts you in the place of the actors.

Read More

In the recreate a scene assignment, Peter, Alex, and I experimented with trying to create a VR camera headset. The purpose was, similar to the binaural head recording device, to put someone in the body of one of the actors. We wanted them to look down and see their body, to have their hands in the space, creating greater presence (even if they couldn’t interact with the world). We went about this by taping two theta cameras to a hat, one in the front, one in the back. We filmed the scene from every character’s perspective in this way, with the hope of editing in between them. We were able to stitch the two images together in Premier by simply overlaying the two video feeds and using masks, and then changing the offset so that they better fit. However, there was a problem with the headspace, which meant that there was a gap in the space covered by both devices, making it so we could not get the right stitch line. In the future, you will need use cameras with greater than 180 degrees of capture in order to use this method.


I’ve been experimenting a lot in tiltbrush as well. I’m very interested in the idea of “drawing the world” creating objects in tiltbrush that would make up a VR space: trees, flowers, etc. It is much harder to draw than I expected (I’ve never been a good drawer in 2D space, it’s harder in 3D space). The idea would be to import these things into a Unity environment and create something where a user could walk around in the space. I’m trying to preserve the drawing look as a style, instead of importing more photorealistic objects. Still working on it.

Read More

Here is our initial documentation

Read More

Storyboarding in VR (like everything else VR) is an ever-changing, totally flexible art. How you do it really depends on what best suits your specific story / experience.

What’s fruitful to do first, no matter what your story/experience is, is to write out a user walkthrough. This should describe the experience — in detail — that at least one user might have from end-to-end (even if it’s not the only possible experience—which in VR it never is). I think it’s easiest to do this is by imagining that you just tried the experience and you’re retelling it to a friend. Make sure to take into account the physical presence of the user/viewer, and the emotion you are trying to elicit (if that’s appropriate). For instance:

I walked into a dark room and someone handed me a headset. When I put on the headset, it was dark for about 10 seconds. I heard a sound coming from the left headphone, so I looked to my left and I saw a video of shadowy figure. Suddenly, there was a burst of confetti from the ceiling, and at first it shocked me, but eventually there was 3d confetti flying all over the place and it was really amazing.

This allows for a few things things. First, it is the step one in clearing up any confusion about what is going on in the headset (and out of it). It also centers the experience around a user (which can be tricky, esp if you’re coming from film). Even if your user doesn’t have a role in the environment per se, VR is always an embodied experience. Last, if you are trying to make something with an emotional/story arc, this is a way to see what the core of that arc is.

If you’re going to be doing something in the realm of linear 360, a standard storyboard on paper can be super useful, with a sort of top down view like this one, that splits the circle into four quadrants, keeping in mind the lessons from Jessica Brillhart about how shots will align at the cuts. Feel free to annotate with text, and if you’re going to be moving the camera (virtual, or physical if you’re shooting with a 360) make sure to be clear about that.

(image from this awesome article on VRpop)

Always keep in mind that the user will only be seeing  only a portion of the entire environment, that matches the size of the field of view they are looking through (be it a headset, or panning on a screen), and that that field of view is ALWAYS dynamic (so it shouldn’t be thought of as a frame).

It may also be very useful — especially if you have an installation element (which to some degree, you ALWAYS do with VR)  like a controller that engages the physical body as WELL as the virtual body to show both the “in VR” and “out of VR” view. This can be especially important if the user is going to moving around (either in virtual or physical space) or controlling something with their hands.

To actually test a quick drawing out in a headset, you can draw something in photoshop on a 2:1 canvas (or even take a photo and scan) using something like this for reference. I wouldn’t worry too much about accounting for the top and bottom if you use this method (so you don’t have to worry about the distortion), unless looking up and down is integral to your story:

(image from tmarrinan’s github)

Pop it into premiere, edit it into a slideshow, and view it on a headset. It won’t be perfect (especially at the top and bottom), but it will be quick! This will give you some insight into stuff like blocking and timing if you’re making a linear experience.

For an even lower tech solution — or if you want to rapidly prototype something that may have some interaction, you can even get silly and try it with a really long sheet of poster paper that you tape into a cylindrical shape and hold around a user tester’s head. This affords mocking up some interactivity, too. For instance, if you want to change the scene once the viewer looks at the clock, you can wait till the viewer looks at the cake and then swap it out for the next cylinder.


(This would be an interesting thing to try for prototyping for stories you make using a tool like WondaVr, which allows for adding hotspots to that trigger cuts. If I recall correctly, Wonda might actually have built-in storyboarding, but sometimes paper does the trick in the earliest phases!)

Now, even if you test using the above method, adding interactivity means that you’re going to want to have a system diagram of some sort. I made this quick one using a tool called Gliffy. This is going to vary a lot depending on what your experience is, but having something like this (maybe even integrated WITH your scene) will be super helpful — AND it gives you a jumpstart in organizing your project in a way that is more code-able!


If you’re going to be working in true 3d (as in, with volumetric 3d models), and especially if your piece is highly interactive, you might want to think about storyboarding in 3d by using objects (legos work really well! — or just postit notes in a pinch) to represent the environment and the viewer. Write the properties of interactive objects on postits, then tap in to your inner child to do some ~*make believe*~. Can the users pick up the objects and move them? Do they play sound when they are picked up? Your storyboard can be a series of pictures of your make believe sesh, or even a video you can cherish forever.

(imagefrom sidlaurea.com)

There are also plenty of digital tools being developed for prototyping VR — some of which you make on your regular flat screen, some of which you can actually do IN VR. I find that tiltbrush is a great tool for brainstorming 3d spaces, or even 3d user interfaces. That’s for another post! And, to be honest there is nothing like the tried and true quickness of putting pen to paper.

The most important thing to remember is that even though the techniques differ from traditional storyboarding (and even from typical UX design, since physical reality plays such a critical role) — their use remains the same: to help refine your idea though really imagining it, to communicate  your idea effectively to your team and others, and to help you plan your next steps in production.

Excited for you all to share your methods!


Read More

Murch pins editing success to taking the viewer where she wants to go next when she wants to go there. The trick to editing in VR is finding the new rhythms, the new seamlessness.

I really enjoyed Brillhart’s take on VR editing being a transition between concentric circular worlds. The image she presented of this made me start thinking of tree rings and how they represent that movement through time, that experience of different worlds one after the other for the tree. When you go back to look at these rings, you can piece together the story of that tree, the journey it has experienced.

Unlike traditional editing, you are editing for multiple viewer paths through the experience. That is what is exciting to me about VR–give the viewer ownership of the experience. It’s your job as the editor (just like Murch) to really understand and anticipate what all of those potential paths are and aid the viewer on their journey along the path so that they don’t get lost. Aligning offsets for points of interest is a key method for creating match of attention cuts. However, you have to plan for all eventualities (make sure the cut works in all 360 degrees) to anticipate journeys you have not thought of.

The key will be to put yourself in the viewer’s seat. Really explore the world of your story. Where do you want to go? Where could someone else with different world experiences want to go?

I just wish there was a way to editing in a headset so that you can make changes as you watch the experience and feel things are off.


Read More



Above is the link to the Google Slides “Interactive Storytelling” presentation from the March 8 class.



Read More