Jim Schmitz

I designed and implemented a new approach for applying style transfers to 360 pictures and video. Using contemporary and historical paintings and images from Google Street View, I created art that encourages viewers to appreciate what is beautiful or poignant about the modern world, right in front of us, and to look at it with new eyes.

Description

Applying a style transfer algorithm to 360 imagery allows us to create immersive experiences that provide the feeling of being inside a painted world.

A style transfer is a computational technique involving a neural network that can re-imagine a photograph in the artistic style of a selected painting. A 360 image is a panoramic (spherical) image that provides the viewer with the opportunity to look in every possible direction instead of the single direction provided by traditional fixed-view images.

The standard style transfer algorithm can be applied to 360 images, but the result has undesirable artifacts because the algorithm doesn’t work well with the distortions of the equirectangular projection used to store 360 image data. For my thesis, I developed a new approach for applying style transfers that eliminates these artifacts and properly transfers a painting’s style to a 360 image in an even and continuous fashion.

Google Street View is an expansive resource of 360 imagery capturing the modern world. In aggregate the images are a reflection of every way the world could possibly be described: beautiful, distressing, amazing, opulent, disorderly, tragic. This richness and complexity is often overlooked by people using the service for mere navigation purposes. By applying my style transfer algorithm to these images, I can draw attention to what is remarkable or meaningful about these locations and encourage viewers to look at them with new eyes and to consider the actual physical locations they depict with a new perspective.

Classes

Thesis