Through the Interface

Robbie Tilton

An exploration of future interfaces through gesture-based interaction, environmental computation, and non-traditional displays.



It is a very exciting time for interfaces. Depth sensors are getting cheaper and more ubiquitous, innovations in display technologies allow for transparent screens, curved multi-touch surfaces, and autostereoscopic imagery. Mobile devices are becoming faster and cheaper, enabling new wearable computation devices like heads-up displays and smart watches. But where are we headed? How is our relationship to computers changing? What does this mean for you and me? My thesis explores these questions through a series of interface experiments and provides an in-depth examination on interactions with computers, wearable devices, and volumetric displays.

Background
Research for this project included the topics: volumetric displays, stereoscopic and auto-stereoscopic display systems, dlp projectors, binary file formats, webGL, GLSL shaders, computer vision, gesture-based interfaces, human-computer interaction, the history of computer interfaces, and interaction design.

Books that I read include: The Singularity is Near by Ray Kurzweil, Make It So by Nathan Shedroff and Christopher Noessel, The Media Lab: Inventing the Future at MIT by Stewart Brand, Art of Human-Computer Interface Design by Brenda Laurel, Einstein's Dreams by Alan Lightman.

Articles that I read include: John Brown + Mark Weiser - Calm Tech, Mark Weiser - The Computer for the 21St Century, Alan Kay - A Personal Computer for Children of All Ages, Michael Halle - Autostereoscopic Displays and Computer Graphics, Gestures in Human-Computer Communication by Gordon Kurtenbach and Eric Hulteen, Talking and Listening to Computers by Joy Mountford and William Gaver, Illustions in the Interface by Chris Schmandt, Virtual Interface Environments by Scott Fisher, Why Interfaces Don't Work by Don Norman, User Interface Races by Ben Shneiderman, Through the Looking Glass by John Walker, Designing a New medium by Tim Oren, Nicolas S. Holliman, Neil A. Dodgson, Gregg E. Favalora, and Lachlan Pockett - Three-Dimensional Displays: A Review and Applications Analysis, Richard Bold - “Put-that-there”: Voice and gesture at the graphics interface, Dávid Lakatos and Hiroshi Ishii - Towards Radical Atoms – Form-giving to Transformable Materials, Tovi Grossman, Daniel Wigdor, and Ravin Balakrishnan - Multi-Finger Gestural Interaction with 3D Volumetric Displays

Websites worth noting include: Timo Arnall - No to NoUI, Scott Berkun - NoUI Debate is Rubbish, Frank Chimero - The Cloud is Heavy and Design isn't Invisible, Jared Spool - Great Design Should be Experienced Not Seen, Krishna Golden - The Best Interface is No Interface, Reena Jana - What Tech Trends will Affect our lives in 2020, Mark Wilson - Why ‘Intuitive’ Interface Is A Myth, Steve Mann - My Augmediated Life, Gregg Favalora - Hitting Every Angle with Autostereoscopic 3-D DisplaysVolumetric User Interfaces, 3D Displays, NYU Media Research Lab, Computer Vision Lab, Barry Blundell, MIT Tangible Media Group, 3D Lenticular Printing Tutorial, How Minority Report Trapped us in a World of Bad Interfaces

People I spoke with include: Eric Rosenthal, Shawn Van Every, Patrick Hebron, Tom Igoe, Ken Perlin, Michael Naimark, Gregg Favalora, Matt Parker, Sean Kean, Josh Nimoy, Kyle McDonald, Nathan Shedroff, Nick Sears, and Greg Borenstein.

Implementation
My experiments can be seen here:

- 3D Display (https://vimeo.com/59377788)

- WebRTC Headtracking with Lenticular Lens (https://vimeo.com/62731717)

- Iron Man 2 interface (https://vimeo.com/60357734)

- Optical mouse iPhone App (https://vimeo.com/61026976)

- Gesture-based interaction: Concept designed for use on the Google Glass (https://vimeo.com/65438813)