Multisensory Music

Michael Simpson

A tool for creating audio/visual experiences that provides the composer with realtime multi-sensory feedback (audio/visual/haptic) tied to their physical interaction.

http://mgs.nyc/syntk/

Description

This project offers the viewer a series of different musical interfaces (software) and allows them to interact with both the audio and the visual elements which are being projected onto a screen. The idea is to provide the audience with a multi-sensory AV experience by tightly weaving sound, graphics, and physical gestures (via mouse/wacom/drum pads/faders).

Classes

Data Art, Interactive Music, Nature of Code (Animation and Physics), Nature of Code (Intelligence and Learning), Shader Studio

Holokit

FY Zhu

Hololens is amazing and meanwhile expensive. Now we proposed the HoloKit, this is just 3 dollar cardboard components in open source, and can turn your mobile phone into a powerful Mixed Reality devices and actually performance better than Hololens.For more information, please visit my documentation page.

http://itpzhufy.com/2017/05/03/holokit/

Description

Holokit is an open source mixed reality toolkits which can turn your mobile phone into a hololens style MR glass devices. Generally speaking, it contains several components as below:

1. Cardboard Kit: This kit is the main structure parts, it will provide the user with the stereoscope vision which is totally like what hololens did. The arrangement of the Cardboard Kit can be seen in this video here:

https://drive.google.com/open?id=0B0EKS8hqR17hSmxTS0FSWXZHXzA

2. Inside out tracking and environment detection. This part is the core function which will make the Holokit not to be just something head-lock experience but can make it really walking around, and meanwhile recognize the space in the world

3. Multiplayer network support. Here we also provide a spectator for our guest. For what the spectator looks like, please see the video below:

https://drive.google.com/open?id=0B0EKS8hqR17hVmxZSkx1anJhbUk

PS: For my final demo, I hope to invite my friends outside the ITP to help me during the process. He has also helped something during the design process. If you have any concern, please contact me via email.

PS2: If you think that thing can't pass, I can also demo another project based on Tango, video linkage here: https://drive.google.com/a/nyu.edu/file/d/0B7B5PKi4GojvUzNiR2M2bEd2WGc/view?usp=sharing

Classes

Nature of Code (Intelligence and Learning), Thesis