Posted on

Surrounds

An app or a solution that uses AR technology to manage user conversations while doing live streaming.

Jinzhong Yu

https://youtu.be/K1oP1jwbKFQ

Description

Surrounds is an app or a solution that manages chats and conversations while doing live streaming. It adopts AR technology to categorize user comments and binds them to specific items in the streaming to make conversations organized.

Currently, there exist several ways of interaction during live streaming — Virtual Gifts, Comments Systems, or Danmaku Systems. However, all information or chats are totally wrapped up and mixed together. They are vaguely bound only to the streaming itself, rather than the specific topics inside. Survey shows that most users find it annoying when over 25% of the screen is covered by danmaku (a set of scrolling texts floating on the screen), especially when the texts are little related to the topic. So, when the streaming is covering large scale of topics or presenting complicated ideas or scenes, chats are completely garbled and hard to locate.

So, I designed Surrounds, which uses Google ARCore to detect and track objects in the streaming in order to categorize the ungrouped messages into various chatting groups. By using Firebase Realtime Database, the messages can be synchronized along with the streaming data and kept permanent. When the user opens a live streaming session, the application will automatically find objects on the screen and attach a clickable indicator on them. By clicking the indicator, the user will join a public group where all the messages here are specifically related to the object. This design allows users to receive only the intended information they are interested in, and hence reduce the spam of information.

Although the system is not perfectly designed and lots of other issues need to be concerned, it sophistically provides users a structured and clearer view of information during live streaming. It makes the users target the topic they are favor of and hence helps the users refine their inspiration on streaming.

IMA/IMB Shanghai
INTM-SHU.401.3
Capstone Studio (Shanghai)
VRAR,Mobile
Posted on

What is a computer?

CPU's anatomy.

Liyu Chen

https://youtu.be/VNbDj7aVyW8

Description

In this project, I plan to make the working theories of how computers work more accessible to audiences without prior background. I designed and implemented a 4-bit Arithmetic and Logic Unit from scratch using 74-series Integrated Circuits. The circuit is controlled by a microprocessor hosting a website on the Local Area Network. After logging into the website, the user would follow the “machine code-like” instruction prompted, input the designated values, select the mode of computation, then submit the form. The circuit would show the inputs in binary form, then calculate, show the output in binary form as well. By following the instruction and seeing the binary results being returned, the user can understand better how a computer functions from a more “down-to-earth” perspective.

IMA/IMB Shanghai
INTM-SHU.130.1
Working with electrons
Education,Art
Posted on

Music Wheel

A rotating electronic music box.

Haorui Li, Liyang Zhu

https://youtu.be/wtYCK3cwIN0

Description

It is common for music boxes to be powered by spiral springs while the plectrums make the sound. However, we are thinking of the possibility of combining the idea of the music box with the electromagnetics we learned in our course – Working with Electrons. Our project is basically an electronic music box that rotates like a sky wheel. As a result, we named it “Music Wheel”.

This project requires a comprehensive understanding of the motors (especially the stepper motors which we are using in this project), NE555 timer, operations on Arduino, PCB design, and also basic circuit operations. To make a speaker produce sounds, one of the ways is to use a NE555 oscillating circuit. In order to make multiple frequencies of sounds, we decide to attach the resistances with different values to the different axles of the wheel. When the wheel rotates, we can switch the resistance connected to the NE555 IC and produce a different sound.

However, the trickiest part of our project, is the stepper motor. Basically, a stepper motor is a kind of motor that can rotate certain steps in response to the electric signal given to it. The advantage of this kind of motor is that it produces almost no noise (compared to brush motor). And to produce such a set of electric signals, we need to use Arduino and ULN2003 IC to make it work – a challenging work.

Lastly, as an art piece, it also takes us a lot of time working on the delicate design of the wooden structure of our wheel. We have looked through some of the world’s most famous sky wheels and learn from their grant but well-organized design.

Student information: Tom Zhu (tomzhu@nyu.edu), Harry Lee (harrylee@nyu.edu)

IMA/IMB Shanghai
INTM-SHU.130.1
Working with electrons
Art,Music
Posted on

EM instrument

We want to build a EM instrument showing the existence of radiation.

Chenglin Li, Jamie Wang

https://www.youtube.com/watch?v=Bzm4ZOfuaj8

Description

We design a box just like the mp3 music box to play the electromagnetic sound. We put arduino, the circuit and the charger in this box. You can put this box near the electronic device to hear the electromagnetic waves they generate. The higher the frequency of the em waves, the higher the sound the box produces. Different from the music, what you hear is the invisible em waves.

IMA/IMB Shanghai
INTM-SHU.130.1
Working with electrons
Music
Posted on

From Border to Border

An Immersive Short Story in Virtual Reality

Sarah Tahir

https://youtu.be/ZFpWjliTV-w

Description

From Border to Border is an immersive Virtual Reality experience that explores nostalgia, growing up, and being ensnared by the past. The experience takes the form of a journey in three parts: The Sandbox, The City, and The Sea. Throughout this journey, the visitor is both propelled forward by a vision of the sea and called back by an idealistic picture of the sand. In the end, the visitor finds the sea and realizes it was not what they envisioned. They are adrift and helpless, with only one way out.

Please visit the project website to watch the film and learn more. This experience is best viewed through a VR headset on YouTube VR, but may also be viewed as a 360 video in browser.

IMA/IMB Shanghai
INTM-SHU.401.2
Capstone Studio (Shanghai)
VR\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\AR,Narrative/Storytelling
Posted on

Haptic Breathing Guidance Device for Flute

A haptic interface for breathing guidance in flute learning.

Yinmiao Li

https://youtu.be/SX_Ju6qKnJ8

Description

Haptic devices have made improvements in music learning. They can resolve the problems of space and time constraints in traditional learning methods and reduce the difficulties of learning music notations and terms. Previous haptic systems for flute fingering position learning proved that the learning rate is faster, and forgetting chance shrinks compared with learning from videos. However, flute performance consists of fingering and breathing. And existing haptic interfaces deal with fingering positions only. Research on technological-aided breathing guidance is mostly visual guidance or audio guidance. Haptic interfaces related to breathing were not used as guiding methods but mainly used to measure the people’s body position while breathing and provide feedback accordingly. Therefore, I intend to expand the flute learning from partial haptic guidance to full haptic guidance. This research will design a haptic interface for breathing. And this breathing guidance interface is a wearable device that is used to haptically instruct flute learners only on breathing control for playing the flute. The device leads the flute beginners breathing at the precise point during the song. The research intends to examine the users’ acceptance of using haptic guidance methods for breathing. This study not only provides a more convenient and accessible learning method for people with interests in flute or wind instruments but also has the potential to be applied to breathing guidance in the medical field.

IMA/IMB Shanghai
INTM-SHU.401.2
Capstone Studio (Shanghai)
Education,Wearables
Posted on

Music Harry

A casual game to explore more about music

Yudi Jia

https://youtu.be/_PxYbcnqwKI

Description

This project is an interactive game to help amateurs learn more about music in an interactive and interesting way. Many music lovers want to further explore the world of music, such as composing their own songs and making musical accompaniments and arrangements but are restricted by their limited knowledge of basic music theory. However, existing traditional music theory classes are not that playful and interactive, and also need a long time to learn. Other musical apps imitating the piano/guitar interfaces only allow users to get a sense of one instrument. Thus, to help users learn and explore music in an interesting way, this project will illustrate music elements vividly using cartoon-like interface and characters. It is not gamification learning, it is a game which hopefully can let players know more about musical elements during their leisure time. While gamification learning focuses more on how to design the learning methods and material by imitating game mechanics, my project is purely a casual game which provides a bit of learning value.

This game is developed using Processing and p5.js, and imitates the interface of Super Mario so that users can quickly understand how to play the game. The notes and chords will be visualized as cartoon symbols or figures, floating in the air or moving on the ground. When the user captures the cartoon figures, musical knowledge will be displayed on the screen, and total scores will be updated each time the user interacts with the game characters. After playing this game, users will get a sense of musical notes (C,D,E,F,G,A,B), possible/popular chords that can be used in composing a music piece, and might also be able to design a chord progression for a set of melodies.

IMA/IMB Shanghai
INTM-SHU.401.2
Capstone Studio (Shanghai)
Play/Games
Posted on

Magic Wish

A magic space for children to create & cast their own spells, with the hope that they can learn from Positive Psychology embedded.

Quinny He

https://youtu.be/ITuAWD6XaIw

Description

We have many tools that help kids to learn: Scratch for kids to access programming, Variation Playground for kids to learn music, Cognimates for kids to understand AI concepts, and we even have a whole museum exploratorium for kids to explore scientific phenomena at their own pace. However, all these learning experiences focus on talents, which are non-moral (such as understanding physics). Seldom do people consider helping children develop strengths, which are moral and contribute to virtues (such as the ability to love and be loved). In the field of Positive Psychology, strengths development contributes to personal happiness in both childhood and adulthood. Can we design a learning tool specifically for kids to learn happiness? Located in a dark room, the project creates a magical space for children to create and cast their own magic spells. With the guidance of a magic book teaching children how to use our magic wand, children can use different gestures to control the floating spells projected on a giant transparent canvas and design a new spell for their own wish. For instance, a “heart” shape spell for the wish that “I want my parents to get back together”. This project draws inspiration from Positive Art Therapy, the art therapy for positive psychology interventions, provides our young audience with a magical space to learn optimism, overcome grief, and regain hope.

IMA/IMB Shanghai
INTM-SHU.401.2
Capstone Studio (Shanghai)
Education
Posted on

MediaMoney

MediaMoney provides simple analytics for any public Twitter account. With this handy tool, you can even estimate how much advertising money your favorite Twitter accounts can make with each tweet.

Hank D Wu

https://youtu.be/qo2QN_LnkZU

Description

The chief purpose of MediaMoney ties in directly with a relatively new phenomenon: the monetization of social media. Despite rapid changes in the social media landscape, such monetization has remained rather opaque to content consumers. While many consumers now understand that social media can be a real business that drives very real profits, there remains quite a bit of misinformation regarding the quantity of such payments. Some consumers tend to underestimate the revenue generated by social media influencers (likely because few formal transactions ever take place directly between the consumer and the influencer). However, others also validly overestimate revenue, especially after viewing content depicting the lavish lifestyles of such influencers. The reality is much more nuanced. Consuming “free” media on platforms such as YouTube, Instagram, and Twitter is a large part of an important value creation process. Thus, the primary objective of this project is to cut through the opaqueness and shed light on what truly happens.

IMA/IMB Shanghai
INTM-SHU.401.2
Capstone Studio (Shanghai)
Education
Posted on

Subjective Mapping: (Re)visit NYUSH and its memories

A digital subjective map of the 8th floor IMA space at the NYUSH Pudong campus, where users leave and share their memories attached to any location.

Zheng Zhang

https://youtu.be/plVQvrSiIhQ

Description

Distinct from traditional mapping/cartography which intends to represent a territory as close to reality as possible, subjective mapping aims at producing a map using one’s own physical sense, emotion and memories that are not necessarily objective. Subjective mapping can be understood as the cognitive process that one reproduces his/her “story of experiences” at a specific location through making a map of the place (Nomadways). In creating such a map, one “stores and expresses his/her emotions into minimalistic shapes”.

The physical territory of the IMA spaces, added with personal annotations, is a community-created subjective map. The idea of this project is born from the context that our school is moving to a new campus in the Shanghai Qiantan area, and the current building will no longer be accessible by the NYU Shanghai community.

This project is aimed at creating a digital heritage of the current campus with students’ memory attachments, so that we can go back and revisit our alma mater when we are not able to do so physically in the future. Given the COVID-19 situation, the goal of this project can be extended to provide a community space for students, fellows and faculties to bond with each other while the campus is inaccessible. The project utilizes 1) Mozilla Hubs as the frontend and backend to host both the virtual space of the Pudong campus as well as personal memory pins in various forms, and 2) Spoke by Mozilla to create the 3D model of the campus itself. Given time constraints, this project focuses on objectively mapping the 8th floor of the Pudong campus and intends to create a subjective map of spaces related to IMA. The project will allow users to immerse themselves in the building, interact with other online users, and to drop down “pins of memory” at any location.

In a broader sense, this project will enable the current and future NYU Shanghai community to have an accessible space to document their individual attachments about the Pudong campus. Furthermore, it can also be a place for people to share their attachments, and thus enhance connections within the community.

IMA/IMB Shanghai
INTM-SHU.401.2
Capstone Studio (Shanghai)
VR\\AR,Tool\\Service