https://youtu.be/PR42Sxhw9O4

Description

Nowadays, the interaction in live streaming is limited to verbal comments either on top of the content or at the side. However, streaming as entertainment, its content should be a joint effort of both the streamer and the audience. This project aims to find a new medium that could primarily extend the interaction dimension while streaming and watching streaming content. Therefore, Augmented Reality (AR) will be the ideal medium for streamers and audience in this project.

In detail, this project will add three interactions:

1) Tap & Text. It is like traditional verbal comments, but users(audience & streamers) can leave comments on any physical location. In this way, the comments should have a stronger connection to the surroundings/context.

2) Tap & Place. It is like the Tap & Text, but it places 3D models. Later, I want to make those models interactable to users by just getting the user's figure gestures from the camera.

3) Tap & Play. Instead of placing models and texts, this interaction will place multimedia(Video, Gif, and even other Streaming Content) to the physical location. The above two forms of interaction aim to connect the streamer and audience by offering a mutual interactable item ob both sides.

The whole idea of this project is to bring more dimensions to the live streaming interaction system and allow more audience to engage in the streaming content. As T.L Taylor comments on live-streaming by saying, “It is a form that plays with the boundary lines between audience and producer.” I hope this project could show the futuristic view of what audience-oriented live streaming could be.

IMA/IMB Shanghai

INTM-SHU.401.3

Capstone Studio (Shanghai)

Mobile,VR/AR

https://youtu.be/YdMwjbrhpM4

Description

NEU is a mobile game aiming at publicizing the little-known stories behind our everyday decision making. Backed by NYU famous neuroscientist Paul Glimcher, we adopt a playful approach to unravel the secrets of how different brain parts work collaboratively to make optimal decisions. Users will be able to test the beta version of game while learning the neuroscience firsthand.

IMA/ITP New York

ITPG-GT.2974.001

Playful Communication of Serious Research

Mobile,Play/Games

https://www.youtube.com/watch?v=sSuCb1i4HoU&feature=youtu.be

Description

Diary: Chroma is an app that allows you to pick up and quickly save colors from a live camera screen. When you need to find the perfect color for a project or simply want to see what colors have caught your eye, you can open your diary and see all of your saved colors.

IMA/ITP New York

ITPG-GT.2180.001

Mobile Lab

Mobile,Product Design

https://youtu.be/K1oP1jwbKFQ

Description

Surrounds is an app or a solution that manages chats and conversations while doing live streaming. It adopts AR technology to categorize user comments and binds them to specific items in the streaming to make conversations organized.

Currently, there exist several ways of interaction during live streaming — Virtual Gifts, Comments Systems, or Danmaku Systems. However, all information or chats are totally wrapped up and mixed together. They are vaguely bound only to the streaming itself, rather than the specific topics inside. Survey shows that most users find it annoying when over 25% of the screen is covered by danmaku (a set of scrolling texts floating on the screen), especially when the texts are little related to the topic. So, when the streaming is covering large scale of topics or presenting complicated ideas or scenes, chats are completely garbled and hard to locate.

So, I designed Surrounds, which uses Google ARCore to detect and track objects in the streaming in order to categorize the ungrouped messages into various chatting groups. By using Firebase Realtime Database, the messages can be synchronized along with the streaming data and kept permanent. When the user opens a live streaming session, the application will automatically find objects on the screen and attach a clickable indicator on them. By clicking the indicator, the user will join a public group where all the messages here are specifically related to the object. This design allows users to receive only the intended information they are interested in, and hence reduce the spam of information.

Although the system is not perfectly designed and lots of other issues need to be concerned, it sophistically provides users a structured and clearer view of information during live streaming. It makes the users target the topic they are favor of and hence helps the users refine their inspiration on streaming.

IMA/IMB Shanghai

INTM-SHU.401.3

Capstone Studio (Shanghai)

VRAR,Mobile

https://expandingexperience.kr/Bobst

Description

“School libraries exist throughout the world as learning environments that provide space (physical and digital), access to resources, and access to activities and services to encourage and support student, teacher, and community learning.”

– IFLA School Library Guidelines, 2015

The Elmer Holmes Bobst Library is the main library at New York University, the core place where research takes place. Students and faculty visit the Bobst Library for their study and to interact with library functions as listed by the IFLA guidelines.

When I visited Bobst Library last semester, I had an unpleasant experience, where I spent a long time wandering around the library trying to find the right room and right bookshelf. There was no clear physical or digital way-finding indicator. This experience made me wonder if there are ways to help students and faculty navigate the library more efficiently. This marks the beginning of the project, Bobst.

Bobst focuses on the core function of the library, which is searching and finding a book.

The app shows you where to go and where to check, you can also keep and manage a record of the books you want. Furthermore, you can check the seat availability in the library to study. The original website asks users to open a new page to find book information, while the app’s intuitive user interface design reduces the steps it takes to find a book.

IMA/ITP New York

ITPG-GT.2099.00002

Thesis Part 2: Production

Mobile,Product Design