Posted on

Runway

Cristobal Valenzuela

Constant increases in computer power and the proliferation and access to large databases have allowed machine learning to rapidly impact our society. While commercial applications can be found everywhere, experimentation of new prototypes and techniques remains – for the most- confined to computer scientists or engineers and has a limited reach to other fields. My thesis will try to explore who can a digital tool simplify the approachability and exploration of machine learning models in other disciplines. I will specifically focus my research on giving better access to state of the art machine intelligence techniques to artists and designers.<br /><br />
<br /><br />
Given this framework, I will like to explore questions like:<br /><br />
<br /><br />
– How can modern machine learning models be accessible for people outside of the realm of computer science?<br /><br />
<br /><br />
– Can we have machines and humans collaborate in a generative process using tools that simplify how the two work together?

https://runwayml.com/

Description

Building on previous and current work I have been developing at ITP, I will like to produce a system that allows artists and designers to experiment with state of the art ML models. My thesis project, called Runway, will be a tool that will take the form of a desktop application for all major operating systems; MacOs, Windows, and Linux. Wrapping typical and common practices when handling and using machine intelligence architectures. Input manipulation, process, modification, evaluating and outputting will be managed and automatized. This will enable the creators to focus on handling the model the right inputs and on using the outputs the model will produce in any given way.

Just as Computer Vision (CV), another branch of artificial intelligence, gain popularity among artists in the early 2000s with tools like OpenCV, ML has the opportunity to have a higher reach among artists due to its general set of applications in image processing, text analysis, and content generation.

One main feature of Runway will be that it will not be focused on data pre-processing, training, transfer learning, architecture optimizing or any other characteristic of creating custom and personalized machine learning models. Instead, it will be centered on using pre-trained, proven and working models that can output constant values for given inputs. For example, some of this models might include: image recognition, pose estimation, feature detection, image segmentation, text and sound generation and general classifiers.

I envision the working prototype as a GUI that is compromised of a series of variable options allowing artists and creators to experiment and explore ML models. As part of this process, I would like to create a series of collaboration with featured artists, whose practice has not been influenced by this technology, and have them propose and execute small works with Runway.

Classes

Thesis

Posted on

Manifest Energy

Jasmine Soltani

A series of artifacts that explore the limitations and opportunities for ecologically sustainable computing, with a focus on the energy consumed in the production of our digital devices.

http://www.jasminesoltani.com/energy

Description

The lifecycle of a product includes the mining and processing of natural resources, manufacturing, transport, product delivery, use, and its reuse, recycling or disposal. Embodied energy is the energy consumed by all of the processes associated with production: the life-cycle up until the use phase. Manifest Energy is a series of lenticular stickers, a mechanical animation, and website that illustrate the life-cycle and embodied energy of digital devices.

Electronics are unique in that they have exceedingly high embodied energy, exacerbated by planned obsolescence, their difficulty to repair, lack of supply chain accountability, and the increasing ubiquity of computing. The energy economy, on which the ecologically destructive digital economy is based, is unsustainable in itself. Can illuminating the resource demand of our digital devices change our consumption habits, our relationship with our devices, the way we design and legislate their use, and handle end-of-life?

In addition to shifting the paradigm by which we think about energy consumption, I hope my project informs people of actions that can be taken. By extending the lives of our products through repair, we can reduce demand for new items. By developing more modular electronics, we can make these devices easier to repair. By putting pressure on companies and legislators, we can achieve greater supply chain transparency and improve the health of people working on or near production sites. By developing renewable energy sources and eliminating carbon emissions, we can mitigate the ecological impact of the production of digital technologies.

Classes

Thesis

Posted on

Applying AR to Cardiac Rehab

Ran TIAN

"Heart health" is an argument reality health platform to deliver education and improve medication Adherence

http://www.tianrandesign.com/

Description

Through a visually rich AR experience, the Heart Health helps cardiac patients understand in layman’s terms the impact of the medication they are taking directly on their heart health, and improves medication adherence by gamifying medication management; it also provides heart condition monitoring by integrating apple watch’s heart sensor.

Classes

Thesis

Posted on

Open Kinetics: Move, Play, Lean

Nathaniel Padgett

Open Kinetics are low cost kits that inspire learners of all ages to explore programming and electronics through movement-based play. The aim of Open Kinetics is to create an affordable, accessible, engaging platform that teachers can easily bring into their classrooms to foster technological agency in their students.

https://itp.nyu.edu/thesis/journal2018/stefani/author/ngp249/

Description

An exciting aspect of the rapid proliferation of technology over the past decade is that much of it is open and remixable by anyone with access to a computer and the internet. This has created unprecedented opportunities for anyone to contribute to and use these technologies for research, to start businesses, or to make art.

But despite these advances, access to technology is still largely concentrated in the wealthiest communities. A major reason for this digital divide is cost. Open Kinetics addresses this problem with engaging DIY technology kits designed to be affordable and accessible. Students learn programming, electronics, and design by making a simple motion controller used to play free movement-based games that they can also remix and build themselves. Through building these kits, they learn to use technology for their own creative ends, developing technological agency and preparing them for careers of the future.

By pairing hardware with games, Open Kinetics kits give context to what students can do once they have built their controller. Open Kinetics give students a virtually endless array of experiences to choose from, whether moving a character, playing virtual tennis, or making kinetic art. Because these games are movement-based, they engage students in active, physical participation that facilitates learning. Built for kids ages 8-16 but great for all ages, Open Kinetics turn players into makers and empowers students to contribute to rather than simply consume technology.

One simple project for Open Kinetics is a twist on the Atari classic Pong. In this game has players move paddles up and down the screen to deflect a puck and prevent their opponent from scoring. Rather than using joysticks or buttons, players use the Open Kinetics motion controller they build to move the paddles with their arm movements, making a sedentary game into an active one.

Classes

Thesis

Posted on

Ojo Oro – Thesis

Ari Melenciano

In applying emerging technologies to analog inventions, and developing human-centric, visceral business strategies, what are the possibilities for an experimental neo-retro camera company?<br />

www.ariciano.com/ojooro

Description

I am interested in exploring the art, philosophy and science of business and customer experience along with the possibilities when using design to engineer moments of euphoria by applying emerging technologies to the analog invention of a camera in the form of an experimental neo-retro camera brand.

Ojo Oro (golden eye) is a line of experimental neo-retro cameras. Neo-retro is the coined phrase that was developed in the creation of Ojo Oro, meaning to apply emerging technologies to analog inventions in innovative and explorative forms. The way that Ojo Oro is neo-retro is by its use of augmentation of the analog/film camera experience in a digital setting.

The way that the camera works is by using digital mechanics to take a photo, that is then immediately and randomly altered to appear as if it has been taken with a film camera. To further mimic the film experience, there is no screen on the camera, forcing the photo to be “developed” in the “cloud” and sent to the user's paired app.

This is both a hardware and software business.

The strategy of the business is to organically start from inward, out. What is the core mission of Ojo Oro? To allow users to explore their artistry (golden eye – Ojo Oro) in an intentional and well-designed holistic and immersive experience.

My audience are recreational photographers (people that enjoy photography as a hobby) between the ages of 15-30 years. These consumers also value design and style within products and their branding.

The camera will be used recreationally. Once the user has turned on their camera, they can begin exploring the different settings and taking photos. They eventually move to the app, where the photos have been “developed” and ready for view. These photos have all sorts of spontaneous film-like distortions on to the photos, whether it is a few light burns, or intense film grain. The photos can be immediately shared to social media. The user can also scroll through the app to discover photos taken by other Ojo Oro owners – building a community.

The key features of the Ojo Oro camera line include a product design of high aesthetics (intricate camera design), seamless communication between the camera and paired app, a neo-retro approach to film photography by means of digital, interchangeable lenses, innovative e-commerce opportunities within film, eventual personalization of camera design, and more.

In my first semester of ITP, during Pcomp, I was able to build the experiamera. I intend to return to the work I began then and further develop the prototype camera into an entire camera business.

Classes

Thesis

Posted on

Invisible Sculptures

Yeseul Song

Invisible Sculptures is an experiment that challenges human perception with sculptures that can only be “seen” by senses other than vision. The audience will incorporate different sensory abilities and activate their imagination to interact with the sculptures made of sound, temperature, and smell.

http://yeseul.com/Invisible-Sculptures

Description

There are five identical display plinths. Each holds an invisible sculpture.

Sculpture I is a solid 3d form made of sound. To experience the work, move your hand over the plinth. Concentrate all of your senses to find the connection between what you touch and what you hear. You will soon find yourself in the land of synesthesia. Once you are aware of the shape, try to recreate the form using a handful of clay that will be handed to you.

Sculpture II is an obscure pattern floating on top of the second plinth. Move your hand over the plinth to hear and experience the sculpture. Is it rough or soft? Is it dense or light? Is it spiky?

Sculpture III is made of heat. The sculpture accompanies a real time thermal camera feed of the audience interacting with the piece.

Sculpture IV is made of cold air.

Sculpture V is stinky… You might not want to get too close to this one!

(See User Scenario below.)

Classes

Thesis

Posted on

Commodity Fetishism

Miao Tian

The purpose of this project is to create an emotional linkage between overconsumption due to our social mentality and its harmful effect .

Description

Consumption is inevitable. It is in our nature and it is part of the whole ecosystem. However, excessive consumption is a social mentality. In our recent past, overconsumption was viewed as greed, a moral vice. In our current time however, greed is viewed rather differently. Living in our current economic society, we as consumers become further and further away from the production and waste department. Because of that consumption seems independent from the guilt of wasting someone’s hard work or polluting a piece of land somewhere. Consumers enjoy an exhilarating sensation by satisfying their greed.

For the project, instead of announcing statistic numbers of overconsumption and its effect, I’d like to have audiences experience it directly. I hope to generate an emotional linkage between our daily consumption life and the amount of waste we produce.

There are three players to participate three parts of this project: a shopper to a commercial shopping website; a consumer to a trash can controller device; a receiver to an interactive VR film. The shopping website represents the drive that pushes us to shop more and more. The trash can symbolizes the end of a product’s commercial life and the start of its waste life. The VR film visualizes waste that we don’t see. The shopper “buys” products and corresponding waste will show up in VR for the receiver. The consumer will step on a physical trash can and release the trash in VR. The receiver will endure the outcome, then the players can rotate and experience each other’s action.

Classes

Thesis

Posted on

Expressive Tactile Controls

Hayeon Hwang

What if physical controls such as buttons, dials and sliders had more expressive personalities? How could this inform and deepen our interaction with them? ‘Expressive Tactile Controls’ is a series of reactive tactile controls designed to react to their user in a variety of ways. This is a playful investigation into pushing the limits of our controls.

http://hhayeon.com/Expressive-Tactile-Controls

Description

Push buttons, sliders, switches and dials—we use such controls everyday, and yet we barely notice them. ‘Expressive Tactile Controls’ is a series of playful research experiments that ask: how could our relationship to tactile controls be augmented by giving these controls more personality? How would buttons that have a stronger, more evocative personality affect and improve our relationship with the applications they are built into? How would people react to buttons that ‘talk back?’

This question was approached by constructing and testing a series of experimental pushbutton prototypes able to express themselves with various tactile and kinesthetic feedback (vibration, tension, and movement), according to the user’s touch or environment.

The first series included controls designed to feel Stubborn, Sensitive/Impatient, Lazy, Timid, Spontaneous, Resolute, and Excited. These ‘personality-enriched’ buttons were then used to provide users with a more intuitive tactile feedback, in accordance with the application they were serving.

Classes

Thesis