Type ( ) Body
Cathy Li
Advisor: Tiri Kananuruk
Type ( ) Body is a two-part digital experience that explores how emotion is expressed through the body language and written language—where gestures breathe life into letters, and emotions weave through type, inviting users to step into the unseen architecture of feeling and form.
Project Description
Exploring the Emotional Architecture of Language
Language is not only a vessel for meaning—it is a living structure shaped by written form, emotion, and body movement. Type( )Body is a two-part digital experience that reimagines how written language intersects with the expressive potential of the human body, and how emotional depth can be revealed through dynamic visual structures.
The first experience, The Body Writes, transforms the act of writing into embodied play. Users sculpt letterforms using gestures from different body parts, collectively generating a living typographic system in real time. Here, the human form becomes the tool—designing, shaping, and re-defining language through motion.
The second experience, Beneath the Lines, invites users to input a single sentence reflecting their feelings. An AI interprets this text, expanding it into a nuanced narrative while typography shifts responsively to the sentiment and emotional tone. Through hand gestures, users reveal and reshape these emotional layers—making visible the tension between intention, perception, and machine-generated empathy.
Together, these works challenge the boundaries between form and feeling, between structure and spontaneity—proposing a new kind of literacy where emotion is not just read, but felt.
Technical Details
Software and Programming Languages
Frontend Development:
p5.js: for rendering interactive visuals, typography animations, and real-time interaction.
ml5.js: utilizing TensorFlow.js libraries (poseNet or handpose) for gesture-based interaction and body tracking.
Backend/API Integrations:
OpenAI GPT-4 API: Generating emotionally responsive, positive narrative continuations.
Sentiment Analysis: Implemented via ml5.js sentiment classifier or external sentiment analysis APIs.
Hardware Requirements (for optimal gesture interaction):
Standard Webcam: For capturing real-time body tracking and gesture detection.
Computer/Laptop: Running the website through standard browsers.
Research/Context
Type ( ) Body is grounded in interdisciplinary research across typography, affective computing, and embodied cognition.
Contemporary research in embodied cognition suggests that our understanding of language is not only cerebral but deeply connected to our bodily experiences. Conceptual metaphors are grounded in physical experience. Body language, therefore, becomes not merely supplemental to speech or writing but fundamental to the way meaning is constructed and understood. Type ( ) Body engages this by translating physical motion into typographic form, an act that makes the invisible scaffolding of thought and emotion visible through kinetic marks.
Typography traditionally aims for clarity and consistency, but expressive typography emphasizes the emotional and narrative capacities of type. This thesis draws inspiration from such practices, but moves beyond static or time-based expression into responsive, real-time emotional visualization. By linking sentiment analysis to typographic variation, Beneath the Lines tests how viewers perceive emotional authenticity in generated text and questions the emotional literacy of AI-driven narrative.
The rise of large language models opens new territory for examining how machines interpret and reproduce emotional nuance. Yet, despite technological advancements, emotional misalignment between intention and AI-generated output persists. By combining sentiment classification with typographic modulation and gesture-driven navigation, this project examines not only what machines “understand” about human emotion - but how their responses can feel uncanny, comforting, or disconnected. This sets the stage for a more critical and creative dialogue between human intention and machine interpretation.
This project considers the challenges of cross-linguistic expression, especially when language fails to capture the full scope of human emotion. Nonverbal cues such as posture, movement, and rhythm are often more universally understood than spoken words. Type ( ) Body builds upon this idea by creating a shared visual system of communication derived from physical gestures and affective typography—offering an alternate language that lives between cultures and beyond translation.
