\n Exploring the Emotional Architecture of Language
\nLanguage is not only a vessel for meaning\u2014it is a living structure shaped by written form, emotion, and body movement. Type( )Body is a two-part digital experience that reimagines how written language intersects with the expressive potential of the human body, and how emotional depth can be revealed through dynamic visual structures.
\nThe first experience, The Body Writes, transforms the act of writing into embodied play. Users sculpt letterforms using gestures from different body parts, collectively generating a living typographic system in real time. Here, the human form becomes the tool\u2014designing, shaping, and re-defining language through motion.
\nThe second experience, Beneath the Lines, invites users to input a single sentence reflecting their feelings. An AI interprets this text, expanding it into a nuanced narrative while typography shifts responsively to the sentiment and emotional tone. Through hand gestures, users reveal and reshape these emotional layers\u2014making visible the tension between intention, perception, and machine-generated empathy.
\nTogether, these works challenge the boundaries between form and feeling, between structure and spontaneity\u2014proposing a new kind of literacy where emotion is not just read, but felt. <\/p>\n
\n Software and Programming Languages
\nFrontend Development:
\np5.js: for rendering interactive visuals, typography animations, and real-time interaction.
\nml5.js: utilizing TensorFlow.js libraries (poseNet or handpose) for gesture-based interaction and body tracking.
\nBackend\/API Integrations:
\nOpenAI GPT-4 API: Generating emotionally responsive, positive narrative continuations.
\nSentiment Analysis: Implemented via ml5.js sentiment classifier or external sentiment analysis APIs.<\/p>\n
Hardware Requirements (for optimal gesture interaction):
\nStandard Webcam: For capturing real-time body tracking and gesture detection.
\nComputer\/Laptop: Running the website through standard browsers. <\/p>\n
\n Type ( ) Body is grounded in interdisciplinary research across typography, affective computing, and embodied cognition.
\nContemporary research in embodied cognition suggests that our understanding of language is not only cerebral but deeply connected to our bodily experiences. Conceptual metaphors are grounded in physical experience. Body language, therefore, becomes not merely supplemental to speech or writing but fundamental to the way meaning is constructed and understood. Type ( ) Body engages this by translating physical motion into typographic form, an act that makes the invisible scaffolding of thought and emotion visible through kinetic marks.
\nTypography traditionally aims for clarity and consistency, but expressive typography emphasizes the emotional and narrative capacities of type. This thesis draws inspiration from such practices, but moves beyond static or time-based expression into responsive, real-time emotional visualization. By linking sentiment analysis to typographic variation, Beneath the Lines tests how viewers perceive emotional authenticity in generated text and questions the emotional literacy of AI-driven narrative.
\nThe rise of large language models opens new territory for examining how machines interpret and reproduce emotional nuance. Yet, despite technological advancements, emotional misalignment between intention and AI-generated output persists. By combining sentiment classification with typographic modulation and gesture-driven navigation, this project examines not only what machines \u201cunderstand\u201d about human emotion – but how their responses can feel uncanny, comforting, or disconnected. This sets the stage for a more critical and creative dialogue between human intention and machine interpretation.
\nThis project considers the challenges of cross-linguistic expression, especially when language fails to capture the full scope of human emotion. Nonverbal cues such as posture, movement, and rhythm are often more universally understood than spoken words. Type ( ) Body builds upon this idea by creating a shared visual system of communication derived from physical gestures and affective typography\u2014offering an alternate language that lives between cultures and beyond translation. <\/p>\n","protected":false},"excerpt":{"rendered":"
Type ( ) Body is a two-part digital experience that explores how emotion is expressed through the body language and written language\u2014where gestures breathe life into letters, and emotions weave through type, inviting users to step into the unseen architecture of feeling and form.<\/p>\n","protected":false},"author":0,"featured_media":2434,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[30],"tags":[16,13,11,14],"class_list":["post-214","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-30","tag-interaction-design","tag-machine-learning","tag-narrative-storytelling","tag-product-design"],"_links":{"self":[{"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2025\/wp-json\/wp\/v2\/posts\/214","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2025\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2025\/wp-json\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2025\/wp-json\/wp\/v2\/comments?post=214"}],"version-history":[{"count":2,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2025\/wp-json\/wp\/v2\/posts\/214\/revisions"}],"predecessor-version":[{"id":980,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2025\/wp-json\/wp\/v2\/posts\/214\/revisions\/980"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2025\/wp-json\/wp\/v2\/media\/2434"}],"wp:attachment":[{"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2025\/wp-json\/wp\/v2\/media?parent=214"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2025\/wp-json\/wp\/v2\/categories?post=214"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/itp.nyu.edu\/thesis\/archive\/2025\/wp-json\/wp\/v2\/tags?post=214"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}