I think my interests lie less in ‘whether machines are conscious’ and more in ‘what happens when we cannot tell the difference’.

~~~~~

Switchboard operator as an early “Ghost in the Machine” // Aesthetic parallels to the Neural Network

 

Types of Phone Calls/Voicemails:

• Catch up
• Business
• Appointment Reminders
• Spam voice
• Job Interviews

 

Voices to include:

• Family
• Friends
• Coworkers
• Receptionists

 

Potential Futures:

• Leave real voicemails on people’s phones (Twillio API)?
• Can I buy phone numbers off the dark web to use (Hypothetically)
• Play a message and call someone at the same time

 

Progress Photos:

Stream of consciousness again, apologies if things aren't coherent:

---

“In the future, it’ll be much harder to tell where the human stops and the machine begins”

 

Both Show-A-Thing and Mentor meetings were enormously helpful for me. By forcing myself to communicate my project concept to multiple different people from multiple different backgrounds, I was able to hone in on the parts of my project that felt solid and consistent. Currently, I’ve landed on the following concept:

 

I am trying to understand the philosophical theory behind “The Sublime” and focus on connecting those ideas to the state of modern technology. Specifically, exploring ideas of infinity through Machine Learning and Large Language Modeling.

 

The Sublime is traditionally defined as “the exalted”, a feeling of beauty that is grand and dangerous¹. There are many philosophers who attempted to define the sublime (Kant, Schiller, …) but there are two distinctions that define early Romantic perspectives on the feeling:

• The Mathematical Sublime – An understanding of the universe’s scale and our insignificance
• The Dynamic Sublime – Our understanding of the universe’s power, and our awe and fear

As technology has progressed, we have come to shift our source of the sublime from nature to technology. It is still possible to feel close to the sublime in specific parts of the world (Grand Canyon, Niagra Falls), but as our control over the environment increases we find our control over technological systems to weaken and weaken. This feeling will increase exponentially as time continues, as our built environment multiplies in complexity.

 

Here are some slides of the visual and conceptual thinking I shared during the sessions this week:

 

I know I want to try and tap into the sublime using machine learning and generative media, so I’m prototyping the following concept:

 

Voicemails from the Sublime: Machine Learning as a compass to navigate collective infinities

I want to explore Large Language Models (LLMs) as material for simulation, to bring attention to the size and scale of the technology’s possibilities as well as be a reflection of the scale of the human collective.

 

With this project, I’ll use LLMs (GPT-3/4), Text-To-Speech, and Physical Computing to explore simulated realities. These simulated realities aim to:

1. Explore early Romantic concepts of the mathematical and dynamic sublime
2. Talk to ideas on consciousness, intelligence, and embodiment through simulation
3. Question how AI and ML will redefine what it means to be human
4. Draw attention to the scale and complexity of modern technological infrastructure

 

Prototyping Summary:

Testing prompt generation of the following script:

 

Prompt:

const voicemailPrompt = 
`Generate a voicemail a grandmother would leave on her grandson's phone.
The grandson has a generic male name.
The grandmother hasn't heard from him in a long time and wants to see him for easter."
`

 

Example Generations:

 

 

 

Testing Text Generation –> Audio Generation

 

 

This is with the first generic voice, and you can easily change/train your own models to customize it. I’ve also written code to run this generation pipeline on my own, so I can easily type in a name/topic, and have the model generate a voicemail and accompanying audio recording:

 

 

Doesn’t feel too “S U B L I M E” yet but at least the pipeline is working to go from Text to Voice output.

Also I’ve been using GPT-4 to write everything and it’s been amazing for productivity. Will continue hacking this weekend and make another post. I want to do some prompt engineering to get more interesting output from the model, and see if I can generate a bunch of text/audio pairs.

 

References:

1. https://www.nextnature.net/story/2011/the-technological-sublime
2. Figma Slides: https://www.figma.com/file/ruwB0T8Nx6TagewOJGwl6A/Sublime-Technology—Talk-Overview?node-id=6%3A94&t=nvByZ5tLXhW39zn1-1
3. TTS API: elevenlabs.io
4. LLM: GPT-3.5 

 

Half-baked ramblings after a day of binge-scrolling Twitter below:

---

I feel quite overwhelmed by the recent progress in AI and LLMs with GPT-4.

I say this not only about the general breakthrough in science, as the technology is incredibly impressive, but the downstream impacts of what this means for humanity as a whole. All weekend my Twitter feed has been flooded with content on the topic, and the only other time I’ve recognized “the collective” speak to something at this scale was the brief period between the first few cases of COVID and the first week in March when things began to shut down. (Caveat: I might also be a product of my information bubble, and not realize how deep I truly am)

I’m really not sure what this will lead to, and I hope I’m not being dramatic, but I think we’ve just accelerated a fundamental shift in society to the scale of what we’ve seen with electricity, the internet, and mobile phones.

Or maybe it’s all just hype 🙂

 

I’ve also noticed quite a few tweets discussing how this moment relates to ideas of consciousness, spirituality, and religion, all topics quite relevant to my thesis. Through reviewing my proposal feedback, more reading, and personal research, I think I want to experiment with GPT-3/4 LLMs and Voice Processing+Generation to explore the gray area between intelligent machines and anthropomorphizing of non-human entities, while speculating what this may look like in the future.

What this may look like (crude 11pm sketch below):

1. Use a physical landline phone as an interface between human and “machine”
2. Use GPT-3+TTS APIs to simulate a conversation
3. Fine-tune/steer these conversations to have distinct personalities
4. ????
5. Have a successful Thesis presentation

Step 4 is looking to be the most difficult rn. Will sit with the concept a bit more and run it by Sarah and residents for further thoughts.

 

This concept was also inspired by a Tweet I saw earlier today. Someone managed to run a model similar to GPT-3 locally on device without a need for a network connection, meaning the technology in the movie “Her” just got a whole lot closer. This is a fascinating breakthrough and I want to experiment with it.

Introducing LLaMA voice chat! 🦙

You can run this locally on an M1 Pro pic.twitter.com/jetTQ5FxcS

— Georgi Gerganov (@ggerganov) March 26, 2023

 

Some earnest and satirical thoughts/links/ideas/garbage for reference:

And a few more links to keep here for my own reference:

• https://en.wikipedia.org/wiki/Anthropomorphism
• https://danielmiessler.com/blog/ai-is-eating-the-software-world/
• https://github.com/ggerganov/llama.cpp
• https://github.com/ggerganov/whisper.cpp/blob/c353100bad5b8420332d2d451323214c84c66950/examples/talk.llama/talk-llama.cpp#L177

Thank you to the anonymous reviewers of my proposal, your feedback was thoughtful, relevant, and constructive for my thesis development.

All my reviewers came back with similar feedback, that my proposed concept was quite broad and not specific enough to make progress on or adequately critique. Which in retrospect is true 😅. With that feedback, I really spent time with my thoughts and references to specify exactly what I wanted to focus on in terms of AI, Spirit, Consciousness, and Spirituality. These are very broad ideas with plenty of rich overlap, but for the sake of my thesis, I need to narrow !!

 

After more reading and writing, I realized that in order to have discussions on non-human intelligence, consciousness, and spirituality, it would be helpful to answer the question: What is spirit?

Here’s what I think my updated (and more actionable) thesis concept should be:

• How can ideas and theories of spirit and spirituality help us develop empathy for non-human systems, and help us further develop a more positive, collective ecology with nature?
• How can I develop sculpture that challenges popular ideas of Anthropocentrism, and reorient the human’s relationship with nature?

Or put more plainly:

Can I develop interactive experiences that challenge ideas of human superiority and exceptionalism? Can I develop experiences that act as a mirror for reflection of human interaction and behavior?

 

I think this is what I originally tried to ask with my first proposal, but this feels more specific and actionable. I know I need to start making, but this reframe feels productive and tighter in scope. Will report back soon 🫡

It’s been a very productive week! Managed to:

• Finish Thesis proposal (link)
• Create concept map of AI, AI Art, Spirituality, and Consciousness (link)
• Updated are.na channel for project: (link)
• Books came in – Making my way through “The Age of Spiritual Machines” – Ray Kurzweil

 

I think my next step will be to sketch a quick zine to highlight the history of the field, and articulate the important questions I want to build on as part of my thesis. The questions on consciousness and intelligence still are still important parts of the project, but in my research there are a few more ideas popping up. Specifically, I’ve been sitting on:

• Embodiment, and what this means for consciousness/intelligence, and the physical presence of ‘artificial’ systems.
• Creating meaning from noise, and religion as a tool for meaning making
  • Parallels to current paradigms in Machine Learning – Diffusion Modeling
• Who is in control in generative systems, the artist or the machine?
• Identifying the contrast between Eastern and Western religions discussing the idea of God as an entity above vs an entity within us.

 

I also need to start making, so I’m going to brainstorm a quick Raspberry Pi project to get started with ML/AI and get a prototype up by the end of the week. Probably a simple camera image/video recognition project to get my development environment set up correctly.

 

Plan for this week

• Complete a draft of the first zine – AI, AI Art, and
• Get through the first section of The Age of Spiritual Machines
• Use StableDiffusion to generate some imagery
• Get Physical – Can you create a quick AI-powered sculpture? Can you get a recognition algorithm running on a Raspberry Pi?
• and other things …

4 hours into my flight to SF for the week and thought it’d be good to spend the flight working on my proposal draft. WIP but sharing here to work in public:

---

 

Proposal Draft

Proposal Questions

Below is an outline of what you should address in your thesis proposal. In its most straightforward form, you can simply answer these questions. However, if you want to include supplemental material (images, other media) that’s great, as long as the below questions are covered. Anything additional that you believe will clarify your intention and vision is helpful!

Working Title (Optional)

You don’t really need a title at this stage, but it can be fun and even helpful!

On the Sublime and Divine: Explorations into AI & Technological Spirituality

Three Keywords that Describe your Proposed Work

These can be topics, themes, technologies, moods…

Sublime, Artificial Intelligence, Physical Computing

What is your central question?

(approx 2-3 sentences)

Your thesis should have more potential for expansion than a one-off assignment or project. In order to ensure that, it can be helpful to frame the idea you are exploring as a research question. What is a central question, problem statement, or area of inquiry that you will try to investigate and address through your research and creative practice? Here are some guiding principles we have found helpful in working through your question:

1. Is it something that you are genuinely interested and curious about?
2. Is it broad enough that it doesn’t allow for simple factual answers (yes/no responses)?
3. Is it narrow enough that it can be answered/addressed given the available time and resources?

If you are looking for inspiration, the 2015 ITP thesis archive was organized by central questions.

 

If spirituality can help us process systems greater than ourselves, how might we use its teachings and aesthetics to better understand rapidly advancing technology? Specifically, how might we use music as an intermediate space between Technology and The Divine, in order to help us better understand our relationship with both?

Guiding Questions:

1. What are the basic principles of spiritual and religious aesthetics?
2. What are the fundamental principles that invoke a feeling of wonder and awe?
3. How will our relationship to technology and machines change as they scale in “intelligence”?
4. How might religion and spiritual teachings on higher consciousness help us process and understand this shift?

What is one way you envision realizing this as a project?

(250-350 words)

At this stage – you probably know SOMETHING about your thesis – even if you think you don’t! There are (at least) two ways of thinking through this question:

Describe a process including “known unknowns”

For example: I plan to do x type of research, for x amount of time, with a focus on x. This could culminate in x, which I could achieve in x, y, or z method. I don’t know which method, but I plan to let my research guide me based on x.

In this version, it’s helpful to express whatever you know AND articulate your “known unknowns”. Some of you may be starting with a clear idea of a form and less of the context. Some of you may have an idea for a research trajectory, but aren’t sure what kind of form it might take. Some may have totally different “knowns” and “unknowns”. Talk it out, how do you plan to figure it out?

Take a leap of faith: describe something concrete (even if you’re unsure).

In this version, you should try to paint as clear a picture as you can, in your reader’s mind, of what the end result of your work could possibly be (even if you’re totally unsure and making it up). Then, try your best to explain how you think you would execute that work, including a basic timeline.

In either case, answering these questions will help:

• What are the key steps necessary to make this a reality?
• Are there key milestones you will need to meet?
• What techniques, tools, materials are you planning to use and why?
• Is there a community, space, or audience you want to access?
• What further research and/or skill-building will you need to do?
• What form (or forms) might your work take?
• How does what you plan to do address your Thesis Question?
• Which element is the focus of your work? (For instance: a specific research methodology, an innovative technique, framing the work, gaining expertise in a specific topic, producing a final product, honing a technique there are many possibilities here!)

 

By the end of the term, this project will manifest itself through the following artifacts:

1. Knowledge product(s) on music, machine learning, and AI (blog post, poster, graphic series, public notes, zine)
2. Knowledge product(s) on wonder, the sublime, and religious architecture (annotated are.na channel, documentation blog, guide, zine)
3. Prototype for a Musical Interface powered by Machine Learning, expressed through the aesthetics researched in #2

It will be important to spend the next 10 weeks focused equally on Research and Making. Each week should end with a new step in research (new book, article) and art that attempts to explore the research (p5.js sketch, paper sketch, zine, sculpture)

Jumping ahead, I imagine this will look like a physical sculpture/interface that allows a user to explore musical latent space through movement, and have this movement dictate musical output and accompanying visuals.

 

Why is this important to you?

(approx 2-5 sentences)

• Why do you want to do this? Be as honest as possible!
• Why you, why here, why now?
• What impact do you hope your thesis has?

 

This concept is interesting to me for a few reasons:

1. I want to learn how to develop engaging artwork, and in the future pull from this research on wonder and divinity to build powerful interactive installations
2. I have a strong curiosity about the fundamentals of music and want to learn more about how music works both physically and emotionally
3. I work in AI, and want to learn more about the engineering behind machine learning systems
4. I want to build physical objects, and a musical interface is a great way to further develop my skills in physical computing

Why me, why here, why now?

1. AI systems and generative modeling are extremely relevant right now, and will continue to be important for the foreseeable future
2. The world is becoming increasingly harder and harder to understand as the systems we engage with become more and more opaque
3. As a technologist fluent in art, technology, music, and ai, I have a unique perspective on the intersection between ideas and a want to communicate these ideas to a broader audience

What impact do you hope your thesis has?

1. I hope this work can inspire awe and wonder in participants, and create moments of reflection on the systems that we exist in that are larger than ourselves.
2. I also hope this work can raise critical questions on AGI, and explore how a spiritual framing of AI may be beneficial/detrimental to humanity

 

What are your influences and inspirations? What is the “ecosystem” of your work?

(10 links, each with approx 1-2 sentence description – this is your initial bibliography!)

• Share 10 links to references that directly relate to your project. These can be books, films, other artworks or related projects. For each, write a sentence or two as to how they relate to your work.
• Please include: at least 1 article from an academic journal, 1 prior art reference (ie, similar work), and one theoretical text (could be a book or article – something which points to a social or critical context). Consider reaching out to Margaret Smith (ITP/IMA Librarian) for guidance!
• You may also include links to any projects you have already done yourself that you intend to leverage and/or build upon.

 

Music Theory: A collection of resources discussing the physical and conceptual fundamentals of music theory

1. Ableton: Learning Synths – An introduction to electronic music and the sound waves that make modern music
2. Sonic Writing: Technologies of Material, Symbolic, and Signal Inscriptions – (From Luisa Pereira’s NIME) Explores how contemporary music technologies trace their ancestry to previous forms of instruments and media
3. Theremin in the Press: Instrument remediation and code-instrument transduction – Shows how the theremin as a new musical medium enacted a double logic throughout its century-old techno-cultural life

Musical Interface Design: Materials on musical interface design

1. Push, Turn, Move – Teaches how electronic musical instruments are designed for the special needs of the most expressive, lyrical, and demanding things in the music world: human beings.
2. A NIME Reader: Fifteen Years of New Interfaces for Musical Expression – What is a musical instrument? What are the musical instruments of the future? This anthology presents thirty papers selected from the fifteen year long history of the International Conference on New Interfaces for Musical Expression (NIME).
3. Music as a mirror of mind

AI & Machine Learning: Introduction to the math and computer science behind Artificial Intelligence and Machine Learning

1. NN → Zero to Hero – A course by Andrej Karpathy on building neural networks, from scratch, in code.
2. https://github.com/aaronsherwood/artintel – Machine Learning for Artists (NYUAD Spring 2018)

Philosophy of Wonder, Spirituality, and the Divine: Non-denominational background texts on spirituality, religion, and sublime aesthetics

1. Constructing the Ineffable – Contemporary Sacred Architecture
2. Sacred Spaces: Contemporary Religious Architecture
3. Strange Wonder: The Closure of Metaphysics and the Opening of Awe (Insurrections: Critical Studies in Religion, Politics, and Culture)
4. Reclaiming Wonder: After the Sublime (Incitements)
5. Wonder: A Grammar
6. Wonder : from emotion to spirituality
7. The Age of Spiritual Machines

Further Inspiration

1. Google Magenta
2. Brian Oakes
3. James Turrell
4. NIME
5. Thor Magnusson

 

Optional Questions

(no more than 1 sentence each)

• What are your areas of strength that you think you can leverage in this process?
• What areas do you hope to work on through this process?
• What specific technologies, methodologies, or processes do you hope to explore in more depth?

 

My areas of strength:

1. Technical expertise
2. Knowledge of the field
3. Curiosity for aesthetic exploration and art direction

What areas do you hope to work on:

1. Research and research presentation
2. Experiential design
3. Visual Design
4. Fabrication and Industrial Design

Technologies to explore:

1. Machine Learning and AI (Tensorflow/Pytorch)
2. Visual Design, Layout, Typography (Figma/Illustrator)

Had a great call with Sarah the other day. Feeling like I should stick with the ML-Interface for music creation, and trying to blend a bit of what I’m doing with what’s going on at work. Some insight from today’s group and 1:1 calls:

• How do you make the interaction still feel rewarding to beginners and experts? Many ML-based music projects have strong guardrails to prevent “bad output”. Can you allow room for the good and bad outputs of your system and still make the experience feel rewarding?
• How can you combine both concepts (Sublime/Awesome Technology + ML Musical Expression) into a single trail of work? Is there room to blend the two ideas?
• Go to the MoMA and check out Refik Anadol’s piece, then read the critique here

Overall I’m feeling good where I’m at, with the next steps being to start reading materials, reviewing my concept with Luisa and asking for course materials, speaking with Yotam, and doing an initial dive into the technologies I want to use.