Computation and Data (Old Structure)

We experience screens daily in many forms: in our hands, on our desktops, on walls and public installations as we travel. This course will explore the creative possibilities of real-time interactive and reactive art on screens in various forms. Using the recently developed p5VideoKit we will create standalone installations. p5VideoKit is a new library of live video effects – building on p5js – presented as a dashboard for mixing video in the browser. This library allows the user to apply visual effects to live video from connected cameras and sensors or streaming from devices on the internet. p5VideoKit is open source and can be extended with the user’s p5js code for a plethora of visual effects and interactivity. One possible application of p5Videokit would be a public facing installation allowing anonymous people on the street to use their hand held devices to interact with large street facing screens, thereby collaborating on real time creation of “digital graffiti”.

Building on Creative Computing, students will learn how to adapt simple sketches into components of p5VideoKit so that algorithms can be quickly composited and orchestrated into more complex works. Students will also learn how to edit and share code beyond the p5js editor, use nodejs/javascript to automate deployment of installations, and remotely configure dedicated computers with long running installations. Several dedicated computers and screens will be available to preview installations on the floor and street facing areas of the 370 Jay Street campus.

Prerequisites: Creative Computing or equivalent coding experience.

About John Henry Thompson: http://johnhenrythompson.com

This class asks students to think about thinking. Based on first person introspection, meditation and readings in psychology, students will examine the experience of their minds. Then we look at how computation, in particular recent developments in AI, can better work as a medium to capture and share that experience. Class time is evenly divided between conceptual discussions around the psychology of media, looking at student work, and then learning coding skills for the following week. The early course materials direct students towards developing and implementing multi-user web apps to improve our society’s social media ecosystem but final projects often take different directions. 
 
On the technical side, the class gently picks up from any introductory javascript coding class.  Compared to Creative Computing it moves away from the p5.js in favor of vanilla javascript in an environment like Visual Studio Code assisted by AI.  The class encourages students to find a healthy balance of using “vibe coding” while maintaining the ability to specify overall architecture and debug individual lines of code. In particular the technologies covered are Replicate.com’s API’s for Machine Learning Models for generation and relation of text and images, Firebase tools for server based databases, realtime sharing, storage and authentication. Libraries like UMAP for dimension reductions, Three.js for realtime rendering, P5LiveMedia for Audio and Video sharing, and Colab for running python notebooks.

Each week students will post a quick sketch experimenting with the technology as well a short written response to a prompt together with a short reading or video.