Local LLM Lab 🤖

Frontier AI is rented from a handful of unscrupulous corporations running unsustainable datacenters. Meanwhile, a capable large language model (LLM) can run from a file that lives in your documents folder: offline, irrevocable, unmetered. Local LLMs have advanced considerably in the past year.

This hands-on lab session is about running open-weight language models on laptops. We'll get models running via 👋 Jan, probe their behaviors, and discuss how they work and why that matters. What do you give up, and gain, by owning your tools?

HOMEWORK

The models we will use can be up to 20GB. If we all try to download at the same time, it will take forever, so there is homework.

Homework details: https://gist.github.com/atait/59ed2e84263df42e73344ffd17743263

Homework tl;dr,

1. Obtain a suitable* laptop (ER or BYO)
2. Install Jan
3. Download a LLM file (this takes ~20 minutes)

What if I don't do the homework? - Come anyways; you can buddy up while waiting for the download.

*Apple Silicon architecture (M1, M2, etc) is recommended. Windows/Linux/Intel/AMD should work but will be very slow. Apple Silicon laptops are available for check out from the ITP Equipment Room.