A custom engine and quantization that run a modern 27B model on a 2016 Tesla P100, in one command.
Hey, I'm Shannon.
I build local AI that's open, measured, and cheap to run.
By day, Principal Architect of Agentic AI at LiveRamp. The rest of the time I build in my home lab, ship it open source, and run Hackers in the Loop, a community with a seat at the table for you.
I work on things that solve my problems and the community's. Three goals drive most of it:
goal 00 · hackers in the loop
Build a place where others can grow, learn and build together.
goal 01 · small models
Prove or disprove that smaller models can do better than larger ones.
goal 02 · cheap compute
P100s still sit around $100. There's so much raw compute there for cheap.
What I ship
All projects →
Open-source autoresearch. Agents propose. A judge they can't see or change measures every attempt and keeps only what it confirms.
A lean, local-first core for bots, chats and files. Works with local models and hosted ones. Add only the skills and MCP servers you want.
A source-backed catalog of hosted LLM, embedding, speech, image and research APIs with genuinely free access. Prompt privacy is audited separately from price.
Serves one quantized checkpoint, measures it, and compares it to its full-precision original. Thinking on and off.
Still live, still in use
Not new. Still running, still used every day.
Free tools for learning Mandarin: short stories, translation, flashcards and more.
I built them for my own study. Lots of people use them now.
Stupid Simple Presence Server. The old website hit counter, back, with one script tag.
I built it for my own sites, then opened it up for anyone to use for free.
A score is never just the model's.
Model, version and engine, measured with one harness held constant.
Earlier work: zd (Zen Suite) · Thresher · Loom ecosystem. All 11 projects →
There's a seat at the table.
Hackers in the Loop is a free community for people building with tech, learning it, or still finding their footing. People invested in me. This is how I pay it forward.
-
Free Learning Hub
Books and short video lessons on how modern AI models work.
-
New open models, every night
One post per model in #new-models at 8 PM.
-
The HACK Lab
Compute for open research and projects, after review. Share what you learn.
Latest from the build
@iamMrDuncan on X →- Qwen 3.8 27b on 1x P100. Nearly 30 tok/s now... So great for a card with no tensor cores!
- >start a hill climb on fully local models and go to bed >wake up to +216.3%
- Hackers in the Loop started as a place to talk. It's grown into a lot more.
- Ways an agent can make the number go up without making anything better.