Projects / tensorless
tensorless
A $100 GPU card. We made it fly.
A custom engine and quantization that run a modern 27B model on a 2016 Tesla P100, in one command.
The receipts
Measured, with a public source for every number.
131 → 28
Judged attempts, then confirmed wins, in the Thermite campaign that took the engine from 7.74 to 30.06 tok/s
tensorless.dev ↗Why I built it
I see the price tags on Sparks and other nice cards, and then I see P100s still sitting around $100. 16 GB of HBM2, no tensor cores, passive cooling, official support dropping left and right. And so much raw compute power there for cheap.
So I pointed Thermite at the engine and let hill climbs run overnight on fully local models. It went from under 8 tokens a second to 30.
I want P100s to be useful to others for local AI, without spending thousands of dollars on GPUs. This is the first piece.
What it is
tensorless run pulls a pinned engine image and a model revision, checks both, and serves them.
PN is a custom 3- and 4-bit weight format built for this card. The engine is a fork of llama.cpp, tuned for it.
It serves an OpenAI-compatible API on 127.0.0.1:8080, so your existing clients just work. No account, no telemetry.
tensorless doctor checks the card, driver, container runtime and disk, and prints the fix for each problem. tensorless bench measures your setup so you can compare it to the published receipts.
Try it
$ curl -fsSL https://tensorless.dev/install.sh | sh
$ tensorless doctor
$ tensorless run iamMrDuncan/Qwen3.8-27B-PN3-GGUFHonest caveats
- Built for one card: the NVIDIA Tesla P100 16 GB.
- Linux on x86-64 with AVX2, driver R525 to R580, and podman or docker.
- Long-context results (32K and 65K) are not measured yet.
- It's v0.1.0. Young, and it will change.