A lab notebook · real runs on secondhand hardware
Real experiments with local AI on low-budget hardware
Running 27B to 80B models on an $89 Tesla P100 and an 8 GB gaming card, keeping AI agents safe, making video with local models, and whatever the lab takes on next. Every post comes from a real run, with the commands, the numbers and the dead ends.
New here? Start with the speed series, or browse by topic.
Topics
All topics →Pick what you came for. Each topic starts with the best way in.
Local AI and GPUs
Running big models on cheap, secondhand and mismatched GPUs: how to split them, which settings move tokens per second, and what still runs on old cards.
See the topic →AI agents and security
Giving local AI agents real tools without handing over the keys: prompt injection, safe tool setups, and what to lock down.
See the topic →
AI media: images, video, voice
Making images, video, speech and sound with local models on consumer cards.
See the topic →Latest
All posts →- Why I wrote my own inference engine for two P100s
Patching llama.cpp got a 27B model to 32 tok/s on two P100s. A from-scratch CUDA engine built for just this card and model reached 49 tok/s plain and about 118 tok/s with the model's own draft head.
- P100 power caps, clocks and cooling: what the numbers say
The P100 is a passive 250 W card. What a 125, 150 and 250 W cap does to its clocks and temperatures, why nvidia-smi -lgc fails on Pascal, and the fan mistake that let both cards hit 79 °C.
- Beyond chat: video generation and fine-tuning on two P100s
Two P100s can make a short AI video clip and can fine-tune a 27B model. Both work. Neither beats a single RTX 3070 for video, and fine-tuning is measured in days.
- Securing a local AI agent: best practices
An agent that can read your mail, run commands and browse the web needs more than a good prompt. Ten practices for the tool layer, where the real risk is, plus a checklist.