Local AI sounds like a software topic until I actually try to run it. Then it becomes a hardware constraint problem almost immediately. How much VRAM do I have? Can the model fit? What happens if part of it spills into system RAM? Is the second GPU actually detected? Is the PCIe slot fast enough to matter? Do I have enough regular RAM to avoid making the whole machine miserable?
The appeal is obvious. I like the idea of local models for plant notes, manuals, tag references, scripts, and home lab automation. Not because I think local AI is magic, but because it is useful to have something I can experiment with without turning every question into a cloud integration.
The constraint is that model names hide the real cost. A 27B or 35B model sounds like a number, but the practical question is how it runs on the hardware I actually own. A 16GB GPU is a lot until it is not. Two 16GB GPUs sound like 32GB, but that does not automatically mean everything works cleanly. Drivers, model splitting, inference engine support, PCIe lanes, and thermals all get a vote.
What I checked was the same kind of thing I check on a NAS build: where are the bottlenecks, what is actually connected, and what am I assuming because the spec sheet made it sound simple? The GPU is not just a GPU. It is a slot, lanes, power, driver state, and software support.
The surprise is that the useful local AI setup may not be the biggest model I can barely run. It may be the smaller model that responds fast enough, fits cleanly, and can be wired into a tool. A slow genius sitting in my basement is less useful than a decent model that can search notes, summarize logs, or help write scripts without drama.
Notes for next time: measure tokens per second, not vibes. Write down the model, quantization, VRAM use, RAM use, and what else was running. Do not design around a second GPU until the system actually sees it. Practical AI starts with practical hardware.