AINews wire, 19 Nov 2026
Three used V100 GPUs run a 35B local AI model at about 100 tokens per second
Reported by Tom's Hardware

A homelab builder has tested three used Nvidia Tesla V100 graphics cards running Qwen 3.6 35B, a large language model, on a home server. According to the builder's published benchmark, the setup reaches about 100 tokens per second with a 4K context window. The complete build is listed at $1,042 and uses only cards the builder calls e-waste, meaning decommissioned enterprise hardware. Tom's Hardware reported the results from the builder's blog post.
The blog post compares nine builds using one to three K80, P100 or V100 cards on the same Supermicro dual-Xeon board. The tests used llama.cpp and covered more than 130 runs across several models, including Qwen 2.5, Gemma 4 and Qwen 3.6. They used Q4, Q8 and F16 precision and context sizes from 4K to 128K. The builder also logged GPU energy for each run. This round follows a July test of 15 e-waste cards. The board, CPUs and power supply stayed the same for every build, so only the GPUs changed.
The main result is for Qwen 3.6 35B at Q8. Its speed falls from about 100 tokens per second at 4K to about 73 at 128K. The builder uses 70 tokens per second as the bar for real-time interaction, so the setup stays above it across the tested range. The model is a mixture-of-experts design that activates about 3 billion of its 35 billion parameters per token. The dense Qwen 3.6 27B model is much slower, falling from about 24 tokens per second at 4K to about 14 at 128K. The mixture-of-experts model is faster by a factor that ranges from roughly 4x to 5x, depending on context size.
Other tests show a split between models. Two V100s clear the 70 tokens per second target with the mixture-of-experts Gemma 4 26B at Q8. Dense models of about 30 billion parameters reach only about 21 tokens per second at 4K. At Q4, two V100s run Qwen 3.6 35B faster than at Q8. The builder says extra cards add memory for larger models rather than speed. Two P100 setups reach 40 to 55 tokens per second on the same model at Q8, and only a smaller model reaches the target.
The builder compares the V100 rig with two other systems. Nvidia's DGX Spark, with a list price of $4,700, reached 45.8 tokens per second in a separate benchmark using Ollama and short prompts. The M5 Max Mac Studio, priced from $2,499 and $5,399 as tested, reached 43.9 tokens per second with a prompt of about 195K tokens. The Mac leads at 4K and 32K, while the V100 build leads at 64K and 128K, by 19% at the longest context. The runtimes differ, so the builder says these figures should be read loosely.
Modern data-centre GPUs are far faster. A comparison sponsored by the inference provider Parasail reports about 315 tokens per second on two H100s and about 455 on one B200 at 128K. The hosted cards ran the full 16-bit model, while the V100s ran an 8-bit version. For energy, a 128K run on the V100 setup used about 0.02 kWh of GPU energy. The P100s used about twice that, and two or three K80s used 0.35 to 0.45 kWh. These figures cover the GPUs only, not the whole system at the wall.
Several things are still unknown. The builder plans a benchmark for always-on "software factory" workloads, where speed matters less. A cooler for the cards is in development, but no release date has been set, and beta testers will be recruited later. The builder also plans to compare inference backends and test speculative decoding to improve performance.
Enjoying this? Get the AI brief
Every Thursday: AI that matters for your home.
✓ Free · seven editions · one click to leave
Original reporting by the sources above. Summary by smart-home.space.