AINews wire, 20 Nov 2026

Lawyer's 12-GPU home AI server now runs all DeepSeek V4.1 Flash experts

Reported by Tom's Hardware

Image: Tom's Hardware

A lawyer has expanded his private AI server from eight to twelve used Nvidia V100 graphics cards. The larger setup keeps every expert of the DeepSeek V4.1 Flash model on the cards in 4-bit NVFP4 format, according to a report by Tom's Hardware. The owner, Nick Mermiges, paid $9,450 for the system. He says it can outrun published DGX Spark setups when handling a single request, while costing about half as much as three of those units.

The earlier build had eight 32GB cards with 254 GiB of usable memory. The chosen model's NVFP4 experts need 278.9 GiB. His previous workaround kept every expert at 2-bit precision and converted the most-used ones, 5,949 of them or about 38.7 percent, to NVFP4 until the memory was full. The new third board, with four cards, lets all 15,360 experts stay at full precision. According to his GitHub repository, none are compressed further. He says the expert values match those in DeepSeek's own release, while other parts of the model use compact forms.

There is a technical limit. V100 cards have no FP4 or FP8 tensor cores, so every NVFP4 value must be unpacked in software into half precision before the card can use it. The compact format still helps, he argues, because the cards have less data to read from memory, even though they do not compute faster. The model's layers are split across the three boards: layers 0 to 13, 14 to 27 and 28 to 39. Each board's share is spread over its own four cards, which are linked by Nvidia's NVLink. Hand-offs between the first two boards stay inside one PCIe switch. The third board sits behind a separate switch, so the second hand-off crosses the host's PCIe bridges, which adds delay.

He also changed the power and clock settings. Each card's power cap rose from 150 W to 175 W, and its clock cap fell from 1,522 MHz to 1,452 MHz. His measurements show that the first code token at a 256K context arrives in 58 seconds. After his previous upgrade it took 108.4 seconds. Prompt processing past 32K context reaches about 4,500 tokens per second. That is more than twice the roughly 2,200 tokens per second of his first build, although the higher power cap probably explains part of the gain. Code decoding is 23 percent faster at 1K context but shows no real gain at 32K and above. The main benefit is therefore a shorter wait before the first token appears.

He tried running each card at 200 W for a day, but the host failed once when four long chats ran at the same time. The failure suggests the power draw was too high. He also tested four optimizations that missed their targets and were switched off. They were a feature that keeps chat history on disk, a leaner hand-off between sections, tighter expert packing and exact arithmetic for the four-card sums.

Running several requests at once often raises total speed, but this setup does not work that way. Three requests at a 128K context reached roughly 360 tokens per second combined for code and 213 for legal text. The owner notes that the engine is not built for batched decoding, so more requests than sections bring little extra speed. He plans to study batched decoding next. He also plans to enable the storage feature so that a case's documents survive a restart. He admits the benchmarks do not show answer quality in real use. Even so, the machine has been his default DeepSeek system since Monday, and he says this is the first local model that works like Claude Opus 4.6 for his tasks.

Enjoying this? Get the AI brief

Every Thursday: AI that matters for your home.

✓ Free · seven editions · one click to leave

Topics
Story continues on Tom's HardwareAdd us as a preferred source on GoogleSee our stories first in Google’s Top Stories.
Get alerts on big newsOnly the big news, at most once a day. No email, no account.

Original reporting by the sources above. Summary by smart-home.space.