AINews wire, 1 h ago

A week with a free local AI model instead of Claude: where it fell short

Reported by How-To Geek

Image: How-To Geek

Tony Phillips, a senior author at How-To Geek, spent one week trying to replace the free tier of Claude Sonnet 5.5 with a local AI model that ran entirely on his laptop. Most answers were good enough for everyday work, but he went back to Claude by the end of the week.

He ran Qwen3-4B-Instruct-2507, a compressed model file of about 2.5GB, through LM Studio. The machine was an Intel Core i9-13900H laptop with 16GB of RAM. He skipped formal benchmarks and used the kinds of questions a person might ask an assistant at work. He compared the results with Claude using its default Medium effort setting.

The early tests went well. An Excel question about the LET function received an accurate explanation, correct syntax and a sensible example. The model also summarised a long Windows announcement, picking out the main changes and the key updates. The author said the summary was solid, although producing it put a heavy load on the laptop.

The weak point was judgment rather than factual answers. When he asked for headlines, article ideas and story angles, the model proposed a finding that USB ports slowly damage phone batteries. It then described a testing method and a result, as if the research had already been done. No such testing had taken place. In another exchange, asked about an article on his week of testing, it brought up themes he had never mentioned, including digital sovereignty, cloud independence and digital freedom.

The model improved when he challenged these assumptions and gave a more grounded answer after correction. Even so, he had to check its reasoning again and again. He said he often spent as much time judging the model's logic as he spent using its ideas.

He gave Claude the same prompt. According to the author, Claude questioned his framing and suggested that the more useful story was which tasks really need a cloud model. He also reported that Claude stayed closer to the evidence he had supplied, even when he disagreed with it. In his words, talking to Claude felt like a conversation, while the local model felt more like a batch process that got him most of the way there.

The cost showed up in time and in hardware. One Windows summary took roughly five minutes to finish. During that wait, Task Manager showed system memory use at around 97 percent, and the cooling fans ran loudly. Claude completed the same job before he had checked the clock. Over the week he became more careful about prompt length, RAM use and whether a question was worth the wait.

The local setup had clear advantages. The author was no longer uneasy about how much usage he had left on Claude's free plan, and the local model never imposed a limit. He could pick the model, choose the quantisation level and decide when to update. The model also works offline, and local processing keeps prompts on the device, which is a privacy benefit. After the initial setup, he said, the model cost him nothing to run.

The report has limits. It describes one person's test on one laptop with one small model. It does not provide benchmark scores, and the author chose not to run them. Results will vary with other hardware, other local models and other kinds of tasks. The excerpt does not say whether the author plans to return to local AI for specific jobs.

Enjoying this? Get the AI brief

Every Thursday: AI that matters for your home.

✓ Free · seven editions · one click to leave

Topics
Story continues on How-To GeekAdd us as a preferred source on GoogleSee our stories first in Google’s Top Stories.
Get alerts on big newsOnly the big news, at most once a day. No email, no account.

Original reporting by the sources above. Summary by smart-home.space.