dreythomas
← All daily notes

Local models are not as sharp, and that is fine

A model running on your own laptop is almost always smaller than the big models running in a company's data center, and it shows. It may miss context, give a shorter answer, or need a clearer prompt to get anywhere useful. That is not a flaw in your setup. It is the tradeoff for keeping everything private and running on hardware you own. I keep a mental note of the kinds of prompts my local model handles well versus the ones where it visibly struggles.

The trick is matching the tool to the job. A local model handles a quick rewrite, a summary of your own notes, or a first pass on an idea just fine. Ask it to do deep research or handle a long, complicated document, and you will feel the gap right away.

Once you stop expecting a local model to match a cloud one, the disappointment goes away and you start using it for what it is actually good at.

Try this today: give your local model one small, well-defined task, like shortening a paragraph, and judge it only against that job.

Want to build one of these with me?

Join the Fall 2026 Coaching Group, seven live Thursdays where we build AI systems on your business, or apply for a 1:1 Build-With-Me Session.

Pick your door →