Model Routing

TL;DR Local models are how I run my community work, my volunteer projects, my side hustles, and my musings. Two reasons, both simple: Cost. None of this work should bill against a frontier account. Trust. Client and customer data stays on my own disk, away from the labs. Bottom line up front: I am running Qwen3.8-27B (4-bit) on my MacBook Pro M3 Max, 128 GB of unified memory and a 40-core GPU, my personal dev rig, driven by an OpenCode harness. ...

TL;DR My agents do real work on frontier models. Every dollar of that work is metered against my OpenAI and Anthropic bills. Meanwhile a perfectly capable local model, gpt-oss:20b, sits on a Mac Mini in my office costing me nothing. The obvious question: for which tasks could the free local model do the job just as well? Today I answered it with a system instead of a guess. I taught my eval framework to automatically replay every writing task on the local model right after the frontier model runs it, then grade both with the same judge and record the gap. The result across 10 like-for-like rematches: a mean quality delta of -0.05 — statistical parity — and four tasks where the local model actually scored higher than the frontier model, by as much as 1.7 points. ...