The 14-billion-parameter model that couldn't -- and the one that could

A bigger model is not automatically the right model. A documented run in this operation handed a broad, open-ended prompt to a 14-billion and a 30-billion parameter local model and watched both fail it -- then handed a small, narrowly scoped, schema-constrained task to the same 14-billion model and watched it pass. The lesson was not "upgrade the model." It was "match the task to what the tool can actually do," and that lesson only got learned because someone was willing to run the failing case on purpose and write down what broke.

The human moment

I was staring at a run log that should have been boring. A local model, running well within its advertised capability, had just been asked to reason broadly across an open-ended prompt -- the kind of task a person would call "just figure it out." It didn't figure it out. The output wandered, missed the actual ask, and would have shipped wrong if nothing downstream was checking it. The next run down the log, the same model was asked something much smaller and much more specific, boxed into a strict schema. It nailed it, every time.

The gap seen -- opportunity taken

The easy, comfortable move here was to treat the failure as a benchmark problem: this model isn't good enough, try a bigger one, throw more compute at it until the failure stops showing up in the log. That is the "good enough" path most operations take, and it quietly trades a real diagnosis for a bigger bill.

The gap I saw was that the failure wasn't about model size at all -- it was about task shape. A model that cannot reliably reason across an unconstrained prompt can still be completely reliable at a narrow, well-defined one, and routing broad, ambiguous work to a small, fast, cheap model while reserving larger or more capable calls for what actually needs them is a design decision, not a consolation prize. Recording the failing case in the run log, instead of quietly retrying until something worked and forgetting the miss ever happened, is what turned a bad run into a routing rule.

Teach the concept -- deterministic-first, model calls bounded by task shape

Before any model gets a turn, this operation runs a deterministic-first pass: plain scanning, scripted checks, and scoped bounded model calls handle whatever a script or a smaller check can answer more cheaply and more reliably than a large model would. A model is only asked the part of the problem that a human or a script genuinely cannot answer more cheaply. That ordering matters more than which model eventually gets called, because it shrinks the surface area where a model's judgment -- and a model's failure mode -- can actually reach the output.

The same discipline extends past which model answers a prompt, into what happens when a provider itself is unavailable. A resilience standard -- model and vendor fallback wired in for AI-dependent work -- is verified, end to end, in at least one product in this portfolio's own code: a router and a circuit breaker that hold when a preferred provider does not. It is not yet verified everywhere in the portfolio, and this chapter says so plainly rather than rounding a partial rollout up to a universal guarantee.

Why it matters -- what breaks without this discipline

Skip the deterministic-first ordering and every task, no matter how small, gets routed to the most expensive, least predictable tool available, because "just ask the model" is the path of least resistance. The failure that follows is quiet and expensive at the same time: cost climbs on tasks that never needed a large model's judgment, and the tasks that genuinely did need it get buried in noise from all the ones that didn't. Skip recording a model's failure honestly, and the same wrong-shaped task gets handed to the same wrong-shaped model again next quarter, because nobody wrote down that it failed the first time.

How I approached it -- the tradeoff accepted

The simpler path was one model, called the same way for every task, regardless of whether the task actually needed that much capability. What I gave up by rejecting that path is administrative simplicity -- there is now a routing decision to make and a run log to keep honest, instead of one call site everyone uses out of habit. What I bought is a system that spends model capability where the task shape actually requires it, and a documented record of exactly where the fallback standard is proven and where it is still owed.

What to take to your own work

Before you hand any task to an AI model, ask what shape the task actually is -- broad and judgment-heavy, or small and constrained -- and route accordingly instead of defaulting to whichever tool is already open. Then test your own resilience claims the same way: if you say you have a fallback, prove it by removing the primary and watching what happens, and report the result exactly as it happened, not as you hoped it would.

Evidence: The model-tier pass/fail finding (broad prompts against 14B/30B local models failing; small, schema-constrained tasks on the same model class passing) is recorded in docs/technology-portfolio/completed-discovery-runs.md, Run 0B, and corroborated by claims-ledger row C-20 (verified). The deterministic-first, bounded-model-call design is corroborated by claims-ledger row C-02 (verified), drawn from the same discovery-run record. The fallback/circuit- breaker resilience standard is corroborated by claims-ledger row C-40: owner-attested, and verified in code for AI Field Support (providers/router.js, providers/circuit-breaker.js); this chapter states that per-product verification explicitly rather than generalizing it portfolio-wide, per the claim's own open resolution note. Evidence class: internal operating record (discovery-run defect log) plus verified code for the fallback claim, and owner attestation for the parts not yet independently re-verified across every product; verified 2026-08-25. Full record: docs/technology-portfolio/completed-discovery-runs.md.