It's a seductive idea. One smart layer sits in front of every model — Claude, GPT, Gemini — and for each coding task it automatically routes to the cheapest model that can actually do the job, minimizing your cost per accepted pull request. Set it up once, never think about model choice again, watch the bill drop.
I've watched builders spend weeks chasing this. It's worth understanding why the "master orchestrator" is a trap before you build one — and what the boring alternative that actually works looks like. This expands the orchestrator section of the flagship guide into its own argument, because it's the single most expensive detour in AI-assisted coding.
What actually exists: routers, not oracles
To be clear, routing infrastructure is real and useful. LiteLLM supports weighted, rate-limit-aware, latency-based, least-busy, and cost-based routing strategies. OpenRouter gives you one API key, one billing surface, a model catalog, and usage visibility across providers. These are genuinely good pieces of infrastructure — one integration point, unified billing, failover.
But look closely at what they route on: rules and signals you define — cost, latency, load, weights. They don't understand your task. A router can send a request to the cheapest available model, or the fastest, or the least rate-limited. What it cannot do is look at "refactor the auth flow to support SSO" and know whether that's a Haiku job or an Opus job. That judgment is the entire problem, and it's the part the router leaves to you.
That's the gap between a router and the "master orchestrator" people imagine. The infrastructure is real. The oracle is not.
Why static routing rules fail
The next instinct is to encode the judgment as static rules: "Claude for code, Gemini for long-context reading, GPT for reasoning." It feels principled. It doesn't hold up, for a simple reason: a task's difficulty isn't reliably knowable before you attempt it.
"Fix this failing test" can be a one-line change or a three-hour architectural untangling, and they look identical at routing time. A static rule that sends every "fix" to a cheap model will nail the easy ones and quietly produce broken, expensive retry loops on the hard ones — which is exactly the cheap-model false economy the flagship warns about, now automated and running without you watching.
The research points the same way. Work on agentic model routing for coding tasks supports the core intuition — different models genuinely do better on different task types, so routing can help — but it also implies that good routing needs feedback from execution results, not a static "this model for that label" lookup. A router that can see whether the last attempt compiled, passed tests, and got merged can learn to route. A router applying fixed rules to unseen tasks is guessing with extra steps. And building the feedback-driven version is a serious research project, not a config file.
The hidden cost of the orchestrator dream
Here's the part that stings. The "master orchestrator" is exactly the kind of secret-knowledge shortcut that sells — a clever meta-system that promises to make the hard problem disappear. But the weeks you spend building routing logic are weeks you're not shipping product. And your real bottleneck almost certainly isn't access to one more model or a smarter router. It's project discipline: narrow specs, clean repos, tests, task scoping, branch hygiene.
The orchestrator dream is appealing precisely because it reframes a discipline problem as a tooling problem — and tooling problems feel solvable by building more tooling. They rarely are. If your AI coding spend is high, a router won't fix it; better-scoped tasks and a review step will.
What actually lowers cost per accepted change
The thing that beats a master orchestrator is embarrassingly low-tech: a manual policy you run yourself. From the flagship, restated as an operating rule:
- Implement with one strong agent. Claude Code for the build — bounded task, repo, tests, permission to edit.
- Review with a second, independent model when the stakes are high. Have Codex review the PR, write the tests, or solve the same issue in a separate branch. A second model catching the first one's mistakes does more for your accepted-change rate than any router.
- Use a cheap, high-context model as a scout. Gemini/Antigravity for reading lots of files, summarizing architecture, generating alternatives — the mechanical, low-stakes work where cheap genuinely means cheap.
- Match model and effort to the difficulty of the actual step — frontier for architecture, auth, payments, migrations, and final review; cheap for summarization, test scaffolding, and file discovery.
This is "routing," but the router is your judgment, informed by seeing the task — which is exactly the signal a static automated router lacks. It costs you a few seconds of decision per task and saves you the retry loops that a wrong automated route would have generated silently.
When routing does make sense
None of this means routing is useless — it means you're pointing it at the wrong layer. API-level routing belongs inside your product's inference, not as your personal coding control plane. If your SaaS serves LLM calls to users, a router is the right tool to manage cost, latency, failover, and provider availability across that traffic — because there you have the feedback loop (you can measure outcomes at scale) and the volume to justify the engineering.
The distinction is clean: route your product's production traffic, where you can measure and optimize against real outcomes. Don't build a router to pick models for your own dev workflow, where your in-the-moment judgment is both cheaper and better than any rule you could write today.
FAQ
Do I need a router or "master orchestrator" to control my AI coding costs?
No. A manual policy — implement with one agent, review with a second, scout with a cheap model, and match model to task difficulty — lowers cost per accepted change more reliably than an automated router, which can't judge task difficulty and will silently mis-route the hard ones.
Aren't LiteLLM and OpenRouter worth using, then?
Yes — as infrastructure. One API key, unified billing, usage visibility, and failover are real benefits. Just don't expect them to choose models intelligently for you; they route on cost, latency, and load, not on understanding your task.
Why can't static rules like "Claude for code, GPT for reasoning" work?
Because a task's difficulty isn't knowable before you attempt it. "Fix this bug" can be trivial or architectural, and they look the same at routing time. Static rules nail the easy cases and produce expensive broken retries on the hard ones.
When is model routing actually the right tool?
Inside your product's inference, where you serve LLM calls at volume and can measure outcomes. There you have the feedback loop and scale that justify routing. Your own coding workflow has neither — your judgment is cheaper and better.
What should I do instead of building an orchestrator?
Pick one repo, write a lean instructions file, scope tasks narrowly, implement with one agent, and add a second model as a reviewer. That boring discipline beats any router — see the flagship guide for the full stack.
Where to go from here
The master orchestrator is a research spiral dressed up as a shortcut. The unglamorous truth is that the biggest lever on your AI coding bill is discipline — clean repos, narrow specs, tests, and a second model reviewing the first — not a smarter router. Save the routing for your product, where it belongs.
For the complete argument on model choice and the tokens-to-revenue picture, start with Which AI Model Should You Actually Use to Build Software?, then work with me or subscribe to the newsletter for more field notes from building in public.