An AI orchestrator is the part of an AI system that decides how a task gets done. It breaks your request into steps, picks which model runs each step, and rewrites the plan when step three goes wrong. A to-do list planner, except the to-do list rewrites itself as it goes.
It is also the thing that makes a handful of agents into a system rather than a script. Something has to decide which agent runs next, on what information, and whether the last one’s output was good enough to carry on from. That is the orchestrator.
Orchestrator vs harness: what is the difference?
The harness is the body an agent acts through. The orchestrator is the brain that decides what to do with it.
| Harness | Orchestrator | |
|---|---|---|
| What it is | Tools, permissions, the environment, the loop that keeps a model going | The plan: what this task actually consists of |
| What it decides | Whether an action is allowed to run, and what happens when it fails | Which steps happen, in what order, and which model runs each one |
| What it fails at | A tool that breaks or a permission drawn too tight | A bad plan: right steps in the wrong order, or a step it never thinks to revisit |
Harnesses were the last lesson, and the two get confused constantly. The distinction earns its keep the moment a run goes wrong, because it tells you where to look. Nothing happened at all, or the agent was blocked halfway: harness. Five steps ran cleanly and three of them were the wrong steps: orchestrator.
Why does an AI orchestrator use multiple models?
At different levels of the orchestration you can run different models, and that is where your bill is decided.
Ask Claude Code something like “how does our billing system work” and one sentence turns into a sequence. The expensive model goes first, because that question needs research before anything else can happen. It searches every file for the phrases that would appear in billing code, reads what comes back, builds a picture of how the thing hangs together, and often ends with an implementation plan for the bug you reported. Reasoning about what needs doing, in other words, and then working out the order.
Then the orchestrator looks at the next piece of work and decides it is simple. The plan already exists. Applying it does not need the model that wrote it, so the orchestrator spins off a cheaper, smaller one to implement.
The reason is tokens. You are charged by the number of tokens used, and the two halves of a job are not remotely symmetrical. Research is mostly reading: input tokens, the cheap ones. Implementation is writing, and output tokens cost three to five times what input tokens cost. A model that touches ten files has to read all ten in and write all ten back out again, changed. Asking someone what a book is about gets you a skim, whereas asking them to edit it means going through the thing line by line.
So the routing rule is: the expensive model does the work that decides what happens, and the cheap model does the work that has already been decided. In practice a run starts on Opus 4.8, which understands the problem, interrogates you for the details you left out, and finalises the plan. Sonnet or Haiku makes the changes. The small model never needs the whole picture. It needs one objective it can hold in its head: swap out this mechanism for that one.
None of that is a coding problem. The same split shows up when you ask an agent why the same three invoices went out twice last month. Working out what happened and what the rule should be is the expensive half. Applying the fix to every affected record is the half you want running on the cheap model, because it is the half that writes.
What does an orchestrator plan look like?
Give an orchestrator the job of writing a personalised outbound email and it reasons with itself first, then splits the job into five tasks:
- Look up the prospect.
- Look up the company.
- Look up past conversations.
- Draft the email.
- Send it.
The orchestrator builds that plan. A smaller model executes it, and you get a better output, a quicker output, and a lot fewer tokens spent on it.
That example is linear. Most real work is not. The plan says run steps one through five, and then at step three something goes wrong, or information turns up that changes what steps four and five should have been. The orchestrator goes back and forth with the models it delegated to and reshuffles whatever is left. Planning, doing, and changing the plan.
Can you interrupt an AI agent mid-task?
If it is orchestrated, yes. Between turns the orchestrator reasons again, which means you can type into a job that is already running. Add a requirement, add a constraint, correct what you asked for the first time, and your prompt is not sitting in a queue waiting for the run to finish. It goes into the same system that decides where the agents go next, so the steps that have not run yet get rewritten around it.
Memory is what lets the orchestrator remember step one by the time it reaches step five. That is the next lesson.
