Why the harness matters more than you think
Two agents built on the same model can behave very differently. One finishes the task; the other loops, calls the wrong tool or confidently reports success after failing. The difference is rarely the model. It's what the model is given to work with: which tools it can call and how they're described, what context it sees at each step, when it has to stop, and how its output is checked before anything happens.
That surrounding system is the harness. Most of the effort in taking an agent from demo to production goes into it, and it's also where most production failures are fixed.
The parts of an agent harness
- The agent loop. The cycle of plan, act, observe and repeat, with clear stop conditions: a step limit, a time and token budget, and a definition of done.
- Tools. Narrow, well-described actions with typed inputs, validation and error messages the model can act on. Write tools are idempotent where possible, so a retry can't do the same thing twice.
- Context management. What goes into the model at each step: instructions, retrieved documents, tool results and memory, kept within budget by summarising or dropping what no longer matters.
- Permissions and sandboxing. Least-privilege credentials, read tools separated from write tools, code run in a sandbox, and secrets kept out of the model's context.
- Guardrails and human approval. Checks on inputs and outputs, and an approval step before anything that sends, pays, deletes or changes customer data.
- Evaluations. A test set built from real cases, scored automatically on every prompt, tool or model change, so quality is measured rather than guessed.
- Observability. A trace of every run: each step, tool call, input, output, token count, cost and latency, so any failure can be replayed and explained.
- Failure handling. Timeouts, retries with backoff, fallback models, checkpoints so long runs can resume, and a clean hand-off to a person when the agent is unsure.
- Cost and latency control. Smaller models for simple steps, caching of repeated context, and per-run budgets with alerts.
Symptoms that point to the harness
When an agent misbehaves, the fix is usually in the harness, not a bigger model or a longer prompt.
| Symptom | Likely cause | Harness fix |
|---|---|---|
| Works in the demo, fails on real data | Quality was never measured on real cases | Build an evaluation set from real examples and gate every change on it |
| Loops or never finishes | No stop conditions | Step limits, budgets and an explicit definition of done |
| Calls the wrong tool or passes bad arguments | Tools are too broad or poorly described | Split tools into narrow actions with typed, validated inputs and clear descriptions |
| Forgets instructions in long tasks | Context fills up with stale tool output | Summarise or trim old steps and re-inject key instructions |
| Costs spike without warning | Every step uses the largest model with full context | Route simple steps to smaller models, cache context, set per-run budgets |
| Nobody can explain a bad result | No tracing | Log every step, tool call and output so runs can be replayed |
| Took an action it shouldn't have | Broad permissions, no approval step | Least-privilege tools and human approval for risky actions |
How to start engineering a harness
- Pick one workflow with a clear owner and a measurable outcome.
- Write down what a correct result looks like, and collect 20–50 real examples as the first evaluation set.
- Start with read-only tools and add tracing from the first run.
- Add stop conditions and budgets before you add more capability.
- Add write tools one at a time, each behind an approval step.
- Roll out in stages: shadow mode, then a share of real traffic, then full volume once the evaluation scores hold.
How Expedey approaches harness engineering
Expedey builds production AI agents and engineers the harness for each one: tool layers on real APIs, evaluation sets from client data, approval steps for risky actions, and tracing, cost reporting and failure handling from the first build. One example is a multi-agent LangGraph pipeline that reads government solicitation documents, extracts and de-duplicates requirements, and drafts proposal sections.
