The demo gap
A demo proves an agent can do a task once. Production asks it to do the task thousands of times, on inputs nobody prepared, while connected to systems that sometimes fail. The model is usually the same in both. What changes is everything around it: the harness.
Seven failure modes and their fixes
| What breaks | Why | The harness fix |
|---|---|---|
| Real inputs are messier than the demo | Scanned PDFs, typos, missing fields, mixed languages | An evaluation set built from real cases, scored on every change |
| Long tasks lose the plot | Old tool output crowds out the instructions | Context management: trim or summarise old steps, keep the instructions |
| One failed API call ends the run | No retries, no fallback, errors raised as exceptions | Retries with backoff, fallback models, errors returned to the model as messages |
| The agent loops | No stop conditions | Step limits, repeated-call detection and a definition of done |
| Costs climb with usage | Every step uses the biggest model with the full history | Budgets per run, smaller models for simple steps, tracked cost per task |
| A risky action happens unchecked | Broad permissions; policy only in the prompt | Least-privilege tools, approval for writes, policy enforced in the tool |
| Nobody can say if a change helped | No traces, no metrics | A trace for every run and a pass rate for every release |
A demo-to-production checklist
- 20-50 real cases with expected behaviour, scored automatically.
- A step limit, a token or cost budget and a time limit on every run.
- Typed, validated tools; write tools behind approval; rules enforced in the tools.
- Retries for transient errors and at least one fallback model.
- Old context trimmed before each model call.
- A trace per run with tokens, cost, latency and every tool call.
- A way for the agent to hand off to a person, and a staged rollout: shadow mode, then partial traffic, then full.
