Harness engineering: everything around the model

A coding agent is a language model running in a loop. It reads, calls a tool, looks at the result and decides what to do next. The model supplies the judgement. Everything else is the harness: the instructions it loads, the tools it may call, the files that carry its memory between sessions, the tests and linters that tell it when it is wrong, the sandbox that limits what it can break, and the logs that let a person see what happened.

Harness engineering is designing that surrounding system on purpose. Mitchell Hashimoto, who named it in February, put the central habit simply: whenever an agent makes a mistake, change its environment so it can’t make that mistake again, and give it ways to check that it did the right thing.

The term spread fast because it named something teams had already found. The same model with a better harness behaves like a different, much better engineer. With a poor one, it confidently tells you half-built work is finished.

Prompt, then context, then harness

Each phase widened what engineers treated as part of “the AI”. Prompt engineering tuned one message. Context engineering, the term that took over in mid-2025, tuned everything in the model’s window. Harness engineering also takes in everything outside the window: the repository, the tooling, CI, observability and the human review process.

What a harness is made of

Vendors draw the boundary in different places, but the same parts keep coming up. A May paper on arXiv by Zhong and Zhu lists eleven jobs a harness does: task specification, context selection, tool access, project memory, task state, observability, failure attribution, verification, permissions, entropy auditing and intervention recording. They group naturally around the agent loop.

Guides and sensors

The clearest framework so far comes from Birgitta Böckeler at Thoughtworks, writing on martinfowler.com in April. She splits the harness into guides, which steer the agent before it acts, and sensors, which watch after it acts so it can correct itself. Each can be computational, meaning deterministic and cheap, or inferential, meaning another model’s judgement: slower, costlier and probabilistic.

	Guides (before)	Sensors (after)
Computational	Project templates, code generators, a Makefile with one obvious command per job, type definitions	Compilers, type checkers, unit and end-to-end tests, linters, module-boundary checks, a build that finishes in under a minute
Inferential	AGENTS.md, design docs, specs, skills, examples of good code	AI code review, an evaluator agent clicking through the running app, an LLM judge with a rubric

Push work into the computational row wherever you can. Those checks are fast, and an agent can’t argue its way past a failing test. Böckeler also names the weak spot: checking that the software actually does what users need is still largely unsolved, and relying on tests the agent wrote for itself “puts a lot of faith into the AI-generated tests”.

The ratchet

Hashimoto’s version is practical. Every line in the AGENTS.md for Ghostty’s inspector, he writes, exists because of something an agent once got wrong. Next to it he keeps small scripts, such as one that takes a screenshot or runs a filtered set of tests, and the file tells the agent they exist. Each failure tightens the harness by one notch, and it doesn’t slip back.

Four teams, four harnesses

The published case studies differ in scale, but they agree on a lot: short maps instead of long manuals, state written to disk, mechanical checks wherever possible, and a separate judge for anything subjective.

OpenAI: a product with no hand-written code

Ryan Lopopolo’s team started with three engineers and, over five months, shipped a product of about a million lines in roughly 1,500 merged pull requests, with no code typed by hand. What made it work:

• AGENTS.md kept to about 100 lines and used as a table of contents into a structured docs/ folder, which is the real system of record.
• A strict layered architecture (types → config → repo → service → runtime → UI), enforced by mechanical rules and structural tests rather than by asking nicely.
• An app the agent can see: it boots it, drives the UI, and reads its logs, metrics and traces itself.
• Builds kept under a minute. Most human review moved after merge, with people sampling code instead of reading every PR.

Anthropic: engineers working in shifts

Long tasks don’t fit in one context window, so Anthropic treats each session like an engineer starting a shift with no memory of the last one.

• An initializer session writes init.sh, a progress file, a JSON list of 200+ features all marked failing, and the first git commit.
• Every coding session reads the git log and progress notes, runs a smoke test, picks one failing feature, and leaves the repository clean enough to merge.
• The list is JSON because models are less likely to rewrite it than Markdown. Agents may only flip the passes field.
• Testing in a real browser, “as a human user would”, caught bugs that unit tests and curl missed.

Anthropic Labs: planner, generator, evaluator

For building whole apps, a planner turns a one-line prompt into a product spec, a generator builds it, and a separate evaluator uses Playwright on the running app and grades it. The split exists because agents overrate their own work: a skeptical outside judge is easier to tune than a self-critical builder. Before each sprint, the generator and evaluator agree a “contract” of testable criteria. One sprint had 27.

LangChain: same model, better rank

LangChain held the model fixed and changed only the system prompt, tool descriptions and the middleware around model and tool calls. Their coding agent went from 52.8% to 66.5% on Terminal-Bench 2.0, from outside the top 30 to the top 5.

A harness costs more, and delivers more

Anthropic published its costs. A single agent with no harness built a retro game maker in 20 minutes for $9, and its core feature didn’t work. The full three-agent harness took six hours and $200, and the result worked. A later version on a newer model built a music workstation in under four hours for about $125.

Harnesses decay

Models keep improving, and parts of every harness turn into dead weight. Anthropic added context resets because Claude Sonnet 4.5 would rush to wrap up as it sensed its context filling. With Opus 4.5 that behaviour went away and, in their words, “the resets had become dead weight”. With Opus 4.6 the sprint structure went too, because the model could break the work down itself. The planner stayed: without it, the generator scoped too small.

This is the “bitter lesson” argument: hand-built structure tends to lose to raw capability over time. Lopopolo describes the same tension between investing in the harness and investing in training. His answer is to put guardrails into what the model already produces well, such as code, tests and docs, rather than into elaborate external scaffolding that is “prone to being scrapped”.

So treat your harness as a set of guesses about what the model can’t do. When a new model ships, rerun your tasks with pieces removed, and delete whatever no longer pulls its weight.

A first harness in eight steps

1. Write a short map. An AGENTS.md of under a page: how to build, test and run, where the docs live, and the few rules that matter. Link out instead of pasting everything in.
2. Give every job one command. make test, make dev, ./init.sh. Agents waste whole sessions working out how to start an app.
3. Make feedback fast. A build-and-test loop under a minute. Slow checks get skipped.
4. Turn taste into checks. Every review comment you repeat becomes a lint rule, a structural test or a type, with an error message that says how to fix it.
5. Let it see the running thing. A browser driver, logs and screenshots it can call, so “done” means checked, not claimed.
6. Put state on disk. A progress file, a task list and small git commits, so a fresh session can pick up where the last one stopped.
7. Separate the judge. For anything subjective, a second agent with a rubric and examples, tuned to be skeptical.
8. Limit the blast radius. A sandbox with no production credentials, narrowly scoped tokens, and a human sign-off on anything that ships.

Then work the ratchet. Every time the agent gets something wrong, ask which of these eight would have caught it, and add that.

What nobody has solved yet

• Behaviour. Linters and types are easy to harness. Knowing the software does what users need isn’t, and tests written by the same agent share its blind spots.
• Intent. No check catches a requirement nobody wrote down. More of the human’s job becomes writing precise specs and acceptance criteria.
• Coverage. When the checks stay quiet, it’s hard to tell whether quality is high or detection is weak. There’s no agreed way yet to measure how complete a harness is.
• Old code. Codebases without tests or clear boundaries need a harness most and are the hardest to fit one to.

Further reading

• My AI Adoption Journey, Mitchell Hashimoto, 5 Feb 2026
• Harness engineering: leveraging Codex in an agent-first world, Ryan Lopopolo, OpenAI, 11 Feb 2026, and his Latent Space interview
• Effective harnesses for long-running agents, Anthropic, 26 Nov 2025
• Harness design for long-running application development, Anthropic, 24 Mar 2026
• Managed Agents, Anthropic, 8 Apr 2026
• Harness engineering for coding agent users, Birgitta Böckeler, 2 Apr 2026
• Improving Deep Agents with harness engineering, LangChain
• AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents, Zhong and Zhu, 13 May 2026