Same Model, Different Answers: What an AI Harness Actually Is

“Harness” went from nowhere to everywhere this year, and most explanations of it are circular. Here is the version that actually lands.

A harness is everything around the model that turns it into an agent.

The formula people use is Agent = Model + Harness, and LangChain’s definition is the cleanest I have found: every piece of code, configuration and execution logic that is not the model itself.

That is it. The model is the engine. The harness is the rest of the car.

The part that makes it click

Give three tools the same bug and the same model. One fixes it. Two make it up.

Not because the model changed. Because the harness decided what it got to see and what it was allowed to do.

flowchart LR
    P["One prompt:<br/>the login test fails, fix it"]:::starter
    P --> A["Agent CLI<br/>loads project rules,<br/>runs the test,<br/>reads the real error"]:::context
    P --> B["IDE assistant<br/>sees the open file<br/>and nearby code"]:::context
    P --> C["Raw API call<br/>sees one sentence"]:::context
    A --> M1["Same model"]:::gemini
    B --> M2["Same model"]:::gemini
    C --> M3["Same model"]:::gemini
    M1 --> R1["Patch, verified<br/>against real output"]:::action
    M2 --> R2["Plausible patch,<br/>never executed"]:::action
    M3 --> R3["A guess from<br/>training data"]:::action
    classDef starter fill:#FEF3C7,stroke:#D97706,stroke-width:1.5px,color:#1E293B
    classDef context fill:#DBEAFE,stroke:#2563EB,stroke-width:1.5px,color:#1E293B
    classDef gemini fill:#EDE9FE,stroke:#7C3AED,stroke-width:1.5px,color:#1E293B
    classDef action fill:#D1FAE5,stroke:#059669,stroke-width:1.5px,color:#1E293B

Your prompt
The harness
The model
What you get back

Three identical purple boxes. Three different green ones. Everything that produced the difference happened in the blue column.

The bottom lane is worth sitting with, because it is what a chat window is. One sentence in, a guess out. The model is not being lazy there. It genuinely cannot know what your test prints, because nothing ran it.

What a harness is actually made of

Strip it down and you get five jobs:

Context assembly. What goes into the prompt that you did not type. Project conventions, open files, retrieved docs, prior turns.
Tools. What the model can do besides emit text. Run a command, read a file, call an API.
The loop. Feed the tool result back, let the model react, repeat until done.
State. What survives between turns and between sessions.
Policy. What needs approval, what is forbidden, what happens on failure.

Change any one of those and behaviour changes, with the weights untouched.

Here is the one that keeps surprising people: the loop is where verification lives. A harness that can run your test suite produces answers grounded in real output. A harness that cannot is producing fiction that compiles. Same model, both times.

Why everyone is suddenly saying it

Because the model has stopped being the differentiator.

Frontier models are now close enough on raw capability that the gap between two agent products is mostly harness: which tools it has, how it recovers from an error, what it remembers, what it is allowed to do without asking you first.

The uncomfortable version: most “our AI agent is better” claims are really “our harness is better.” And since the harness is ordinary software, that gap is one you can close yourself.

The honest caveat

“The harness matters more than the model” is a practitioner argument, not settled fact. The loudest sources are vendor blogs and preprints written by people with an interest in the conclusion.

It is directionally well supported and oversold at the edges. A better harness will not make a weak model strong. It will reliably make a strong model useful, which is a different and more practical claim.

You are probably already writing one

This is the part I did not expect when the term clicked for me.

If you keep a conventions file that your coding agent reads at startup, that is context assembly. If you have written a task-specific procedure for it to follow, that is a harness component. If you have told it which commands it may run unattended and which need you, that is policy.

I have all three in my own repos, and I had been thinking of them as notes. They are not notes. They are the harness, and they are why the same model behaves one way in my project and another way in yours.

The sharpest lesson I have had from this: I once shipped a fix and reported it green, because I checked a command’s output by predicting it rather than running it. A reviewer caught it. The fix was not a better prompt or a better model. It was changing the rule to check exit codes, never grep the output for words. That is a harness change, and it is the kind that compounds.

The one thing to take away

When an AI tool disappoints you, the useful question is not “which model should I switch to.” It is:

What did this thing actually see, and what was it allowed to do?

Nine times in ten the answer is “less than you assumed,” and that is a harness problem you can fix.

Related, if you want the adjacent pieces: building an MCP server is harness work (you are adding tools), prompt injection is a harness attack surface (untrusted text reaching the loop), and how I actually use AI to ship code is the workflow version of the same argument.