Your policy is secretly a harness: the decision geometry of agent interfaces

The problem

Language model agents are post-trained and deployed through a harness, which encompasses the prompts, tools, and control logic that present tasks to the model and execute its outputs (arxiv.org). Harnesses are frequently revised and judged by how an already trained model performs under them, which assumes that what a model learns under one harness carries over to the next (arxiv.org). However, prior work notes that the same interface change can raise one model's accuracy while lowering that of a fine-tuned model (arxiv.org). Automated harness design scores each candidate by how a fixed model performs, relying on the assumption that earlier learning transfers successfully (arxiv.org).

The idea

The authors ask what a policy learns from its harness and what survives when the harness changes (arxiv.org). They formalize interfaces and examine how training updates policies under KL-regularized control (arxiv.org). Deployed return is shown to equal the ceiling of the harness—the best return it offers—minus the mismatch of the policy, which is the return lost to its own suboptimal decisions (arxiv.org). The authors perform a controlled study using six harnesses acting on three MiniWoB++ task families, post-training Qwen3.5-2B with exact policy gradients where the frozen base model serves as the reference (arxiv.org). They also evaluate coding-agent harnesses on Terminal-Bench Lite (arxiv.org).

The results

In MiniWoB++, post-training makes a model's preferences specific to its training harness (arxiv.org). Reversing the order of observation fields leaves every optimal decision unchanged yet lowers a trained model's success from 85.0% to 19.2% (arxiv.org). Conversely, every new harness that a fixed-model comparison rejects but that raises the ceiling outperforms the original agent after further training under it, with ceiling and mismatch measured beforehand predicting these gains (arxiv.org). For coding-agent harnesses on Terminal-Bench Lite, ceiling minus mismatch estimated with a stronger model as a stand-in for the optimum reproduces the accuracy ranking, whereas either term alone does not (arxiv.org).

Why it matters

Understanding the decision geometry of agent interfaces clarifies why transferring trained models across different harnesses often fails unexpectedly. By splitting performance into a harness ceiling and a policy mismatch, developers can better predict whether further training under a revised harness will yield performance gains, avoiding flawed evaluations that mix what a harness offers with how well a specific checkpoint uses it (arxiv.org).

Caveats

This is a preprint study (arxiv.org). The authors note limitations in their experimental setups, such as relying on a stronger teacher model as a stand-in for the optimum in settings like Terminal-Bench Lite where no true optimum is directly available (arxiv.org).