The World That Fights Back

Chapter 1: Wrap, Don't Rebuild

Before you can wrap an environment, you need to know exactly what surface you are wrapping. So start where every agent framework starts: with the handful of function calls that every agent environment — a text adventure, a Docker container full of source code, a live browser session — already exposes. This chapter builds the formal object, then the software contract, then the wrapping transformation, and ends with the property that makes the whole paper trustworthy: the verifier survives.

The interface every environment already speaks

Strip any agent benchmark down to its plumbing and you find the same loop, standardized years ago by the Gym / Gymnasium tradition in reinforcement learning:

The crucial observation is sociological as much as technical: this interface is already universal. Benchmark authors converged on it independently, because the agent loop demands it. Which means a layer that operates purely on these calls — intercepting a reset here, filtering a step there — inherits universality for free. That is the entire strategic bet of the paper, and it is worth pausing on: EnvHarness does not ask environments to change; it exploits the one thing they already share.

The environment as a tuple — six symbols

To reason precisely about what a wrapper may and may not touch, the paper models an environment as a tuple:

Read every symbol, because the components of Chapter 2 are defined by which of these they transform. S is the state space — every configuration the world can be in (every arrangement of objects in the house, every state of the repository's files). A is the action space — what the agent may do (go to, take, open; run a bash command; click an element). O is the observation space — what the agent gets to see, which is generally less than the state (the room description mentions what is visible, not what is inside closed drawers). T : S × A → S is the transition function — the hardcoded logic mapping a state and an action to the next state. R is the reward induced by the verifier — the success signal. And s₀ is the initial state that reset() constructs.

Notice the separation the tuple already encodes. The state S is the world; the observation O is the window; the transition T is the physics; the reward R is the judge. Hand-built environments hardcode all four — and the insight of this paper is that an external layer can re-window (O), re-referee the rules of play (A, T), and re-position the starting point (s₀), all without ever entering the room where S, the native T, and R actually live.

The wrapping transformation

Formally, an EnvHarness component is an environment-agnostic transformation w:

with a strict discipline attached: w reshapes the environment at the interface level only, never modifying the underlying simulator backend or implementation. In practice, w customizes the initial state (s′₀), filters the exposed spaces (A′, O′), and updates transition mechanics (T′) — meaning what the agent's actions are allowed to do and what responses come back, imposed from outside, on top of the native physics. Because every intervention remains external, the ground-truth evaluation logic is preserved: the original verifier can still score the episode.

One more property hides in the phrase "environment-agnostic": w is defined as a transformation of the environment alone. A component that hides the mug in a drawer works identically no matter which policy plays the episode. The choice of which component to apply will be conditioned on a specific policy's flaws (that is Chapter 3's job), but the component itself is a reusable, portable object — which is what lets accepted components accumulate into a growing harness.

Realization: the ActionableEnv contract

Concept is cheap; the paper ships the interface as code, and the details are where the engineering lives. Every environment in the system implements one abstract class:

# The contract every environment (and every wrapper) satisfies
class ActionableEnv(ABC):
    # --- the interaction loop ---
    def reset(self, seed, options) -> Observation
    def step(self, action: Action) -> EnvResponse
        # Action = tool name + JSON-serializable kwargs
        # EnvResponse = Pydantic wrapper of the Gymnasium 5-tuple:
        #   (observation, reward, terminated, truncated, info)
    def observe(self) -> Observation      # re-read world WITHOUT resetting
    def evaluate(self) -> EvaluationResult # the verifier's terminal verdict
    def get_env_state(self) -> EnvState    # runtime-safe view: plain data,
        # no Docker handles, no browser pages, no sockets
    # --- persistence (environment-owned) ---
    def save_state(self) -> dict           # JSON-serializable
    def from_state(cls, d: dict) -> "ActionableEnv"
    # --- optional capabilities, safe defaults ---
    def step_reward(self, step_info)       # dense reward hook, non-fatal
    def notify_replay_complete(self)       # rewind per-episode bookkeeping
    def list_tasks(self);  def close(self)

Two type names in that contract carry more weight than their brevity suggests. An Action is a tool name plus JSON-serializable keyword arguments — the lingua franca of 2026 function-calling agents, chosen so any policy that can emit a tool call can drive any Bridge. An EnvResponse is a Pydantic wrapper around the Gymnasium 5-tuple — typed and validated, so a malformed response from a buggy layer fails loudly at the boundary instead of silently corrupting a trajectory three fields downstream. Typed contracts at trust boundaries: the paper's engineering is conservative everywhere its ideas are radical.

Three of these choices are subtle enough to deserve their own paragraphs, because each one exists to make wrapping possible.

observe() is deliberately separated from reset(). Why have both? Because a component may mutate the environment after reset returns but before the policy acts — that is exactly how a Stage will work in Chapter 2. After the mutation, the outer layer needs to re-read the world to produce the episode's true first observation, and observe() does that without paying for (and without destroying the mutation with) another reset. An interface designed only for playing episodes would never include this method; it exists because the interface was designed for reshaping them.

get_env_state() returns a runtime-safe view — plain data with no Docker handles, browser pages, or sockets — and it is the only state that component hooks are permitted to read. This restriction is what makes component code portable: the same hook that runs against an in-memory puzzle also runs against a containerized repository, because neither ever touches the runtime beneath the state view. It is the classic API discipline — program against the schema, not the implementation — applied to environment internals.

Persistence is environment-owned and intentionally unprescriptive. save_state() returns a JSON dictionary; from_state() rebuilds an instance. A pure in-memory environment serializes its full live state; an environment backed by a container or browser — whose runtime cannot be cheaply cloned — saves only its reset arguments and accepts that restoration is valid at episode boundaries. Each concrete class registers a stable string tag through a registry decorator, so a saved stack can be reconstructed without embedding import paths in checkpoint files.

Bridges: seven benchmarks, one contract

Real benchmarks do not natively speak ActionableEnv, so each one gets a Bridge — its direct implementation of the contract, and the only layer in the entire system aware of the underlying runtime. The paper implements seven Bridges spanning four fundamentally different runtime classes:

Count them off, because "seven Bridges" will recur: Toy24 (the in-memory development environment), ALFWorld, SWE-bench, OfficeQA, SpreadsheetBench, WebArena, and WebShop — the last joining for Chapter 8's reinforcement-learning experiments. Four runtime classes, seven adapters, one contract.

Everything above the Bridge — the policy loop, the orchestrator, and all component code — is shared verbatim across all seven environments. Feel the weight of that sentence: the same Python that reshapes a text adventure reshapes a Docker container full of Django source, unchanged. That is what "domain-agnostic" means when it is implemented rather than merely claimed.

Each Bridge also declares its action space as a tool registry of typed tools, whose signatures are introspected into function-calling schemas for the policy's prompt — and publishes a human-readable env_state_schema() describing the fields component hooks may read. That schema is injected into the designer agent's prompt in Chapter 4, closing a neat loop: what a Bridge exposes is exactly what generated component code can rely on, and the generator is told so.

The registry hides one more design decision worth noticing, because it shows the authors resisting a tempting over-abstraction. The registry serves two roles that the design deliberately decouples: schema generation is universal, while dispatch through the registry is optional. Toy24, the in-memory arithmetic game, routes step() through the registry, because its state simply is the runtime — a Python object the tools can act on directly. But ALFWorld, SWE-bench, and WebArena bypass the registry and drive their engine handle directly, since a TextWorld engine or a browser session cannot be threaded through the data-only state view. A purist design would have forced everything through one dispatch path and broken on the first real benchmark; this one lets each Bridge choose, and keeps the contract uniform where uniformity actually pays — at the interface the components see.

Bridges likewise choose their own persistence granularity along the two patterns from above:

The whole architecture in one diagram's worth of words

The paper's class-structure figure compresses the system into three tiers; hold them as a vertical stack, because every later chapter names one of the tiers. At the bottom, seven Bridges adapt heterogeneous native runtimes to the one abstract ActionableEnv contract — and the figure marks explicitly that further Bridges plug in the same way; the family is open. In the middle, EnvHarness as an abstract decorator over the same contract, with the three shipped components deriving from it — each holding its wrapped environment as an inner field and, by construction, never accessing the runtime beneath the interface. At the top, the instance view: one live environment is an ordered stack — e.g. Rules over Setups over a Bridge — where each layer sees only the layer below, and the outermost face is indistinguishable from a bare environment.

Two sentences from the paper's description are the architecture's oath, worth quoting exactly: a policy, an orchestrator, or a component layer "programs against one abstract type and cannot distinguish a raw benchmark from a benchmark wrapped in an arbitrary stack of EnvHarness components," and "any class honoring the contract is a valid component, and the family is open to extension." Uniformity at the interface, freedom below it, extension above it — the whole design in one breath.

What the verifier sees — and what it never sees

One more walk along the interface, this time following the judge. When an episode ends, evaluate() is called on the outermost layer of whatever stack is present, and by the decorator discipline it delegates inward until it reaches the Bridge — where the benchmark's own evaluation logic runs: the hidden test suite executes, the game engine checks its goal, the site state is inspected. The verifier therefore sees exactly what it was built to see: the final state of its own environment, reached through its own transition function.

What it never sees: the harness. No component intercepts evaluate() — there is nothing for a component to add, and everything for it to corrupt, so the method passes through untouched by convention and (in the designer's case, Chapter 4) by enforced omission. The returned EvaluationResult flows back out through the stack unmodified. Keep this picture — verdicts always bubbling up from the bottom layer, interventions always happening above it — because it is the one-sentence answer to every "but couldn't the wrapper cheat?" objection the rest of the lesson might tempt you toward.

Before the data flow, one more word on the smallest method in the contract, because its existence pattern repeats across good infrastructure: step_reward(step_info), the optional dense-reward hook, is non-fatal by contract — exceptions inside it are recorded but never episode-terminating. Optional capabilities that can crash the mandatory path are not optional; the contract's authors knew it, and wrote the failure semantics into the interface rather than into documentation nobody reads.

A step() call, end to end — the data flow

Make the abstraction concrete with an actual round trip, in the shape the system really uses. The agent, working a SWE-bench task, emits an action; here is what crosses the interface, first with no harness, then with a Contract installed (the very Contract you will meet in Chapter 4):

# ---- agent emits an Action ----
{ "name": "bash", "kwargs": { "command": "git diff | submit_patch" } }

# ---- no harness: EnvResponse straight from the Bridge ----
{ "observation": { "text": "patch submitted." },
  "reward": 0.0, "terminated": true, "truncated": false,
  "info": { "exit_code": 0 } }
# verifier will now grade the patch — agent never ran the tests. It fails.

# ---- same Action, Contract interposed on the T axis ----
{ "observation": { "text": "githook: pre-commit hook 'verify-tests' failed.
    Run the test suite before submitting." },
  "reward": 0.0, "terminated": false, "truncated": false,
  "info": { "exit_code": 1 } }
# the episode continues; the shortcut is now a dead end; the habit
# the agent must learn — verify before submitting — is the only way out.

Study what changed and what did not, field by field. The observation text now carries the fabricated githook message; terminated flipped from true to false (the episode continues); the info exit code reads failure. The action format is identical; the response format is identical; the underlying repository was never touched (the Contract faked the githook message in the response, rather than installing a real hook in the container). The agent cannot tell whether it is facing a strict environment or a wrapped lenient one — and that indistinguishability is the design. From the policy's side of the interface, the harness does not exist; there is only a world with different physics.

A reset, wrapped — the other half of the data flow

The step() trace showed the Contract's half of the story; here is the Stage's, because the reset path is where the observe()/reset() split earns its existence. Watch the call order when a Stage wraps an ALFWorld episode:

# policy calls reset() on the OUTERMOST layer (the Stage)
1. Stage.reset(seed=7)
2.   -> inner.reset(seed=7)          # Bridge builds s0: mug on countertop
3.   -> inner.step("take mug 1")      # replay of delta begins — ordinary,
4.   -> inner.step("open drawer 1")   #   legal transitions through T
5.   -> inner.step("put mug 1 in drawer 1")
6.   -> inner.step("close drawer 1")
7.   -> inner.notify_replay_complete()  # rewind step budget & guards:
                                        # the agent isn't charged 4 steps
8.   -> obs = inner.observe()        # re-read WITHOUT resetting —
                                     # a reset here would undo steps 3-6!
9. return obs                        # the episode's first observation:
                                     # a room with no mug in sight

Line 8 is the one to stare at. A naive interface with only reset() would leave the wrapper no way to produce the post-mutation observation — calling reset() again would rebuild s₀ and erase the replay. The observe() method exists precisely so a layer can mutate the world after reset returns but before the policy acts, then re-read it cheaply. Interfaces reveal their intended uses in their smallest methods; this contract was designed by people who knew wrapping was coming.

Why not the two obvious alternatives?

The paper positions the wrapping move against the two paths the field was already walking, and the contrasts sharpen the design (they are spelled out in its Appendix B):

There is a common thread: each alternative pays for flexibility with trust — trust in the transitions, in the state logic, or in the verifier. The interface-level wrap is the one point in the design space where flexibility is gained and trust is inherited rather than spent. Everything the harness does is expressible as: legal calls, in a different order, with a filter in between.

A closing thought for this chapter, because it explains a strategic choice that might otherwise look like modesty. The paper could have proposed a new environment standard — a richer interface purpose-built for reshaping — and asked benchmark authors to adopt it. It did the opposite: it took the interface the ecosystem already converged on, added nothing to what environments must implement, and put all the new machinery on its own side of the boundary. That is why adoption costs a Bridge (an afternoon) instead of a migration (a community). Interfaces are moats when you own them and bridges when you honor them; this paper chose the bridge, and the five-benchmark table in Chapter 6 is what honoring an interface buys.

Exercises

1. Bridge your own benchmark. Take an agent system you know (a coding agent on your repos, a browser bot, a SQL agent) and sketch its Bridge: what reset() constructs, what an Action tuple contains, what the runtime-safe env_state view may expose (remember: no handles, no sockets, plain data), and which persistence pattern applies. Check: if your env_state includes a live connection object, you have broken portability — component hooks must run identically against any runtime.

2. Attack the invariant. Try to design a wrapper — using only legal reset/step/observe calls and response rewriting — that makes the original verifier report success for an episode that did not genuinely accomplish the task. Check: you should fail, and the failure is instructive: the verifier reads final environment state, which only the native T can produce; a Contract can lie to the agent about consequences, but it cannot manufacture state the physics never reached. (The one loophole — replaying the entire solution via a Stage — produces a genuinely solved environment, which is a scaffold, not a cheat: the verifier is correct that the task is done.)

3. The interface archaeology. List which ActionableEnv methods exist only because wrapping was anticipated (versus what a plain benchmark runner would need). Check: observe() split from reset(), notify_replay_complete(), get_env_state() as a restricted view, and save/from_state with registry tags — four wrapping-shaped methods; a plain runner needs only reset, step, evaluate.

Chapter 2: Stage, Contract, Chain

The transformation w of Chapter 1 is a general interface: any mapping that follows it is a valid EnvHarness component. The paper ships three concrete types, chosen to cover three fundamental modes of customization — where an episode starts, how it plays, and how far it extends — and explicitly expects more to follow. This chapter builds all three on a single running example, the paper's own: the ALFWorld task "put a clean mug on the desk", whose default instance leaves the mug sitting in the open and ends the moment it is placed.

Hold that default in your head as the "before" picture. A capable agent solves it in a few steps: spot the mug, clean it, place it, done. Nothing about it exercises searching, memory, or persistence. Watch how each component turns this one frozen task into a different lesson.

Why these three, of all possible transformations? Because they partition the anatomy of an episode with nothing left over. Every episode has exactly three parts a wrapper can reach through the interface: how it begins (the initial state reset() hands over), how it proceeds (the step-by-step conversation of actions, transitions, observations), and how it ends (the termination boundary). Stage owns the beginning, Contract owns the middle, Chain owns the ending — moving it past where the base task would stop. That is why the paper can call them "three fundamental modes of environment customization" while simultaneously expecting more components to follow: future components will refine these territories (a stochasticity component is a Contract-flavored occupant of the middle), but the territories themselves are exhaustive.

Stage — changing where the episode starts

A Stage, w_stage,δ, is specified by a sequence of state-manipulation actions δ = (a₁, …, a_k), applied to the initial state that reset() produces:

Read the right-hand side carefully — it is the "replay, don't write" discipline from Chapter 1 rendered as mathematics. The new start state s′₀ is defined by folding the environment's own transition function T over the action list. Five of the six tuple slots are unchanged; only the starting point moves — and it moves along legal transitions, so it is reachable by construction.

The same mechanism customizes in both directions:

That dual use — obstacles for the comfortable, scaffolds for the struggling — is what will let Chapter 3's diagnosis choose a direction, not just a component.

Pause on how much rides on the humble format of δ: a list of action strings. It is trivially serializable (the saved form of a Stage is the list), trivially auditable (a human reads it in seconds), trivially portable (it names no internals, only public actions), and — because Chapter 3's rigger will generate these lists with an LLM — trivially generatable: writing a plausible action sequence is exactly the kind of thing a language model that plays these environments is already good at. A state-schema format would have scored worse on all four axes at once. The best interface decisions look inevitable in retrospect; this one is the paper's smallest and maybe its best.

Realization. In code the component is called Setups (the release predates the paper's naming), and its implementation is exactly the equation. On reset() it first resets the inner environment, then replays each action of δ through the ordinary inner.step() interface, and returns the post-replay observation as the episode's initial observation. The saved form of a Stage is nothing more than the action list itself — portable, human-readable, benchmark-vocabulary. Two refinements matter in practice. First, after replay it calls notify_replay_complete(), so the inner environment can rewind per-episode counters — step budgets, repetition guards — that must not be charged for the preparation phase (the agent should not start life four steps poorer because the harness moved a mug). Second, replay determinism is inherited from the seeded reset: the same Stage reproduces the exact same start state across rollouts, which Chapter 4's validation depends on.

The same task, before and after — two transcripts

Feel the Stage's effect at the level the agent feels it: the episode transcript. Left column, the default instance; right column, the staged one. Same house, same goal, same verifier — different lesson. (The transcripts are our rendering of the paper's described scenario in ALFWorld's actual command style; the δ and the design intent are the paper's own.)

The right-hand transcript is the product the whole system exists to manufacture: a trajectory in which the missing capability visibly happens, harvested by the same trusted judge. Skill extraction (Chapter 5) feeds on exactly these.

Contract — rewriting the rules of play

A Contract, w_contract,r, is specified by a triplet of transformation maps r = (f_A, f_T, f_O), each defaulting to the identity:

Three maps, three axes of the interaction:

The f_T axis deserves one more beat, because "attach structured feedback to specific outcomes" is the quietly powerful phrase in the paper's list. A static environment answers a bad action with whatever its authors wrote — often a shrug ("Nothing happens"). A Contract can answer it with a teaching signal: the fake githook that names the missing step, the simulated timeout that names the resource limit, the block reason that names the intended alternative. The environment's native physics stays; its pedagogy becomes programmable. Chapter 8's accepted components are a museum of this pattern — every fake error message they emit is carefully worded to make the escape route findable.

Notice which tuple slots a Contract may not touch: S, R, and s₀. Initial state belongs to Stage; terminal success remains the benchmark's own decision. The division is not bureaucratic tidiness — it is what keeps every component individually auditable. When Chapter 4's designer emits a Contract, a reviewer (human or automated) needs to check exactly three hooks against exactly three axes.

Realization. In code, a Contract is the class Rules, and the triplet becomes three pure-function hooks interposed on the step loop:

class Rules(EnvHarness):
    def filter_action(self, action, env_state):        # f_A
        # rewrite the action, or return Blocked(reason) before
        # it ever reaches the inner environment
        return action
    def modify_transition(self, action, response, env_state):  # f_T
        # rewrite the inner EnvResponse on its way out
        return response
    def filter_observation(self, obs, env_state):      # f_O
        # transform what the agent ultimately sees,
        # including the initial observation at reset
        return obs

All defaults are identities; a useful Rules component is a subclass overriding some hooks — and this subclass is exactly what the designer agent of Chapter 4 emits, as Python source. That choice cascades into three consequences worth savoring:

• The saved state of a Rules component is the source string itself. Loading recompiles it in a namespace exposing only the abstract data types — generated code physically cannot import a Bridge or grope for a Docker handle.
• The compiled code executes inside a per-episode subprocess, so faulty generated code crashes one episode, never the framework. LLM-written components are treated as what they are: untrusted input.
• A blocked action never strands the policy. By deliberate contract, a Blocked result leaves the environment untouched and returns the current state — re-observed, then f_O-filtered — alongside the block reason. The agent hits a wall, is told why, and can immediately try something else. A rejection is feedback, not a crash.

And here is what a Blocked result looks like on the wire — the "rejection is feedback" principle as an actual response object. The agent tried the teleport shortcut that a Contract's f_A has closed:

# agent action, against a Contract that removed teleport navigation
{ "name": "goto", "kwargs": { "target": "desk 1" } }

# the Blocked path: inner env NEVER called; state untouched;
# current state re-observed, then f_O-filtered, returned with the reason
{ "observation": { "text": "Blocked: high-level navigation is disabled.
      Move step by step. You are in the middle of the room…" },
  "reward": 0.0, "terminated": false, "truncated": false,
  "info": { "blocked": true } }

Three guarantees are visible in that one response: the world did not advance (no side effects from a rejected action), the agent still knows where it stands (the re-observed state rides along), and it knows why (the reason string). An agent facing this response can immediately try walking — which is the intended lesson — rather than crashing, stalling, or hallucinating about a world that silently ignored it.

Chain — extending the world past its ending

A Chain, w_chain,ℓ, is specified by a pair ℓ = (E_ext, g): an additional environment, and a composition logic that combines the original E and E_ext into one composite exposed through the same interface:

The new spaces are simply unions of the base environments' (A′ = A ∪ A_ext), and R′ acts as the new composite reward — unions because the composite agent must be able to do everything either world allows, a composite reward because neither world alone can judge the whole. On the mug task, one Chain appends "heat a potato and put it on the countertop" in the same house, returning success under R′ only when both environments are verified. The default instance ends the instant the mug is placed — so the mug environment, by construction, can never teach an agent to carry a goal past the point where it would otherwise have stopped. A Chain manufactures exactly that lesson, and Chapter 8 measures what it buys.

The composition logic g is unrestricted: environments can be concatenated, interleaved, or branched dynamically on intermediate outcomes — g can combine both from the start, or use the transition function to hand off once a condition fires. Realization (the class is called Link): the handoff is decided by a per-step hook, and the paper's appendix shows each mode in a few lines:

# Serial (the default, used throughout the paper's experiments):
link = Link(EnvA(), EnvB(), a_done_via="terminated")

# Branch on outcome: harder env if A was solved, remedial if not
class BranchOnOutcome(Link):
    def modify_transition(self, action, response, env_state):
        if not self._a_is_finished(response): return response
        solved = self.env_a.evaluate().success
        dest = AdvancedEnv() if solved else RemedialEnv()
        return self.switch_to(dest)

# Switch mid-task the moment a condition fires:
class SwitchOnAction(Link):
    def modify_transition(self, action, response, env_state):
        if action.name == "trigger_advanced_mode":
            return self.switch_to(AdvancedEnv())
        return response

# Interleave two environments, alternating every single step:
class Alternate(Link):
    def modify_transition(self, action, response, env_state):
        next_env = RedEnv() if isinstance(self.current_env, BlueEnv) else BlueEnv()
        return self.switch_to(next_env)

Every mode is one override of the same modify_transition hook, which intercepts the flow after each step to decide: stay, or route. Serial concatenation, outcome-conditioned branching, mid-task switching, per-step interleaving — a small algebra of episode shapes, each a few lines. (Note for Chapter 9: the paper uses only serial composition in its experiments, and its limitations section explains why the richer modes, though expressible, cannot yet carry a composite verifier.)

A Chain also answers a need neither sibling can touch, and naming it sharpens all three: Stage changes where you start, Contract changes what you may do, but only Chain changes how long the world demands you care. Goal persistence, budget allocation, context switching — the long-horizon virtues — are unteachable inside any single task, because they are properties of the space between tasks. Every benchmark that ends "immediately upon placement" has silently trained its agents to stop caring at the buzzer; Chain is the component that moves the buzzer.

Under the hood, Link earns its keep with four pieces of bookkeeping that only matter when environments are expensive — which, per Chapter 1's Bridge table, most of them are. It masks the sub-environments' termination signals, so only the composite decides when the episode ends (otherwise environment A's "terminated: true" would kill the agent loop at the handoff). It resets B lazily at the handoff point, so a container or browser is never started for a second leg the agent failed to reach. It caches each leg's outcome at its boundary, so evaluation never re-runs an expensive scorer. And the composite verdict is the conjunction R′ = R_A ∧ R_B — each factor decided by the corresponding sub-environment's own verifier, so the chained task inherits trusted verification from both of its parts. Because Link calls nothing beyond the ActionableEnv contract, any pair of registered environments can be linked — including cross-benchmark pairs. A single-task corpus becomes a long-horizon corpus by composition.

Composition — and why order is not a detail

Because all components share one interface, they compose freely. Stacking all three on the base mug task yields a single composite environment:

Count what one line of nesting achieved. In this setup w_stage,δ initializes the episode with the mug hidden in a drawer to enforce spatial search; w_contract,r truncates the observation to two sentences to demand memory under partial observability; and w_chain,ℓ appends the follow-up task to test goal persistence. One frozen benchmark instance is now simultaneously exercising search, memory, and persistence — three capabilities its authors never designed it to teach.

And the transformations are non-commutative: w₁ ∘ w₂ ≠ w₂ ∘ w₁. The nesting order determines how the environment is constructed and which constraints apply during initialization versus active interaction. Feel it on a concrete collision: put the Stage inside the Contract (as above), and the Stage's replayed actions run against the raw environment — the mug-hiding sequence executes freely. Flip the order, with the Contract inside, and the Stage's replay must itself pass through f_A — a Contract that blocks drawer interactions would now block the setup, not just the agent. A stack of wrappers is a program, and like any program, its statements have an order.

Realization — the decorator pattern, named. In code, each environment carries one EnvHarness: an ordered stack of components over its Bridge. EnvHarness is the abstract base every component derives from, and an EnvHarness is an ActionableEnv that wraps another ActionableEnv — the textbook decorator. Its default implementation delegates every interface method to the inner environment, so a concrete component overrides only the methods on the axes it affects. Rules(Setups(Toy24Bridge)) is again an ActionableEnv, and the policy interacting with the outermost layer cannot observe how many components sit beneath it. Persistence is layered accordingly: each component serializes only its own state, and a checkpoint records the environment plus an ordered list of components (innermost first), rebuilt inward-out on load. Any class honoring the contract is a valid component — the family is explicitly open to extension, and Chapter 9 lists the extensions the authors want next.

The combinatorics of one frozen task

Composition is not just elegance — it is where the environment supply comes from, so run the arithmetic once. Suppose the rigger has accepted, for one base task, three alternative Stages (mug in drawer 1, mug in the fridge, cleaning pre-done), two Contracts (truncated observations; teleport removed), and one Chain partner. Even restricting to stacks that use at most one of each type, in the canonical order, the distinct environments number:

— the +1 in each factor being "omit this component type." Allow reorderings (they differ, per non-commutativity) and multi-component stacks and the space multiplies further. This is the quiet answer to "but a benchmark has only 100 training tasks": under wrapping, 100 tasks is not a corpus size — it is a basis, and the harness spans the space around it. Chapter 7's scaling study will draw its 300 environments from exactly this kind of expansion, each one aimed rather than random.

The paper's future-directions list (Chapter 9) reads naturally as new factors for this product: components injecting stochasticity, imposing partial observability, exposing auxiliary feedback channels, or placing several agents in one shared environment — each a new axis, each multiplying the reachable space of every already-bridged benchmark, all through the same reset/step surface.

And one self-test to certify the vocabulary is installed: cover the table above and write, from memory, each component's equation-level signature (what parameterizes it, what it overrides, what it may not touch). If Stage's δ-through-T fold, Contract's identity-defaulting triplet, and Chain's conjunction verdict all came back, proceed; the next two chapters conjugate these verbs at speed.

Zoom out once before the recap. This chapter's deliverable is a vocabulary: three words that name what can be done to a frozen world without breaking its trust. Vocabularies are underrated research contributions — the Gym interface itself was "just" a vocabulary, and it reorganized a decade of RL. The test of a good one is whether new sentences come easily: and already you can say things like "a Stage that pre-fails the flaky path, inside a Contract that surfaces retry feedback, chained into a second incident" — a training scenario for an SRE agent, composed in nine words from parts this chapter built. The rest of the paper is machinery for writing such sentences automatically; the language they are written in is now yours.

Exercises

1. Write the δ. For the mug task, write the action sequence δ that would create each of these training situations: (a) the agent must handle an occupied desk (something already where the mug goes); (b) the cleaning step is pre-done but the mug is hidden; (c) the agent starts already holding a different object it must first put down. Check (a): something like take plate 1 from cabinet 2, put plate 1 on desk 1 — and confirm each of your sequences uses only actions from the environment's own vocabulary; if you wrote "place mug inside drawer" and the environment says "put mug 1 in/on drawer 1", your Stage would fail at replay.

2. Classify the intervention. For each of these, name the component and axis: (i) the agent's ls output is sorted to bury the relevant file last; (ii) submitting twice in a row is rejected with a cooldown message; (iii) the episode begins with the failing test already executed once, its output in the first observation; (iv) after the mug is placed, the agent must also water a plant. Check: (i) Contract f_O; (ii) Contract f_T (rewriting the response to an action) or f_A (blocking it) — either defensible, know why; (iii) Stage; (iv) Chain.

3. Order matters — prove it. Construct a concrete Stage + Contract pair where the two nesting orders produce different environments, and say which order the paper's Eq. (5) uses. Check: any Contract whose f_A would block an action appearing in δ works; Eq. (5) nests the Stage innermost — w_chain(w_contract(w_stage(E))) — so the replay runs against the raw environment and only the agent faces the Contract.

Chapter 3: Observe & Diagnose

The harness of Chapter 2 is a universal instrument, but an instrument does not play itself. Which component? On which task? Pointed at which flaw? Hiding the mug helps an agent that never searches; it wastes the time of one that searches fine but forgets second objects. The specific configuration must be tailored to each target policy and each task — and doing that by hand would just relocate the human-effort bottleneck that motivated the paper.

Note who plays the rigger: the same model backbone as the policy, per Chapter 5's fairness rule — but wearing a different prompt and role. There is no second, wiser model in this system; there is one model alternately playing student (rolling out) and course designer (diagnosing, writing), with the environment as the medium between its two roles. Keep that in mind through the mechanics: everything below is one model reading transcripts of itself.

Enter EnvRigger, the automation. Formally, it realizes the task-policy-conditioned map:

And note what "the current environment" means once the loop has history: the base environment plus every component already accepted for this task. Observation always happens against the newest world — so a flaw that an earlier component fixed no longer generates evidence, and the next diagnosis lands one layer deeper automatically. The rounds of Chapter 8's co-evolution story are this bookkeeping, iterated.

Read the signature like a type declaration, because it encodes the division of labor. The inputs are a base environment E, a base task t, and a policy π; the output is a component stack wrapping E. Each individual w_i is policy-agnostic (Chapter 1: a component transforms the environment alone), but the selection and parameterization of the stack is conditioned on both the task and the observed behavior of π. The semicolon in H(E, t; π) is doing real work: the policy is a conditioning input, never a modified one.

The paper's workflow figure draws the system as two coupled loops, and the picture is worth keeping. On the left, the execution loop: the policy runs against the current environment — the frozen base wrapped by the active EnvHarness containing every already-accepted component w₁…w_k. On the right, the EnvRigger loop: the resulting rollout trajectories feed the four-stage cycle, whose accepted output flows back as component w_k+1. Two loops, one shared artifact (trajectories flowing right, components flowing left), and a ratchet between them: each accepted component permanently changes what the execution loop produces, which changes what the rigger loop sees next. Chapter 7's co-evolution curve is this diagram, run for 300 environments.

EnvRigger runs as a loop with four named stages — Observe, Diagnose, Write, Validate — where the last two form an inner revision cycle. This chapter takes the first two: the evidence-gathering half. One prerequisite the paper states up front: for Stage mutations to be reliably reproduced, the base environment is assumed to support deterministic resets during validation — same seed, same world. Without that, a candidate's five fresh rollouts would each meet a different mutated start, and their aggregate statistics would measure reset noise instead of the component; determinism is what makes the validation comparisons mean anything. File it; it returns as a genuine limitation in Chapter 9.

Observe: what a trajectory actually is

The loop begins by running the policy π on the base task t in the current environment — current, meaning the base environment plus whatever components have already been accepted — and collecting a batch of rollout trajectories. Before anything can be diagnosed, be concrete about the raw material. A trajectory is the full transcript of one episode through the Chapter 1 interface: the alternating sequence of observations and actions, with the responses and the terminal verdict. Here is the shape of one, abridged from the kind of SWE-bench episode the paper mines:

// one rollout trajectory — the ONLY thing EnvRigger ever sees of the policy
{ "task": "django__django-12856", "seed": 3,
  "steps": [
    { "obs":    "ISSUE: UniqueConstraint should check field names...",
      "action": { "name": "bash", "kwargs": { "command": "ls django/db/models" } },
      "response": { "text": "base.py constraints.py ...", "exit_code": 0 } },
    { "action": { "name": "bash", "kwargs": { "command": "pytest tests/" } },
      "response": { "text": "... 4212 collected ... [timed out]", "exit_code": 124 } },
    { "action": { "name": "bash", "kwargs": { "command": "pytest tests/" } },
      "response": { "text": "... [timed out]", "exit_code": 124 } },   // again
    // ... 47 more steps ...
  ],
  "terminated": true, "steps_used": 50,
  "verdict": { "success": false } }   // the original verifier's word

Note the two bookends of the transcript, because they anchor everything: the seed at the top (Chapter 1's deterministic reset, making the rollout reproducible) and the verdict at the bottom (the original verifier's word, making the outcome trustworthy). Between two trusted endpoints stretches the behavior to be mined. Everything EnvRigger will ever conclude about this policy is latent in transcripts like that one. The repeated identical pytest command with the identical timeout is not noise — it is a behavioral signature, and spotting such signatures is exactly the Diagnose stage's job.

The paper is specific about why the batch must contain both outcomes: failures expose the specific weaknesses to be addressed within the task, while successes define the boundaries of those flaws — showing which capabilities are already intact and where they begin to fail. A failure-only view cannot distinguish "cannot do X at all" from "does X except under condition Y", and those two diagnoses call for entirely different environments: the first needs scaffolding, the second needs an environment where condition Y is guaranteed to occur.

See the boundary principle work on a concrete contrast. Two policies both score 2⁄5 on the mug task. Policy P's two successes both happened on rollouts where the mug spawned in plain sight; its failures all began with a fruitless "take mug" against an empty countertop. Policy Q's two successes include a rollout where it opened three drawers to find the mug; its failures all happened after finding the mug, at the cleaning step. Identical success rates — disjoint diagnoses. P cannot search (harden: hide the mug, or scaffold: reveal it); Q searches fine and fumbles the clean-precondition (a Contract enforcing hold-before-clean). Only the successes told them apart. A rigger that read failures alone would have prescribed P's medicine to Q — and validation would have caught the mismatch only after burning five rollouts on it. Reading successes is not thoroughness; it is aim.

What the rigger computes from a batch

From the K raw transcripts, the loop works with aggregate trajectory statistics — the same three families its later decisions will cite:

The distinction in the middle row is the diagnostic heart. Five failures that all show the same repeated pytest timeout are one flaw with five witnesses — write a component. Five failures with five unrelated causes are underdetermined — no single environment change addresses them, and a component written against one risks being validated against noise. Systemic beats anecdotal at every stage of this loop.

How much evidence? The arithmetic of K = 5

The paper's hyperparameter table fixes the batch size: K = 5 baseline rollouts per task (and, later, 5 fresh rollouts per validation). Work out what that resolution actually buys, because it disciplines everything downstream. With 5 rollouts, the observable success rates are exactly:

Six rungs. The rigger cannot tell a 45% task from a 55% task — and it does not need to. The decisions this evidence must support are coarse: is the policy helpless here (0), comfortable (1.0), or somewhere in between? Six rungs resolve that. What K = 5 cannot support is trusting any single trajectory — one lucky rollout moves the estimate by a full 20 points — which is why the designer's instructions (you will read them verbatim in Chapter 4) demand that every accept/reject decision reference aggregate statistics over the K runs, never a single trace. The choice of 5 is a budget decision wearing a statistics hat: each rollout is a real episode against a real container or browser, so K multiplies directly into the compute bill that Chapter 9's token table itemizes.

Diagnose: from transcripts to root causes

EnvRigger analyzes the collected trajectories to identify the root causes of the observed behaviors — root, as opposed to proximate. The proximate cause of the django failure is "the tests timed out"; the root cause is "the policy's test-invocation habit has no scope control, and no recovery behavior when an invocation fails." Components target roots: a wrapper that merely extended the timeout would treat the symptom and teach nothing. The paper names the kinds of systemic issues the diagnosis hunts for:

Note what all three have in common: they are procedural flaws, not knowledge gaps. The policy knows what a test suite is; it loops on one anyway. This is precisely the class of weakness a reshaped environment can fix — you cannot wrap an environment into teaching a model facts it lacks, but you can wrap one into breaking a habit, because habits live in the interaction, and the interaction is exactly what the harness owns.

Sharpen the "procedural, not knowledge" boundary with one example per domain, because the distinction decides what this entire method can and cannot fix:

The test in every row: can the corrected behavior be found by the policy itself, within the reshaped episode, using capabilities it already has? If yes, reshaping converts the flaw into a curriculum. If no, the validation stage will discover it empirically — the reshaped task pins at SR = 0, and the candidate is rejected as unsolvable. The loop does not need to understand the procedural/knowledge boundary in theory; its accept-gate enforces it in practice. That is a recurring pleasure of this design: philosophical distinctions outsourced to rollout statistics.

The fork: scaffold or harden

The diagnosis also determines the customization direction — and this fork is where the adaptive character of the whole system lives.

That second branch deserves a second read, because it inverts the usual instinct. A benchmark maintainer sees SR = 1.0 and celebrates; EnvRigger sees SR = 1.0 and concludes it has lost observability. A task the policy always solves has stopped producing information about the policy. Difficulty, in this system, is not a goal — it is the price of keeping the microscope in focus. (And Chapter 0's bimodal histogram tells you this branch fires constantly: for the measured policy, most ALFWorld tasks sat at SR = 1.0 or SR = 0.)

One property elevates the diagnosis above an LLM's opinion: it is falsifiable, and the loop falsifies it. A diagnosis implies a prediction — "block the shortcut and the failures will concentrate on the intended lesson; the policy will find the escape within the band." Chapter 4's validation stage tests exactly that prediction on fresh rollouts, and a diagnosis whose components keep failing validation is, operationally, a wrong diagnosis — the write-and-validate loop's rejections are the null results of a running science. This is the deep difference from prompting an LLM to "critique the agent's trajectory": critique costs nothing and risks nothing; a diagnosis here must survive contact with five more episodes of reality before anything downstream believes it.

The stage's output is deliberately humble in format: EnvRigger emits its findings as a textual diagnosis — natural language naming the flaw, the evidence, and the direction. Something with the shape of: "The policy resolves the issue but submits without running the failing test in 4 of 5 rollouts; successes occurred only when tests happened to be run for unrelated reasons. Direction: harden — make unverified submission fail." Text is the right interface here for the same reason it is everywhere else in 2026's agent stacks: the next stage's consumer is itself an LLM, and a diagnosis in prose keeps the pipeline inspectable by the humans supervising it.

A complete diagnosis, written out

Assemble the stage's product for the trajectory this chapter opened with. Five rollouts on the django task; 2⁄5 succeeded; the three failures all show the whole-suite pytest invocation timing out and being reissued verbatim. A faithful textual diagnosis reads:

## Diagnosis — django__django-12856, baseline K=5, SR 2/5
FLAW (systemic): repetitive action loop on test invocation.
  All 3 failures reissue `pytest tests/` after a timeout (exit 124),
  up to 9 times consecutively; no failure shows a different cause.
BOUNDARY (from the 2 successes): the policy CAN localize the bug and
  write a correct patch — both successes edited the right function.
  The flaw begins at test execution scope, not at code comprehension.
HEADROOM: successes used 31 and 36 of 50 steps — moderate headroom;
  a narrow perturbation is appropriate, not a sweeping one.
RELIANCE: the policy leans on bash/pytest; it never uses conda or
  direct script execution — perturbing those would be irrelevant.
DIRECTION: harden the loop surface. Make broad-scope test invocation
  a hard dead end with instructive feedback (Contract, f_T), so the
  escape — targeted, fail-fast test runs — becomes the only path.

Every line cites the statistics, not a vibe; the boundary line uses the successes; the headroom and reliance lines are the prompt's HOW and WHICH made concrete; and the direction line hands Chapter 4's Write stage a specification it can implement in one hook. When the paper says EnvRigger "outputs these findings as a textual diagnosis," this is the genre of document it means — and in Chapter 8 you will see the actual components this exact diagnosis family produced, verbatim from the appendix.

Reading the baseline like the designer does

The paper's appendix reproduces the system prompt that drives this analysis, and its instructions for reading the K unmutated baseline rollouts are a small masterclass in evidence discipline. The designer is told to read the baseline for exactly three things:

And then one more instruction that elevates the whole design: "Treat the baseline as raw data, not as a hint: decide direction and magnitude yourself." The prompt is guarding against an LLM failure mode its authors clearly met in development — a designer that pattern-matches on surface features of the transcript instead of reasoning from the statistics. The baseline is evidence to be weighed, not a template to be echoed.

What real diagnoses look like, domain by domain

Chapter 9 will tabulate nine weakness→component→skill cases in full; preview the weakness sentences here, because they are the concrete answer to "what does the Diagnose stage actually find?" Every one is procedural, behavioral, and legible to a non-expert:

Read the list twice and a pattern surfaces: every flaw is a missing verification loop of some kind — verify the container state before reaching, verify the full page before concluding, verify the fix before submitting. That is not a coincidence of these nine cases; it is what procedural incompetence in capable models mostly is in 2026 — models that know how to act but skip the checking steps that make action reliable. An adaptive environment is, above all, a machine for making skipped checks fatal.

A note on trust, since the diagnosis is itself LLM-written: nothing downstream believes it directly. The diagnosis proposes; the component operationalizes; the validation adjudicates. Even a hallucinated flaw costs at most one write–validate cycle before rollout statistics expose it — the same containment philosophy Chapter 4 applies to generated code, applied to generated beliefs.

End the chapter by naming what a diagnosis is, information-theoretically: compression with a purpose. Five trajectories of a 50-step SWE agent run to tens of thousands of tokens; the diagnosis compresses them to a paragraph — but not a summary paragraph. It is a causal compression: keep exactly the bits that predict which environment mutation will change future behavior, discard everything else. That is why the same trajectories would compress differently for a different downstream purpose (a capability report, a safety audit), and why the prompt labors so hard over what to extract (WHETHER, HOW, WHICH): the compression target defines the reader, and this reader is a component writer. Chapter 4 is that writer at work.

Exercises

1. Diagnose from symptoms. Five rollouts of a browser agent on "count this month's completed orders": two succeed; three fail by reporting a count equal to exactly one page of results. Write the four-line diagnosis (flaw / boundary / headroom-reliance / direction) and name the component the Write stage should try. Check: flaw = concluding without paginating or filtering; boundary = the successes show counting itself is intact; direction = harden; the paper's own WebArena case for this exact weakness staged the episode on the order grid with the filter bar in view — and a Contract variant could block answer-submission until a filter or scroll occurs.

2. The resolution limit. With K = 5, the rigger observes SR = 3⁄5 on two different tasks. On task A the three successes took 8, 9, and 8 steps; on task B they took 12, 31, and 49. What does K = 5 hide, and which prompt instruction recovers it? Check: the SR is identical but the headroom wildly differs — task B's solutions range from tight to profligate. The HOW instruction (read how it solves, not just whether) recovers it: perturbation magnitude keys off solution shape, not the success rate alone.

3. Black box, steelmanned. List two diagnostic signals you could get from logits/weights that trajectories cannot provide, and then the two costs that made the paper refuse them anyway. Check (one version): per-token uncertainty and internal-representation probes vs. (a) the loop stops working on API-only frontier models — killing the cross-model result of Chapter 7 — and (b) the machinery couples to model internals, breaking the same portability that keeps components benchmark-agnostic.

Chapter 4: Write & Validate

The diagnosis is on the table: "the policy submits patches without running the failing test." Now someone — something — has to turn that sentence into a working environment component, and then prove the component actually helps before it is allowed anywhere near training. This chapter is the second half of the EnvRigger loop: the Write stage that synthesizes candidates, and the Validate stage that accepts, rejects, or sends them back — a write-and-validate cycle with a hard budget of five rounds.

Write: the designer's two levers

The writer is itself an LLM — the paper calls it the designer agent, and crucially it runs on the same model backbone as the policy it is rigging against (a fairness control Chapter 5 returns to). Its system prompt, reproduced in the paper's appendix, hands it exactly two levers, and they map one-to-one onto Chapter 2's components:

The two levers compose freely — S₀-only, hooks-only, or both (the prompt's example: seed a state via in_env_actions, then block the easy escape via filter_action). And one lever is pointedly absent. The prompt states it in a sentence that should sound familiar by now: "The R axis is not exposed: success is the benchmark's own verdict, so reshaping reward cannot move the eval metric." The verifier-preservation invariant of Chapters 1 and 2 is not just an architectural principle — it is enforced at the prompt level, as a capability the designer simply does not have. Hooks may read the env_state schema provided each turn (the Bridge-published schema from Chapter 1) and may import only the standard library.

A candidate is a set of one or more components emitted together — the paper places no cap on the set size; the designer decides how many components a diagnosis calls for. A single flaw may genuinely need two: a Stage that constructs the incriminating situation plus a Contract that closes the escape route. And a candidate is accepted or rejected as a whole, never component by component — because its parts were designed to work together, and validating fragments of a mechanism tells you nothing about the mechanism.

For the record, the vocabulary translation between the paper and its code release — the release predates the paper's naming, and you will meet both in the appendix listings:

The shared system prompt is the same for every benchmark; each benchmark appends only a short block describing its Bridge's tools, its env_state fields, and its domain constraints. The shared part is organized around three shouted headings, which structure everything the designer does:

Read the three headings as a lifecycle and you will never forget them: BASELINE disciplines how evidence enters, PITFALL disciplines what may be proposed, REFINE disciplines how feedback is spent. Evidence in, proposals bounded, feedback conserved — a complete epistemic hygiene for a generative agent, in three shouted words.

One structural line completes the picture: "Your operating mechanism is fixed. The OBJECTIVE that tells you what to optimize is provided each turn." Mechanism and objective are decoupled — the same designer that autonomously hunts flaws can be handed an explicit target instead (a success-rate band, a step-count band, a one-sentence weakness), which is precisely the "environments on demand" mode Chapters 8 and 9 will demonstrate.

The worked example: from weakness to component to skill

The paper walks one case end to end, and it is the clearest window into what the Write stage actually produces. The input is a one-sentence weakness:

The designer's output — a Contract on the f_T axis, verbatim from the paper:

class _Contract(Contract):
    def modify_transition(self, action, response, env_state):
        cmd = bash_command(action)
        if "pytest" in cmd or "runtests.py" in cmd:
            env_state.extras["ran_tests"] = True
        if is_submission(cmd) \
                and not env_state.extras.get("ran_tests"):
            return failed(response,
                "githook: pre-commit hook 'verify-tests' "
                "failed. Run the test suite before submitting.")
        return response

Trace the data flow, because every design principle of the last three chapters is visible in eleven lines. The hook watches the action stream through the interface (never the repository); it accumulates a flag in env_state.extras — the runtime-safe scratch space; and when the forbidden pattern appears — submission without testing — it rewrites the response, faking a pre-commit githook rejection. The container is untouched. The verifier is untouched. But inside the reshaped world, the diagnosed flaw is now fatal: an unverified submission cannot end the episode successfully, and the only path forward is the missing habit.

The policy then plays this world; from the resulting trajectories, one skill is distilled (Chapter 5 covers the extraction pipeline). Also verbatim:

A second verbatim specimen shows the f_A lever working at its most surgical — the round-1 SWE-bench component that closed the policy's three sloppy edit paths, leaving patch as the intended escape. Note that each Blocked reason names both the ban and the sanctioned alternative:

# Round-1 accepted component (f_A axis): blocks the default editing paths
class _Rules(Rules):
    def filter_action(self, action, env_state):
        if action.name == "bash":
            cmd = action.kwargs.get("command", "")
            # 1. Inline python file editing via 'python -c' is off-limits.
            if "python" in cmd and "-c" in cmd:
                if "open" in cmd and ("write" in cmd or "replace" in cmd):
                    return Blocked("Inline python file editing (-c) is "
                        "restricted to prevent file corruption. Use "
                        "standard file utilities or patch/diff tools.")
            # 2. 'conda run' disabled; invoke env binaries by absolute path.
            if "conda run" in cmd:
                return Blocked("'conda run' is disabled... Invoke the "
                    "environment's binaries directly using absolute paths.")
            # 3. Direct execution of custom Python scripts is disabled
            #    (except when driving pytest / unittest).
            if re.search(r"\bpython[0-9.]*\s+[^-\s][^\s]*\.py\b", cmd):
                if "pytest" not in cmd and "unittest" not in cmd:
                    return Blocked("Direct execution of custom Python "
                        "script files is disabled... Use standard "
                        "command-line tools (patch, sed, redirected cat).")
        return action

Three narrow bans, each naming its escape route — and the skill that emerged ("use patch -p1 with a unified diff for precise modifications") is precisely the remaining path the component left open. This is what "the environment's pedagogy becomes programmable" (Chapter 2) looks like when a machine does the programming.

A two-lever candidate, verbatim

The verification Contract used one lever. Here is a candidate from the paper's WebArena cases that uses both at once — the diagnosis was "the policy guesses URLs instead of using the site search, landing on wrong or empty pages":

# Lever 2 — in_env_actions (Stage): start the episode ON the dashboard,
# so the site's own navigation is in view from step one
delta = ["goto('/admin/dashboard/')"]

# Lever 1 — rules_code (Contract, f_A): close the guessing escape
class _Rules(Rules):
    def filter_action(self, action, env_state):
        if "goto" in action_str(action):
            return Blocked("Direct navigation is disabled. "
                           "Use the site's search or navigation menu.")
        return action

Read the two levers as a pincer: the Stage provides the good path (the dashboard, with its search bar visible), the Contract removes the bad one (URL guessing). Neither alone suffices — block goto without staging the dashboard and the policy may simply be lost; stage the dashboard without blocking goto and the old habit still wins races. The skill this pair forced into existence: Search-First Navigation Protocol — prefer the site's internal search over direct URL manipulation. A single flaw, two components, emitted together, validated together, accepted together.

The pitfall the prompt shouts about: unsolvability

Give an LLM the assignment "make this environment harder" and it will, with enthusiasm, make it impossible. The designer prompt devotes its loudest paragraph to this failure mode, and the sentence at its center is the best one-line summary of the system's philosophy:

Connect this to Chapter 0's histogram and the symmetry becomes exact. The static benchmark failed by sitting at the extremes — SR = 0 walls and SR = 1 victory laps, 6% in the band. An unconstrained mutation engine fails the same way, just by manufacturing the extremes instead of inheriting them. The whole point of the loop is the band in the middle, and both cliffs are equally fatal to it.

Trusting untrusted code — the designer's failure modes, contained

The Write stage hands you a genuinely unusual artifact: LLM-generated code that will run inside your training infrastructure, thousands of times. The framework's containment story, assembled from Chapter 2's realization details, is worth seeing as a single defense-in-depth stack, because every layer answers a specific way the designer can fail:

Note the division of labor between the last two rows: mechanical failures are caught by sandboxing, semantic failures by empirical validation, and integrity failures by architecture. No layer relies on the LLM being good; every layer assumes it might not be. If you take one engineering pattern from this paper into unrelated work, take this one — it is the state of the art in letting a model program your system without trusting the model.

Validate: fresh rollouts, three verdicts

To evaluate a candidate, EnvRigger wraps the current environment with it, instantiating E′, and runs fresh rollouts of π on the base task — K = 5 of them, under the same settings as the baseline, so the success rates are directly comparable. Fresh is the operative word: the candidate was written to explain the old trajectories; it is judged only on trajectories it has never seen. Synthesis on the training evidence, evaluation on new evidence — the train/test discipline, applied to environment design itself.

From the fresh batch's trajectory metrics — success rate, failure distribution, timeout count — the rigger chooses among three validation behaviors. (Note the vocabulary discipline: the same three statistic families the Observe stage computed in Chapter 3. Observation and validation speak one measurement language, which is what lets validation results flow back into the Write stage as if they were fresh observations — because they are.)

Why must candidates be judged as wholes rather than component by component? Because components in a set are mechanistically coupled — the Search-First pair above is the clean example. Validate its Stage alone and the policy happily URL-guesses from the dashboard: looks non-challenging, gets rejected. Validate its Contract alone and the policy may flounder with no good path visible: looks unsolvable, gets rejected. Together they are a working curriculum. Per-component validation would discard exactly the multi-part designs that sophisticated flaws require; whole-candidate validation preserves them, at the price of coarser credit assignment when a set fails — a trade the revision loop absorbs, since the designer sees the full failure evidence when it rewrites.

The refinement guidance in the prompt is unusually crisp engineering advice, worth quoting because it generalizes far beyond this system: "Did this mutation move SR toward the target band? If yes, the perturbation TYPE is right — keep the working hooks verbatim and adjust only the magnitude (loosen if overshot, tighten if undershot); do not discard code that paid K rollouts of signal to establish. If no, start over with a different perturbation type." Each validation round costs five real episodes; the prompt treats those episodes as capital and forbids the designer from wasting them by rewriting from scratch what only needed tuning. Decisions must reference rollout statistics — "never a single trace."

The cycle repeats until a candidate is accepted or the revision budget of 5 write–validate rounds is exhausted — after which the instance yields no component at all. That quiet outcome matters: the system is allowed to conclude that a task offers nothing worth reshaping right now. Chapter 8 will show this actually happening at scale — by the third co-evolution round, about a third of training tasks sit at perfect baseline SR, every hardening candidate fails validation, and those tasks contribute nothing to the round's skill bank. A generator that must emit something for every input has no such honesty available to it.

The cost of one instance, by hand

The loop's structure lets you price a worst-case task before running anything — the first of several back-of-envelope computations this lesson will do with the paper's real constants. Per training task, with the Table-8 settings:

Five episodes to observe, then at most five validation rounds of five fresh episodes each. Thirty episodes of a 50-step SWE-bench agent is genuinely expensive — and this is the honest price of the system's central promise. Every rollout is executed against the real environment, graded by the real verifier; nothing is simulated. Chapter 9's token accounting shows where that lands in practice: on WebArena the total footprint (137.3M tokens) is essentially identical to the executing baseline VeriEnv's (137.8M), while the pure-simulation baseline GenEnv is 3.5× cheaper on ALFWorld — and buys hallucinated physics with the savings. The paper's phrase: the extra cost is "the cost of grounding." You will decide in Chapter 9 whether it is well spent; the results of Chapters 6–8 are the case for yes.

For the record, the loop's full hyperparameter card — identical on every benchmark, which is itself a claim (no per-domain tuning):

Five numbers and a rule. If you have ever shipped a system whose config file runs to hundreds of lines, the brevity is the message: the intelligence lives in the prompt and the loop structure, not in knob-tuning — and a five-line card is one you can actually hold constant across five benchmarks, which Chapter 5 showed is the difference between one confirmed mechanism and five tuned demos.

The loop, assembled

All four stages, as one algorithm — this is the complete EnvRigger, and everything in it has now been built from parts you understand:

def envrigger(E, task, policy):                  # H(E, t; pi)
    base = rollout(policy, E, task, k=5)          # OBSERVE
    diag = diagnose(base)                         # DIAGNOSE — text:
        # flaw + evidence + direction (scaffold | harden)
    for round in range(5):                        # revision budget
        cand = write(diag, feedback)              # WRITE: {rules_code,
                                                  #         in_env_actions}
        E2 = wrap(E, cand)                        # decorator stack
        fresh = rollout(policy, E2, task, k=5)    # VALIDATE — fresh!
        verdict = judge(fresh)                    # SR + failure dist +
                                                  # timeouts, never 1 trace
        if verdict == ACCEPT: return cand         # joins the EnvHarness
        if verdict == REJECT: feedback = start_over(fresh)
        else:                 feedback = rescale(fresh)   # keep hooks,
                                                         # tune magnitude
    return None          # budget exhausted: task yields no component

Step back and name what this loop is. It observes a system's behavior, hypothesizes an intervention, implements it, tests it against fresh data, and iterates under a budget — the experimental method, pointed at a policy's habits, with the environment as the apparatus. The paper's conclusion frames the contribution as reframing environment construction "as a wrapping problem rather than an authoring one." This chapter is where that reframe pays: authoring requires inventing tasks and judges from nothing; wrapping requires only reading behavior and adjusting a dial that already exists.

And one paragraph of appreciation for what the budget's existence implies about the authors' theory of LLMs. A revision budget of 5 says: we expect the designer to be wrong, often, and we have priced being wrong into the system. Accept-gating says: we do not trust the designer's claims about its own components. Fresh-rollout validation says: we do not even trust our own diagnosis until behavior confirms it. Every structural choice in this chapter treats the LLM as a fallible proposer inside a verifying loop — never as an oracle — and the reliability of the whole (Chapter 6's never-below-floor result) is manufactured from unreliable parts by exactly this arrangement. That is the general lesson of 2026-era agent engineering, and this loop is as clean a specimen of it as exists.

Exercises

1. Be the designer. Take Chapter 3's written diagnosis (the django whole-suite loop) and emit the candidate yourself: which lever(s), which hook, what feedback string. Then compare against the paper's actual round-1 component in Chapter 8 — a modify_transition that kills broad-scope test runs with a simulated 60-second-timeout message. Grade yourself on: did you choose f_T (fake the consequence) or f_A (block the action)? Both can work — but f_T lets the agent experience the timeout it was already causing, which keeps the mutation inside the distribution of things real environments do.

2. The band arithmetic. A candidate's five fresh rollouts come back 4⁄5 against a baseline of 5⁄5. Another comes back 3⁄5. A third, 0⁄5 with three timeouts. Assign each a verdict and, for the refine case, say what changes in the next write. Check: 4⁄5 — arguably non-challenging (barely bit); likely REFINE, tighten magnitude, keep hooks. 3⁄5 — in the useful middle; ACCEPT if the failure distribution concentrates on the target flaw. 0⁄5 + timeouts — unsolvable signature; REVERSE or loosen, do not stack bans.

3. Price a benchmark. Using the ≤30-episodes-per-task bound and the Chapter 5 split sizes, upper-bound the rigger's episode count for SWE-bench (100 training tasks) and ALFWorld (100 tasks). Check: 3,000 episodes each, worst case — then look at Chapter 9's measured ALFWorld figure (226.6M rollout tokens) and note that most tasks accept before exhausting the budget, which is why reality comes in under the bound.

Chapter 5: The Proving Grounds

A domain-agnostic claim has one honest test: many domains. The paper evaluates on five benchmarks spanning four distinct domains, chosen so that the runtimes underneath have almost nothing in common — a text-adventure engine, a live browser, and Docker containers full of repositories and spreadsheets. If the same wrapping code moves the needle on all of them, "domain-agnostic" stops being a slogan. This chapter sets up the arena: the benchmarks, the learning paradigm, the baselines, and — most importantly — the fairness controls that make Chapter 6's numbers mean something.

The five benchmarks

The heterogeneity is the experimental design. Text engine, browser, container; goal checks, site inspections, test suites, reference answers, cell comparisons — five benchmarks chosen to share as little as possible except the reset/step surface the harness needs. A method that secretly depended on anything beyond that surface would betray itself somewhere in this spread.

Note the deliberate spread of verifier styles — game-engine goals, site-state checks, hidden test suites, reference answers, cell comparisons. Every one of them is inherited untouched, which means the paper's improvements are graded by five different judges, none of whom it appointed. And note the extra metric on SWE-bench: average steps per episode, tracked as a measure of execution efficiency. It will produce one of the paper's most interesting results in Chapter 7.

Two of the five deserve a closer look, because their internal structure becomes load-bearing later. ALFWorld organizes its tasks into six types — clean, cool, heat, look_lamp (examine an object under a light), simple (pick and place), and two_obj (place two objects) — and ships its own seen/unseen evaluation splits. Chapter 9's leave-one-out study will hold these types out one at a time to test skill transfer. WebArena spans four sub-domains with genuinely different interaction textures: Reddit (forum posting and browsing), Shopping (a storefront), Shop Admin (a management dashboard dense with grids and filters), and GitLab (issues, repos, CI) — and the per-sub-domain columns in Chapter 6 will show the harness helping most exactly where dashboard discipline matters (Shop Admin, +6.2).

Splits — and the arithmetic of held-out honesty

Training and evaluation episodes are strictly disjoint on every benchmark. The splits, from the paper's appendix:

Notice also the split ratios: training sets are small everywhere — 100 tasks or fewer per benchmark against evaluation sets up to 4× larger (SWE) and multiples elsewhere. The method is being asked to generalize from little, on purpose: a technique whose value required thousands of training tasks would have nothing to say to the many domains where a hundred trusted tasks is all anyone has. Small-train/large-eval is the deployment-realistic regime, and the paper commits to it on every benchmark.

Two details reward a second look. The SWE-bench split trains on Lite and evaluates on the 407 Verified issues not in Lite — entirely different issues, so a skill must transfer beyond the training instances to score. And run the SpreadsheetBench arithmetic: 299 held-out tasks carrying 897 instances is 897 ÷ 299 = 3 instances per task — each spreadsheet manipulation is tested in three variants; Pass@1 aggregates over base tasks while Mean Score averages over all instances. That is also why ALFWorld reports "In-Dist" and "OOD": those are the benchmark's own seen/unseen splits — differing in whether the task's object–receptacle configuration appeared during training — not splits the authors constructed for themselves.

And because Chapter 6 will throw six different metrics at you, the decoder ring, once:

The fairness control that carries the paper

The full casting sheet, for reference as the chapters accumulate models:

Read the columns downward and the fairness rule is visible as a pattern: within every experiment, one model wears all the hats. The single exception — the RL column — still honors the spirit: the trained Qwen policy consumes environments, it does not design them, and both arms consume from the same designer's output conditions.

The learning paradigm: skills, not weights

The main experiments use skill-based learning (SL) — the training-free paradigm where the policy's weights never change, and improvement arrives through a growing library of retrievable skills. The pipeline, following ReasoningBank:

Since "skill" is the currency of every table to come, hold a complete specimen — the anatomy is always the same two fields, a description that gates retrieval and a content that steers behavior:

# one entry in a skill bank (from the paper's SWE-bench round 1)
{ "description": "When running a large or potentially hanging test
     suite, use the exit-on-first-failure flag to get immediate
     feedback and avoid environment timeouts.",
  "content": "When running tests that may hang or take too long, run
     pytest -x (or pytest --exitfirst) to stop execution instantly
     on the first failing test." }

Notice what a good skill is not: it is not a solution to any particular task, not a memorized repository detail, not a transcript. It is a conditional procedure — a trigger plus a move — which is why it can transfer to the 407 held-out issues in repositories the training set never touched. The paper states the quality bar explicitly in its on-demand section: a skill should combine "a general principle with actionable steps" rather than overfitting to a single task. And the quality of skills is exactly where environment quality enters: the same extractor, pointed at trajectories from a world that forced a behavior, distills that behavior; pointed at comfortable rollouts, it distills the comfort.

One evaluation detail that quietly raises the bar: each evaluation instance is attempted once. No best-of-k, no retry-until-pass. A skill either changes the single attempt or it does not — the strictest possible reading of "did the training help." Combined with three independent runs for the mean-and-variance reporting, the evaluation bill is itself nontrivial — three full passes over 407 SWE issues, 172 OfficeQA tasks, and 897 SpreadsheetBench instances per condition per method — which is worth remembering when Chapter 9 tallies the training-side token costs: careful evaluation is never free either, and this paper paid for it in every row.

And the consumption side, at test time — what "skill-equipped" concretely means when the held-out episode runs:

# one held-out evaluation episode (single attempt)
skills  = retrieve(bank, task_description)     # relevant subset only
prompt  = [ system_instructions,
            tool_schemas_from_bridge_registry, # Ch.1: introspected
            skills,                            # description + content
            first_observation ]
while not done:
    action   = policy(prompt + history)        # frozen weights
    response = env.step(action)                # ORIGINAL env — no harness
score = env.evaluate()                         # original verifier

Note the two absences in that loop: no harness (evaluation environments are unmodified) and no learning (weights frozen; the banks are the only thing training changed). Every number in Chapter 6 is this loop, run once per held-out instance, three independent times for the mean-and-std reporting. The entire causal chain from EnvRigger to the scoreboard passes through one variable: which sentences sit in the skills slot.

Why skills first, and RL only later (Chapter 8)? Because SL isolates the variable cleanly. Weights frozen, retrieval fixed, model fixed — the only thing that differs between conditions is which environments generated the trajectories the skills came from. Whatever separates the rows of Chapter 6's tables was carried entirely by the environments. It is the same experimental instinct you have seen in every good comparison paper: pin everything, vary one thing, attribute the delta.

One honest scoping note, flagged by the authors: the Chain component is excluded from this automated pipeline, because it is difficult for EnvRigger to observe the internal states of joined environments. Chaining is analyzed separately (with random pairing, no rigger) in Chapter 8. So the main tables measure Stage + Contract — the automated core — and the Chain numbers arrive as a labeled add-on.

Where does the extraction itself come from? The paper adopts ReasoningBank — its own team's prior work on scaling agent self-evolution through reasoning memory — as the fixed distillation machine. The relevant property for this paper is precisely that it is fixed: extraction reads trajectories and emits description-plus-content skills the same way for every condition, every baseline, every benchmark. The paper is not proposing a better extractor; it is proposing better trajectories to extract from — and holding the extractor constant is what makes that proposal testable. (One consequence worth noting from the appendix: the skill banks also contain skills distilled from rollouts no component specifically targeted; the appendix's listings highlight the component-driven ones so the causal chain from written component to induced skill stays inspectable.)

The opposition: four skill sources

EnvHarness environments are compared against four alternative sources of skills:

The pattern in the right-hand column is one of the paper's arguments, sitting quietly in a table layout: each generation baseline covers exactly one benchmark, because — as the appendix explains — generation pipelines must reach into an environment's internals and construct verifiers, which binds them to their benchmark. For the office-automation domain no generation baseline exists at all. In Chapter 6's tables those absences render as dashes, and the dashes are data: they are what domain-specificity looks like when you ask a pipeline to travel.

Know each specialist well enough to respect the comparison. GenEnv is generative co-evolution: an LLM environment-simulator produces new tasks and keeps their difficulty "at the edge of the agent's current ability" — so it shares this paper's difficulty-alignment instinct while differing on the decisive axis: its transitions and feedback are LLM-simulated, and (per the paper's Appendix B) that purchase of flexibility is paid for in hallucinated physics and evaluation drift. VeriEnv clones real websites into executable synthetic environments with programmatically checked rewards — real execution, trusted checks, but a pipeline that is a website-cloning system and travels nowhere else. SWE-smith synthesizes new repository-level task instances at scale — the strongest published scaling recipe for software agents, and the reigning champion the scaling study of Chapter 7 is built to challenge. Three serious systems, three different bets, all sharing one property EnvHarness refuses: each had to reach inside its benchmark to exist.

One phrase in the fairness paragraph deserves decoding: EnvRigger operates "under the same oracle verification access" as the baselines. During training, every method may consult the training tasks' verifiers — that is how EnvRigger's validation computes success rates, and how the baselines filter their generated instances. The oracle-access parity matters because validation is EnvHarness's engine: if it had privileged verifier access the baselines lacked, the comparison would be rigged in the rigger's favor. It does not; everyone reads the same referee, and only during training.

The comparison is budget-matched with care. Every baseline shares the same seed instances, the same environment count, the same skill-extraction pipeline, and the same policy model; each generation baseline is run with the same model as EnvHarness and produces the same number of environments. Differences in Chapter 6 therefore come from the generation strategy — not from the data, the model, or the amount of generation. EnvRigger operates strictly on training episodes under the same oracle verification access as everyone else.

The Chain exclusion deserves its mechanics, not just its mention. EnvRigger's loop needs to observe the environment it is customizing — baseline rollouts, env_state reads, validation statistics — and a chained environment is two environments wearing one interface: its internal state is split across sub-environments, with the handoff masking exactly the termination signals the rigger would read as episode structure. Rather than half-support this, the paper draws a clean line: the automated loop drives Stage and Contract; Chain runs in a separate, deliberately simple protocol — random pairing of base environments, no rigger — so its measured effect (Chapter 8) is attributable to chaining itself, uncontaminated by targeting. Method-design hygiene again: when a piece cannot be cleanly automated, measure it cleanly by hand instead of muddily by machine.

What "one implementation" means in practice

Chapter 1 promised that everything above the Bridge is shared. Here is that promise itemized against this experimental setup, because it is the operational meaning of the paper's generalization claim: the interface protocol, the EnvRigger loop, and the skill-extraction pipeline apply consistently across all five benchmarks, requiring only domain-specific prompt templates — each benchmark appends a short block to the shared designer prompt describing the tools its Bridge exposes, the fields of its env_state view, and its domain constraints. That is the entire per-domain cost: a prompt block, not a pipeline. Compare the ledger row for the baselines — an entire bespoke generation system per benchmark — and you have the engineering-economics argument in one contrast.

What does a "domain-specific prompt template" actually contain? The shape, reconstructed from the paper's description (each benchmark appends a block detailing its Bridge's tools, its env_state fields, and its constraints):

## Appended per-benchmark block — SWE-bench (illustrative of the
## structure the paper describes; the real blocks ship in the release)
TOOLS: bash(command: str) — stateless docker exec in the task repo.
ENV_STATE fields readable by hooks: repo_name, last_command,
  last_returncode, step_count, extras (dict, yours to use).
CONSTRAINTS: episodes cap at 50 steps; docker exec times out at 60s;
  the hidden test suite is the verifier — you cannot see or touch it.

That is the entire per-domain footprint of the method: a tool list, a state schema, a constraint list. Compare it to what each baseline had to build — a full generation pipeline reaching into repository internals, or a website-cloning system with programmatic reward construction — and the asymmetry in the engineering bill becomes the practical headline of this chapter.

The whole pipeline on one screen

What would falsify the thesis, and where to look

Before the results chapter, fix the predictions this setup can actually test — write your expectations down, because Chapter 6 will grade them:

• If reshaping is what matters (not just extra practice), EnvHarness must beat Original Envs on every benchmark — same tasks, same budget, only the wrapping differs. A single benchmark where original environments match it would break the universality claim.
• If diagnosis is what matters (not just generation volume), EnvHarness must beat GenEnv, VeriEnv, and SWE-smith on their home turf, despite being the only method that never touches environment internals.
• If the signal is real, gains must show up on held-out, unmodified tasks — especially ALFWorld's OOD split, where the object–receptacle configurations were never seen. A method that only polished in-distribution numbers would smell of memorized recipes.
• And a sharp null to watch: can practicing in an unmodified environment ever hurt? Chapter 0's redundant-skill logic predicts yes. SpreadsheetBench will answer.

One more absence to log as a control: nothing in the training pipeline sees the evaluation instances — not the rigger (training episodes only), not the extractor (training trajectories only), not the retrieval index (built from the banks alone). The only artifact that crosses from training to evaluation is the skill text itself, and its generality is enforced by the extraction format. The pipe between the two worlds is a sentence wide.

A last word on why this chapter matters as much as the results it sets up. Agent evaluation in 2026 is drowning in incomparable numbers — different models, different attempt policies, different splits, different budgets, quoted side by side as if they shared a ruler. This setup chapter is what a shared ruler looks like: every choice justified, every confound named and pinned, every baseline given the same resources as the method. When Chapter 6's deltas land, they will be small-looking numbers — +2.70, +3.27 — and they will mean more than most +10s you have seen, precisely because of what this chapter nailed down. Depth of control is what converts arithmetic into evidence.

Finally, tally what is shared against what varies per benchmark — the operational content of "one implementation, five benchmarks," in one ledger:

Left column: the method. Right column: an adapter and a paragraph. When Chapter 6's tables show five benchmarks moving together, this ledger is why that counts as one mechanism confirmed five times, rather than five bespoke successes stapled together.

Exercises

1. Confound hunt, agent edition. Suppose a rival paper reports: "our generated environments beat EnvHarness on SWE-bench, 53.1 vs 52.58" — using a different policy model, twice the environment count, and best-of-3 evaluation. List every control from this chapter that comparison violates, in order of severity. Check: policy model (capability confound), environment count (budget confound), attempts-per-instance (evaluation confound) — and if their skill extractor differs too, the skill-pipeline control. Any one suffices to void the comparison; this chapter's setup exists to make such a sentence impossible to write about its own tables.

2. Design the missing baseline. The office domain has no generation baseline. Sketch what a "SpreadsheetGen" would need: task synthesis, a verifier per task, and difficulty control — then estimate which part breaks first. Check: the verifier: cell-level correctness for a novel generated manipulation must itself be authored or generated, which is Chapter 0's Limitation 2 all over again. The dash in the table is not laziness — it is the difficulty of the verifier problem, visible as an absence.

3. Split forensics. Why train on SWE-bench Lite and evaluate on Verified-minus-Lite, rather than a random split of Verified? Check: Lite is the standard, cheaper training corpus (and the shared seed set every baseline here draws from, making Chapter 7's comparison meaningful), while evaluating on the disjoint 407 Verified issues guarantees no instance leakage and matches the community's canonical eval set — numbers comparable to every other paper's SWE-bench Verified column.

Chapter 6: The Scoreboard

Predictions on the record from Chapter 5; now the tables. This chapter works through the paper's two main results tables number by number — not to admire them, but to extract the three distinct findings buried in them: the consistency finding, the degradation finding (static practice can hurt), and the David-vs-Goliath finding (one generic interface beating every purpose-built pipeline at home). Every figure below is the paper's own; the deltas we compute by hand.

One definition before the first table, since two of its columns depend on it. ALFWorld's In-Dist and OOD columns are the benchmark's own seen and unseen evaluation splits: they differ in whether the task's object–receptacle configuration (which object goes where, in which room arrangement) appeared during training. OOD is therefore the transfer column — success there cannot ride on remembered configurations, only on carried behaviors — and it is where this paper's headline number lives. Keep asking, all chapter: is the gain bigger where memory helps, or where it can't? The answer discriminates between a method that teaches and a method that drills.

Table 1 of 2 — ALFWorld and WebArena

All numbers are means over three independent runs (the paper prints standard deviations as subscripts; the largest ones sit on WebArena's small sub-domains — e.g. ±8.4 on No-Skills GitLab — so treat single-column decimals there gently).

Warm up on the easiest cell first: the No-Skills ALFWorld average — (62.6 + 60.7) ⁄ 2 = 123.3 ⁄ 2 = 61.65, printed as 61.7. Rounding checks out; the table rounds half-up at the first decimal. Now the one that matters.

Hand-worked example 1 — recompute the headline. Verify the ALFWorld row yourself; it takes four subtractions and one average, and the paper's most-quoted number falls out. In-distribution: 66.2 − 63.3 = +2.9. Out-of-distribution: 70.4 − 61.4 = +9.0 — there it is, the abstract's "up to 9.0-point improvement on held-out instances." The average column: (66.2 + 70.4) ⁄ 2 = 136.6 ⁄ 2 = 68.3, against (63.3 + 61.4) ⁄ 2 = 62.4 for original environments, difference +5.9. And now read the shape, not just the sizes: the OOD gain is three times the in-distribution gain (+9.0 vs +2.9). Reshaped environments helped most exactly where memorized routines help least — on object–receptacle configurations the policy never saw. That is the signature you would predict if the harness teaches transferable behaviors (search before reaching, verify before ending) rather than task-shaped recipes; Chapter 9's leave-one-out study will test this signature directly.

On WebArena, every sub-domain improves over Original Envs (+1.9 to +6.2), with the average up +3.1. Note the two baseline curiosities the table hands you for free: Original-Env skills lose to no skills at all on the average (38.5 vs 38.7) — the first hint of the degradation finding — and VeriEnv, the purpose-built site-cloning pipeline, wins GitLab (38.9 vs 37.7) while giving back everything on Shopping (30.2 vs 35.2). Specialized generation can spike a sub-domain and still lose the war: EnvHarness takes the average, 41.6 to 39.6.

Hand-check the WebArena average while you are here — four sub-domains, equal weight: (40.6 + 37.4 + 50.8 + 37.7) ⁄ 4 = 166.5 ⁄ 4 = 41.625 ≈ 41.6. It reconciles. Getting into the habit of recomputing a paper's aggregate columns is cheap insurance: it catches transcription errors, reveals weighting choices (equal by sub-domain here, not by task count), and forces you to notice which cells drive the average — in this row, Shop Admin's +6.2 carries half the total gain. And one cell deserves its asterisk: GitLab is the single sub-domain where a baseline (VeriEnv, 38.9) tops EnvHarness (37.7) — the exception that keeps the "wins everywhere" claim honest at sub-domain grain while the averages stay unambiguous.

A note on reading the subscripts before we move on. Three runs per number means the std subscripts are estimated from just three samples — coarse, but honest. Use them comparatively: EnvHarness's ALFWorld OOD std (2.3) against Original's (4.3) says the reshaped-environment gains are not only larger but steadier; No-Skills GitLab at std 8.4 warns you that any single GitLab delta under ~5 points is weather, not climate. Papers that print subscripts are inviting you to do exactly this — accept the invitation.

Table 2 of 2 — SWE-bench Verified, OfficeQA, SpreadsheetBench

These are the rounded numbers behind the hero figure: SWE-bench 47.7 → 49.9 → 52.6, OfficeQA 54.2 → 54.4 → 56.2, SpreadsheetBench 46.4 → 45.9 → 49.1 for base agent → real-env skills → EnvHarness skills.

And a WebArena footnote that rewards the sub-domain reader: the four columns are not interchangeable rooms. Shop Admin — the dashboard maze of grids, filters, and pagination — is where the harness's signature moves (+6.2, the largest sub-domain gain), consistent with the Chapter 9 weakness cases that live exactly there: counting paginated rows by hand, guessing URLs, concluding above the fold. Reddit and GitLab, more navigational and stateful, move modestly (+1.9, +2.3); Shopping sits between (+2.2). Where the flaws are procedural and dashboard-shaped, the procedural-flaw machine collects its largest rent — a within-benchmark echo of the cross-benchmark pattern.

Before the findings, fix the units. "Resolved rate" on SWE-bench means the strictest thing it could: of the 407 held-out Verified issues, the fraction where the agent's patch made the hidden, human-verified test suite pass — on one attempt. At that scale each point is about four issues, so the +2.70 over Original Envs is roughly eleven additional real GitHub issues fixed, and the +4.91 over the bare agent about twenty — concrete work, not leaderboard dust. OfficeQA's 172 test tasks make each EM point about 1.7 questions; SpreadsheetBench's Pass@1 spans 299 tasks, so +3.27 is roughly ten more manipulations passed. Whenever a table's points feel abstract, convert them back into the tasks they denominate.

Finding 1 — consistent gains where static environments cannot

The universality prediction survives: skills from EnvHarness-customized environments beat skills from original environments on every benchmark — +5.9 ALFWorld average, +3.1 WebArena, +2.70 SWE-bench SR, +1.80 OfficeQA EM, +3.27 SpreadsheetBench Pass@1 — and beat the no-skill floor everywhere too. The paper credits a specific mechanism for that last, quieter fact: the write-and-validate loop only commits environment components verified by fresh policy trajectories, which is what keeps the pipeline from ever shipping skills that hurt. Remember that guarantee while reading Finding 2, because the baselines have no such filter.

Keep two protocol words distinct while reading, because they answer different skepticisms. Runs (three, independent) vary the stochastic elements of the whole pipeline and produce the means and stds — they answer "is this repeatable?" Attempts (one per evaluation instance, within every run) is the no-retry rule — it answers "is this real single-shot competence or best-of-k survivorship?" Three runs of one attempt each is the strict combination: repeatable single-shot performance. A table built on one run of best-of-five would print bigger numbers and mean less; know which regime any agent paper you read is in before comparing.

Also mark what comparison the office columns can and cannot host: with no generation baseline existing for OfficeQA or SpreadsheetBench, the only possible test there is reshaped-vs-static — which makes those two columns the purest measurement of the reshaping effect anywhere in the paper (no generator confound even in principle), and makes their positive deltas disproportionately informative despite their modest size.

Collect the full sweep in one place — every improvement-vs-Original delta across the five benchmarks, which is the sentence "consistent gains where static environments cannot" rendered as a strip of numbers:

Nine deltas, nine positive signs, four domains, five different judges — and remember what the row is not: it is not nine independent lucky draws, because every column shares one mechanism (diagnose, reshape, validate) and one set of controls (same model, seeds, budget, extractor). A method that only worked on code, or only on games, would show its domain signature here. This row's flatness of sign is the domain-agnosticism claim, empirically.

Finding 2 — static practice can make you worse

Now the null that Chapter 5 told you to watch. On SpreadsheetBench, skills extracted from the unmodified environments score 45.88 Pass@1 — that is 46.44 − 45.88 = 0.56 points below the no-skill baseline. And on SWE-bench, Original-Env skills lengthen execution: average steps rise from 53.58 to 55.01 (+1.43 steps, a 2.7% slowdown) for a modest +2.21 SR. Practicing in the static world made the agent slower there, and on spreadsheets made it outright worse.

The paper's explanation is Chapter 0's drought argument, now with receipts: static environments only allow the agent to practice behaviors it already executes, so they fail to address its specific limitations, "often retrieving redundant or suboptimal skills." A skill library distilled from comfortable rollouts is a mirror of existing habits — including the bad ones — and at retrieval time a redundant skill is not free: it occupies context, steers behavior toward the already-typical, and adds steps. Contrast the EnvHarness rows: same extractor, same retriever, same model — but the trajectories were harvested in worlds arranged so that only corrected behavior succeeded. The difference between a mirror and a curriculum is the whole 3.27 points.

Finding 3 — the generic interface beats the specialists at home

Each generation baseline was built for exactly one benchmark, with full license to reach into its internals. EnvHarness visits each as a stranger, through the public interface — and wins every match:

Stack the SWE gains cumulatively to see what the full pipeline delivers over doing nothing: bare agent 47.67 → with EnvHarness skills 52.58 is +4.91 points, decomposable as +2.21 from having skills at all (the Original-Envs step) plus +2.70 from those skills coming from reshaped worlds — while the steps column runs the same decomposition in reverse: +1.43 steps lost to static skills, then 5.40 recovered by reshaped ones, netting −3.97. More than half the accuracy value, and all of the efficiency value, lives in the second step — the one this paper added.

Hand-worked example 2 — the SWE-smith margin, both axes. Success: 52.58 − 50.12 = +2.46 points over the state-of-the-art purpose-built generator. Steps: 54.72 − 49.61 = 5.11 fewer executions per episode — and with 407 evaluation issues, that is 407 × 5.11 ≈ 2,080 agent-steps of compute the harness-trained agent simply does not spend, while resolving more issues. Now weigh what each method paid for its number: SWE-smith synthesizes entire repository-level task instances; EnvHarness wrote interface wrappers around 100 existing Lite tasks. The paper's summary sentence earns its italics: targeting diagnosed vulnerabilities through a unified interface "proves superior to merely scaling up the quantity of training episodes through domain-specific generation."

Notice also where the GenEnv margin concentrates: +8.5 of it is out-of-distribution. GenEnv generates difficulty-aligned tasks — but blind to this policy's particular flaws, so (in the paper's words) its instances "merely increase repetitive practice without addressing policy weaknesses." Difficulty alignment without diagnosis buys you more reps of the same lesson. Diagnosis is the ingredient, not generation.

Finding 4 — the floor that held

One result is easy to miss because it is an absence: nowhere in either table does the EnvHarness row fall below the No-Skills row. Across nine metrics and five benchmarks, the worst case for the harness is still a gain over the bare agent (+1.01 SpreadsheetBench Mean Score is the smallest). Given Finding 2 — that skill training can go negative, and did for the static control — this floor is not luck; it is the validation gate doing its job at scale. Every component in every EnvHarness environment passed five fresh rollouts before touching training; the redundant, the trivial, and the impossible were filtered at the source. Contrast GenEnv on ALFWorld: 63.3 / 61.9 / 62.6 against No-Skills' 62.6 / 60.7 / 61.7 — alive, but within a point of the floor everywhere. Generation without diagnosis hovers at the floor; static extraction can fall through it; the validated pipeline is the only row that never flirts with it.

And read the dashes one more time, now as a tally: of the fifteen cells the three specialist generators could in principle have filled across the five benchmarks, each fills exactly its own — three filled, twelve dashes. EnvHarness fills all fifteen with one implementation. The dashes are the domain-specificity tax, printed.

Finding 3½ — where the efficiency comes from

The step-count column deserves its own mechanism check, because "our agent is faster" claims usually dissolve on inspection. This one traces. EnvHarness skills cut SWE-bench episodes from 53.58 steps (no skills) to 49.61, while original-env skills pushed them up to 55.01. The paper connects the cut directly to Chapter 3's diagnoses: the efficiency gain "directly correlates with the specific diagnostics from EnvRigger — targeted Contracts and Stages designed to disrupt repetitive action loops and filter verbose observations successfully shorten execution trajectories." Recall the trajectory from Chapter 3, looping on a timing-out pytest for dozens of steps: a Contract made that loop a dead end during training, the escape the policy found ("target specific test files") became a skill, and at evaluation time the loop never starts. Fewer steps is what a repaired habit looks like in the metrics. Chapter 7 finishes this arithmetic and turns it into the abstract's 9.8%.

One row, end to end: OfficeQA

To be sure the table-reading muscles are built, walk the least dramatic row completely — the discipline shows best where the numbers are quiet. OfficeQA is question answering over office documents; its two metrics grade the same answers strictly (Exact Match) and leniently (F1, token overlap). No Skills: 54.23 EM, 55.77 F1. Original Envs: 54.40 EM — a statistically invisible +0.17 — and F1 identical at 55.77: practicing in the static environment changed essentially nothing about answer quality. EnvHarness: 56.20 EM, 57.73 F1 — +1.80 and +1.96 over the original-env row, and (compute it) +1.97 and +1.96 over no skills. The two metrics moving in lockstep (+1.80 ≈ +1.96) is itself information: the gains are whole correct answers, not partial-credit crumbs — if the harness had merely taught verbose answering, F1 would rise while EM stalled. Small numbers, clean story, and every sentence of it came from arithmetic you can redo in a margin.

Make the degradation mechanism tactile with one imagined-but-faithful retrieval moment. The agent opens a SpreadsheetBench task; the retriever surfaces the static-environment bank's nearest skill — say, "when editing spreadsheets, locate the target sheet before writing," distilled from rollouts where the policy already did exactly that. The skill occupies context, the agent dutifully re-confirms the sheet it would have found anyway, spends the steps, and reaches the hard part — the manipulation it actually fumbles — with nothing new to offer there. Multiply by a bank of such skills and you get 45.88: not sabotage, just guidance that arrives where guidance was never needed, at the cost of attention and steps. Now replay with the EnvHarness bank: the retrieved skill exists only because a reshaped world once made its absence fatal — it binds, by construction, to a place this policy actually breaks. Same slot in the prompt, opposite marginal value. That is the entire 3.27-point mechanism, one retrieval at a time.

It is also worth asking what this chapter would have looked like if the thesis were false, because the design gave falsity plenty of room to show. If reshaping added nothing, the EnvHarness and Original rows would interleave within noise across nine metrics — instead of nine same-signed deltas. If the gains were memorization, they would concentrate in-distribution — instead of tripling out-of-distribution. If they were skill-count artifacts, the budget controls would have equalized them. And if they were domain luck, at least one of five judges would have dissented. The table's shape, not just its magnitudes, is the argument.

Reading the table like a reviewer

Three fair skeptical pokes, and what the numbers say back.

"Couldn't the improvements come from the reshaped tasks just being more tasks?" They are not more tasks — that is the environment-count control — but there is a subtler version of the worry: reshaped variants of a task might function as free data augmentation, gains attributable to variety rather than targeting. The scaling study is the clean rebuttal available inside the paper: SWE-smith supplies maximal variety (novel repositories, not variants) under the same budget and finishes last, while EnvHarness's variants — minimal novelty, maximal aim — finish first and keep climbing. If variety were the active ingredient, that ordering would be reversed. Augmentation without aim is just Chapter 0's Limitation 3 wearing a new hat.

"The office gains are barely a point or two." True: +1.80 EM, +1.01 Mean Score — the smallest margins on the board. But context matters twice over. These are the domains where no generation baseline exists at all — the alternative to EnvHarness here is not a weaker competitor, it is nothing — and they are the domains where the static-practice control went negative. Moving SpreadsheetBench from "skills hurt" (−0.56) to "+2.71 over no skills" (49.15 vs 46.44) is a sign flip, not a nudge.

"Why is there no ablation of the Observe/Diagnose stages themselves?" There effectively is one, distributed: GenEnv and SWE-smith are the no-diagnosis condition (generation without observing the learner), and Original Envs is the no-generation condition (learner exposure without reshaping). The 2×2 is complete minus one cell — diagnosis without validation — and Chapter 4's unsolvability discussion explains why that cell would be dominated: unvalidated components have no floor. The baselines are the ablation, wearing other papers' names.

"Are these deltas real, given the run-to-run noise?" The reported standard deviations are the honest lens: ALFWorld's headline gains (+9.0 OOD, with EnvHarness at std 2.3 vs Original's 4.3) stand well clear of the noise; several WebArena sub-domain deltas (+1.9 Reddit against stds up to 9.7) individually do not, though the average trend across all four sub-domains, three runs each, points one way. The strong claims — ALFWorld, SWE-bench, SpreadsheetBench — are the well-separated ones, and the paper's aggregate framing ("consistent gains on every benchmark") rests on direction, not on any single fragile cell.

"Maybe the harness skills are just longer or more numerous." Chapter 5's controls preclude the numerous (same environment count, same extraction pipeline); the SWE step column precludes the longer — the harness-trained agent acts less, not more. And Chapter 7's cross-model table will add the sharpest version: on Gemini 3.5 Flash, EnvHarness skills win the success rate while running over five steps shorter than original-env skills.

If you keep a one-line verdict per benchmark, these are the fair ones. ALFWorld: the transfer showcase — +9.0 OOD is the paper's best number and its best-behaved (std 2.3). WebArena: consistent but noisy — trust the +3.1 average, hold sub-domain cells loosely. SWE-bench: the efficiency showcase — the only method that improves both axes at once, and the venue for every later scaling and cross-model result. OfficeQA: small, clean, uncontested terrain — the purest reshaped-vs-static reading. SpreadsheetBench: the cautionary tale — where static practice went negative and validation-gated reshaping flipped the sign.

Pull the chapter's threads into one sentence each, because the three findings are really one argument in three acts. Act one: with everything controlled, reshaped environments beat original ones everywhere — so the reshaping, not the practicing, carries the effect. Act two: the original environments sometimes carry a negative effect — so "just collect more trajectories" is not a safe default but a gamble whose downside this paper is the first to price on a public benchmark. Act three: purpose-built generators lose at home to a visitor that never touched their internals — so the binding constraint on environment value was never domain expertise; it was knowledge of the learner. Chapter 7 takes that last clause and turns it into a scaling law.

Exercises

1. Recompute every delta. Cover the improvement rows of both tables and regenerate them from the raw cells: five ALFWorld/WebArena deltas, six SWE/office deltas, plus the GenEnv OOD margin. Check the tricky one: GenEnv OOD margin = 70.4 − 61.9 = 8.5 — the paper's "+8.5 in out-of-distribution settings" is measured against GenEnv's OOD cell, not Original's (that one is the +9.0). Two different baselines, two different headline numbers, one table — a classic reading trap.

2. Build the degradation detector. Design the monitoring rule that would have caught the SpreadsheetBench regression in a production skill pipeline before shipping. Check: an ablation gate — evaluate skill-equipped vs skill-free on a held-out slice at every bank update, and block the deploy when the difference goes negative. The deeper lesson: the paper only sees this effect because it always carries the No-Skills row; pipelines that drop the unskilled control cannot detect that their skills hurt.

3. Steps as a metric, stress-tested. Using only this chapter's numbers, construct the argument that "average steps" alone would have ranked the methods wrongly — then say what fixes it. Check: by steps alone, No Skills (53.58) beats Original Envs (55.01) and SWE-smith (54.72) — ranking the weakest method above two stronger ones. The fix is Chapter 7's rule: always read steps jointly with success rate; a step saved by giving up early is not efficiency.

Finally, the checklist this chapter implicitly teaches for evaluating any skill/memory/environment paper (or your own pipeline), distilled from the way its tables were built and read: carry the no-skill floor always; demand a same-budget static-practice control before believing any reshaping/generation claim; check deltas against printed variance, not against zero; find the OOD or transfer column and compare its gain to the in-distribution one; read step counts jointly with success; and locate every dash — what a method cannot be run on is data about the method. Six habits, all cheap, all rare, all exercised on this one pair of tables.

Chapter 7: Scaling & Efficiency

Chapter 6 compared endpoints. This chapter compares trajectories: what happens as you keep feeding each method more environments, more models, more budget. It contains the paper's most strategically important figure — the scaling curve where every baseline flattens and one line keeps climbing — plus the completion of the step-efficiency arithmetic, and a four-model stress test of whether any of this depends on which LLM you happen to be.

The scaling experiment, precisely

First, why "number of environments" is the right x-axis at all — a framing worth two sentences because it was not always obvious. For pretraining, the scaling axis is tokens; for agents, the emerging consensus (and this paper's premise) is that the analogous resource is environments: distinct interactive situations the policy can practice in. Teams now budget environment acquisition the way they once budgeted data acquisition — which makes "how much capability does each new environment buy, under strategy X" exactly the question a scaling figure should answer, and exactly the one this figure does.

The setup deserves care, because "we scale better" claims live or die on their controls. Three allocation strategies are compared on SWE-bench Verified under an identical environment budget: EnvHarness environments, unmodified benchmark environments, and SWE-smith-generated environments. Policy model, budget, and skill-retrieval protocol are held fixed. Each batch of 50 environments yields one skill bank; banks alternate between 2 and 3 skills, totaling 15 skills at 300 environments. Do the bookkeeping yourself: 300 ⁄ 50 = 6 banks; alternating 3 + 2 + 3 + 2 + 3 + 2 = 15. The skill library stays tiny on purpose — scaling here means better environments, never longer prompts.

Operationally, the co-evolving arm runs as a loop over batches — worth spelling out because "co-evolution" often hides in papers as a vibe rather than a procedure:

Contrast the baseline arms at the same grain: their batch k is 50 environments drawn exactly as batch 1 was — from the fixed benchmark or the generator's fixed distribution — regardless of what the policy has learned. Same batch size, same bank cadence, same retrieval; only the arrow from policy to environments is missing. The scaling figure is that missing arrow, priced in resolved-rate points.

The curves

(Rounding note before the arithmetic: the figure's printed labels — 47.7, 54.8, 52.1, 50.4 — are the one-decimal faces of 47.67, 54.79, 52.13, and 50.37; we compute with the precise values throughout.)

Hand-worked example 3 — reading the scaling curve. All three start at the no-skill 47.67. EnvHarness reaches 54.79: a gain of 54.79 − 47.67 = +7.12 points. The same 300-environment budget spent on original environments buys 52.13 (+4.46) — so conditioning on the learner extracted 7.12 ⁄ 4.46 ≈ 1.6× more improvement per environment than real environments, and 7.12 ⁄ 2.70 ≈ 2.6× more than SOTA generation. And the gap at the boundary — 54.79 − 52.13 = 2.66 over original, 54.79 − 50.37 = 4.42 over generated — understates the story, because only one curve still has slope. The baselines have spent their budget; EnvHarness has spent its budget and kept its appetite.

Read the finished figure aloud once, as narration, because that is the form in which you will retell it: three curves leave 47.67 together; for the first hundred environments they climb almost as one; then the two unconditioned curves — real tasks and generated tasks — bend toward horizontal, the generated one lower, while the conditioned curve keeps its slope through the last budgeted batch, ending at 54.79 with daylight still above it. Every phrase of that sentence is a claim you can now defend with a mechanism: the joint climb (wide early boundary), the bend (fixed distributions versus a moving learner), the generated curve's deficit (distribution shift), the surviving slope (re-aimed batches). A figure you can narrate causally is a figure you have actually read.

Where do 300 EnvHarness environments come from, given only 100 SWE-Lite seed tasks? From Chapter 2's combinatorics, now load-bearing: one seed task wears different accepted component stacks in different batches — the drawer-hiding Stage in round one, the entrypoint-breaking Contract in round two, the PATH-pinning wrapper in round three — each stack a distinct training environment with distinct lessons, all sharing one verifier. Three hundred environments from one hundred seeds is an average of three incarnations per task, and the incarnations differ because the policy differed when each was written. The baselines' 300, by contrast, are 300 draws from fixed pools. Same count; different object.

Read the same table marginally — approximate per-50-environment increments off the plotted curves (endpoints exact, intermediate readings approximate), because marginal value is what a budget decision actually consumes:

The first row is the striking one: early, unconditioned real environments are nearly as good as targeted ones — when the policy is far from its ceiling, almost any practice lands near the wide boundary. The divergence is entirely a late-game phenomenon: by the final hundred, the baselines are buying a fifth of a point per 50 environments while the harness still buys ~0.75. Targeting is not uniformly valuable; it is valuable precisely in proportion to how much the learner has already learned — which is why the static-environment era felt fine while agents were weak, and why it is ending as they get strong.

Notice also the qualitative texture of the co-evolution evidence backing these curves: the paper's appendix prints representative skills from each round, and their progression (Chapter 8 reads it in full) is the curve's slope explained — round 1's skills fix invocation basics, round 3's fix interpreter resolution, and no round repeats a previous round's lessons because the previous round's environments already made those lessons stick. A flattening baseline has no such story to print: its round-6 skills would read like its round-1 skills, because its round-6 environments do. When a scaling curve and a skill-content progression agree, you are looking at a mechanism, not a fit.

A fair puzzle before moving on: if the end product is just 15 skills, why not skip the environments and prompt-engineer 15 skills directly? Because the skills are the residue, not the mechanism. Each bank's skills were selected by what actually changed fresh-rollout behavior in a validated environment — grounding that a hand-written "best practices" list lacks twice over: nothing verified that the advice binds this policy's real failure modes, and nothing verified the policy can execute the advice when retrieved. Chapter 6's degradation row is what ungrounded skill content does; the environments are the grounding apparatus.

Why the baselines flatten — the mechanism, not the vibe

You already own every piece of this explanation; assemble it. Chapter 0: learning signal lives at the competence boundary. Chapter 3: unconditioned environments sample tasks blind to where that boundary is. Early in scaling, blind sampling still works — the boundary is wide, and random tasks land on it often. But every bank of skills moves the boundary; the region of useful tasks shrinks and shifts; and a fixed distribution intersects a moving target less and less. The 200th real SWE task teaches mostly what the 50th already taught. The paper's verdict sentence: the gap "confirms that targeting the learner's current capability boundary is fundamentally more effective than unconditioned environment scaling." Diminishing returns are not a law of skill learning — they are a symptom of static task distributions, and conditioning is the cure.

A detail most readers skate past: the generated curve finishes below the real curve (50.37 vs 52.13) — the state-of-the-art synthesis pipeline scales worse than simply reusing the unmodified benchmark. Sit with that, because it quietly reorders the field's priorities. Synthetic task instances carry distribution shift: SWE-smith's generated repositories and issues resemble but do not equal the texture of real GitHub work, so skills distilled from them transfer to the real 407-issue evaluation with a discount. Real environments have no such discount — their only sin is staleness — and staleness costs less than shift, at least at this budget. The ranking (targeted-real > real > generated) suggests the field's energy on ever-better generation was aimed one abstraction too low: the scarce thing was never more tasks of either provenance, but a mechanism for pointing tasks at the learner. (Cross-check the numbers while here: the scaling figure's SWE-smith endpoint, 50.37, agrees with the main table's 50.12 within run noise — two experiments, one story.)

There is a subtle second reading of the same figure worth naming: it is a data-quality result disguised as a scaling result. SWE-smith produces genuinely novel repositories — maximum diversity — and finishes last. The original benchmark's 100 Lite tasks, reshaped, finish first. Novelty of tasks matters less than placement of tasks relative to this learner, this week. If you have a compute budget and a choice between "generate new worlds" and "aim the worlds you have," this figure is the strongest available evidence for aiming.

Before leaving the curves, note what they imply for the maintenance mode of a deployed agent, because that is where most readers will live. A production policy is retrained or re-skilled on a cadence; each cycle faces this figure's choice. The flattened baselines say: past the first cycle or two, re-mining the same static corpus is nearly pure cost. The rising curve says: the same corpus, re-aimed at the policy's current residual flaws, keeps paying. Environment budgets, like ad budgets, should follow the marginal curve — and now there is a measured one to point at.

Closing the efficiency arithmetic: the 9.8%

Hand-worked example 4 — where the abstract's number comes from. Chapter 6 left the step counts on the table: 53.58 (no skills), 55.01 (original-env skills), 49.61 (EnvHarness skills). The abstract claims "9.8% fewer execution steps." Derive it:

So the 9.8% is measured against the skill-equipped baseline — the agent with original-environment skills — which is the fair comparand (skills vs skills, only the source environment differing). Against the bare agent the saving is (53.58 − 49.61) ⁄ 53.58 = 3.97 ⁄ 53.58 ≈ 7.4%. Both real; know which one you are quoting. And notice the sign pattern that makes this remarkable: the ordinary trade is accuracy for steps — think longer, check more, score higher. EnvHarness pays negative steps for positive accuracy on the same benchmark, because what it removed was not deliberation but waste: the loops and re-reads its Contracts specifically made fatal during training.

Price the efficiency in wall-clock terms to feel it: on SWE-bench each step is a real docker exec against a repository — seconds of container time plus an LLM call. Across the 407-issue evaluation, 5.11 fewer steps per episode (vs SWE-smith training) is roughly two thousand avoided executions per full benchmark pass, compounding across every seed, every ablation, every rerun. Efficiency gains at the trajectory level are the rare kind that pay out during both training and deployment, forever after. (And note the lineage of the two scaling figures: the hero figure's right panel and the analysis section's Figure 5 are the same experiment at two zoom levels — if you cite the result, cite the 47.67 → 54.79 versions with the baselines named.)

Four backbones, one loop

Everything so far used two Gemini models. Would a different policy — open-weight, or far stronger — break the loop? The paper runs the full self-contained pipeline (policy = designer = extractor, same prompts, same acceptance criteria) on four backbones spanning open and proprietary, weak to frontier, on SWE-bench Verified:

Hand-worked example 4½ — both flavors of "gain," derived. The figure's percentages are relative gains over original-env skills. Verify Flash-Lite: (40.0 − 36.8) ⁄ 36.8 = 3.2 ⁄ 36.8 ≈ 0.087 = +8.7%. And Sonnet: (72.4 − 69.2) ⁄ 69.2 = 3.2 ⁄ 69.2 ≈ +4.6% — identical absolute gain (3.2), halved relative gain, because the base doubled. The absolute over-no-skills numbers quoted in the text derive the same way: Flash-Lite 40.0 − 30.7 = +9.3; Qwen 52.1 − 41.0 = +11.1 — versus under 5.5 for the two strongest models (Flash 52.6 − 47.7 = 4.9; Sonnet 72.4 − 67.2 = 5.2). Knowing which denominator a percentage rides on is half of reading an ML figure honestly.

Two reading notes on the figure itself. The model groups are ordered weakest to strongest by design, so the eye can track two gradients at once: the absolute bars rise left to right (capability), while the relative-gain percentages printed above them fall (+8.7% → +7.6% → +5.4% → +4.6%) — a shrinking ratio on a growing base, with the absolute gain nearly constant (3.2, 3.7, 2.7, 3.2). And the middle bars matter as much as the outer ones: on every model, original-env skills already beat no skills — skills as such help — but the EnvHarness bar clears the middle bar every time, which is this paper's specific claim isolated from the generic value of skill libraries.

The appendix pairs every one of those success rates with an average-episode-length column; the full grid, because its diagonal reading is the payoff of the next section:

EnvHarness skills beat original-environment skills on all four policies, by 2.7 to 3.7 absolute points, even though the skill-free success rates span 30.7 to 67.2. The paper's reading: the size of the gain is "largely independent of how strong the underlying policy is" — the customization loop neither breaks down on the weakest model nor saturates on the strongest. What the capability level changes is the content of the diagnoses, not the applicability of the loop: a Flash-Lite gets diagnosed for basics, a Sonnet for subtleties, and the machinery is indifferent. (Both kinds of skills help the weakest two models most in absolute terms over no skills at all — +9.3 and +11.1 for EnvHarness on Flash-Lite and Qwen, versus under 5.5 points for the two strongest — the familiar pattern that scaffolding matters most where competence is thinnest.)

The step counts tell a second story — three regimes

The appendix pairs every success rate above with average episode length, and the pairs refuse to fit one narrative — which is exactly what makes them worth reading:

Quantify the regimes before naming them — the percentages are as expressive as the labels. Qwen: 69.8 → 40.8 steps with EnvHarness skills, a (69.8 − 40.8) ⁄ 69.8 = 41.5% reduction (and 46.9% under original-env skills' 37.1 — either way, roughly half the episode was recoverable waste). Flash-Lite: 36.7 → 50.6, a 37.9% lengthening that buys +9.3 success points — persistence purchased with steps. Sonnet: 29.3 → 25.6, a 12.6% trim on an already-tight budget. Three models, three signs of the derivative, one method — because the method never optimizes steps; it repairs behaviors, and the step count lands wherever repaired behavior takes it.

And a texture note the paper adds about what differs across backbones: "what the policy's capability level appears to change is the content of the diagnoses rather than the applicability of the loop." Concretely: on the same benchmark, the rounds that taught Flash-class models basic test invocation and edit hygiene (Chapter 8's round 1) have nothing to teach a Sonnet-class model that already does both — its diagnoses land further down the stack, in the subtler territory rounds 2 and 3 mapped: broken-tooling resilience, interpreter resolution, navigation economy. Same microscope, same focus knob, different depth of field per specimen. The pipeline's indifference to which flaws it finds is exactly what backbone-independence means mechanically.

The paper distills the punchline into one sentence you should steal for every agent dashboard you ever build: "average steps alone is not a quality signal" — a short episode can be an efficient solution (Sonnet) or an early surrender (bare Flash-Lite), and the two sit at nearly the same step count (29.3 vs 36.7). Reading the metric requires the success rate next to it. Skills shorten episodes where the policy flails, lengthen them where it quits, and leave them alone where the policy already knows where it is going. Also settled here: the EnvHarness gains "do not come from simply running longer" — episode lengths of the two skill sources sit within a step of each other on Flash-Lite and Sonnet, EnvHarness is over five steps shorter on Flash, and only Qwen trades 3.7 extra steps for its 3.7 extra points.

The cross-model rows also retire a lazy objection preemptively: "of course a Gemini-designed environment helps a Gemini policy — same model family, shared blind spots." Qwen3.6 and Claude Sonnet 4.6 are neither Geminis nor each other's kin, each served as its own designer, and the margins held (+3.7 and +3.2). Whatever the loop exploits, it is not family resemblance; it is the universal fact that a model's rollouts expose that model's habits, whoever its parents were.

One more arithmetic lens on the scaling result: compression. Three hundred environments distill into 15 skills — a ratio of 20 environments per retained skill. Each skill therefore represents twenty validated worlds' worth of forced behavior, compressed into a description-plus-content pair a policy can carry in context. Under the original-environment arm, the same 300 → 15 compression runs over unaimed material — and its endpoint deficit (2.66 points) is the cost of compressing redundancy instead of correction. Skill-based learning lives or dies on what survives this 20:1 squeeze; aiming the environments is how you decide what survives.

The three-regime table also hands practitioners a free diagnostic they can run tonight, no harness required: plot your agent fleet's episode lengths jointly with success. Short-and-failing means quitting (Flash-Lite's 36.7-step surrender) — interventions should extend persistence. Long-and-failing means flailing (Qwen's 69.8) — interventions should inject procedure. Short-and-succeeding is grace (Sonnet's 29.3) — leave it alone. Three quadrants, three different medicines, and a monitoring dashboard that shows only the average step count collapses all three into one meaningless number.

And three things to watch in the follow-up literature, since this figure will be the one every successor paper re-plots: (1) longer budgets — does the harness curve eventually flatten, and at what multiple of the baselines' ceiling (the exercise-1 question, empirically settled); (2) seed-corpus refresh — combining targeted wrapping with modest amounts of generation, using each where it is strong (generation for new semantics, wrapping for placement); (3) cheaper riggers — whether a small designer model can rig for a large policy without breaking the same-backbone fairness in ways that matter for practice, where distillation is a feature rather than a confound.

Close the chapter by naming what its three studies share. The scaling study varies environment count; the efficiency analysis varies the metric; the cross-model study varies the backbone — and in all three, the only experimental asymmetry favoring EnvHarness is the same one: its environments are conditioned on the learner, and everyone else's are not. Three lenses, one asymmetry, three converging answers. That convergence — not any single number — is what makes the chapter's conclusion feel less like a benchmark win and more like a design principle settling into place: supply is not the bottleneck; placement is.

Exercises

1. Extrapolate responsibly. The EnvHarness curve is "still rising at 300." Sketch three continuations to 600 environments — continued linear, gentle saturation, hard plateau — and for each, name the mechanism from this lesson that would produce it. Check: continued rise = the boundary keeps relocating and the rigger keeps finding it; saturation = Chapter 8's round-3 effect grows (more tasks at un-hardenable perfect SR, more exhausted budgets); plateau = the base corpus's diversity is exhausted — wrapping can re-aim tasks but cannot mint new task semantics (Chapter 9's honest bound). All three are live; the figure alone cannot pick one.

2. The per-environment ledger. Compute improvement-per-100-environments for each source over the full run, then over just the last 100 (200 → 300, reading the endpoints and the paper's plotted shape). Check (full run): EnvHarness 7.12⁄3 ≈ 2.4 points; original 4.46⁄3 ≈ 1.5; generated 2.70⁄3 ≈ 0.9. The last-100 comparison is the sharper one: the baselines' final increments approach zero while the harness still banks measurable gain — marginal value, not average value, is what should drive a budget decision.

3. Design the fifth backbone. You get one more model row for the cross-model table. Which would you add to most stress the "loop is backbone-independent" claim, and what result would falsify it? Check (one good answer): a much smaller open model (3B class) as both policy and designer — the claim's weakest flank is whether a weak designer can still write valid, in-band components; falsification = candidates systematically failing validation or the gain collapsing toward zero, which the limitations section (Chapter 9) already half-predicts by noting weaker designers need more revision iterations.

Chapter 8: RL Co-evolution

Everything so far kept the policy's weights frozen — improvement flowed through retrieved skills. That was the clean experiment. But the sharper question was always waiting: if reshaped environments carry better learning signal, then training on them — actually updating weights by reinforcement learning — should produce stronger policies. This chapter is the paper's answer, in three movements: the RL experiment itself, the mechanism study showing the harness can steer environments to precise difficulty targets, and the longitudinal view — what the rigger writes round after round as the policy it shaped grows past each lesson.

The RL experiment

The setup, from the paper and its appendix. Policy: Qwen3-8B-base — an open-weight model whose parameters will genuinely move. Algorithm: Group Relative Policy Optimization (GRPO). Benchmarks: ALFWorld and WebShop (a shopping environment scored on both a graded score and strict success). Hardware and knobs, for the reproducers: one node of 8× NVIDIA H100s; rollouts served by vLLM; global batch 16; max 4096 prompt / 512 response tokens; history length 50; max episode length 50 steps; an invalid-action penalty of coefficient 0.1 to discourage unexecutable actions; sampling temperature 0.4; 150 epochs; seed 0.

The design is a clean A/B: two distinct policies are trained — one solely on the original static environments, one entirely on environments reshaped by EnvHarness — and both are evaluated on the same held-out instances (original, unreshaped, as always).

Scale the exposure first: 150 training steps at a global batch of 16 means roughly 150 × 16 = 2,400 episode samples flowed through each policy's gradient during training — every one of them, in the EnvHarness arm, drawn from reshaped worlds. The environments are not a garnish on this training run; they are the data distribution.

Hand-worked example 5 — the RL deltas. ALFWorld in-distribution: 87.9 − 81.4 = +6.5 — the introduction's "up to 6.5 points of improvement," recomputed. Average: 88.4 − 85.5 = +2.9. WebShop: score 79.2 − 75.6 = +3.6; success 67.4 − 66.0 = +1.4. Three of four metrics favor the harness-trained policy. The fourth is an honest give-back: ALFWorld OOD dips 89.6 → 88.8, a −0.8 the paper calls "slight, negligible" — and it is worth agreeing, with eyes open: 0.8 points against +6.5, +2.9, +3.6, +1.4 elsewhere. The conclusion the authors draw is the chapter's thesis: the reshaped environments "are not merely auxiliary data but provide a highly effective, independent optimization signal for online policy learning."

And weigh the WebShop pair as a ratio, because it carries a subtlety: the graded score moves +3.6 while strict success moves only +1.4. Reshaped training improved how well purchases matched their specifications considerably more than it flipped outright failures into successes — consistent with a curriculum that polishes execution quality on tasks near the boundary rather than conquering tasks far beyond it. Graded metrics see boundary progress before binary ones do; when both exist, read the graded one as the leading indicator.

Two proportions in the table deserve a beat of context. First, these RL policies start far stronger than Chapter 6's skill agents ever got — GRPO on the originals already reaches 85.5 ALFWorld average, versus the low 60s for the frozen-weights agents — so the harness's +2.9 average here is carved from a much thinner remainder; near ceilings, points cost more. Second, the choice of GRPO is not incidental to the story: it is the group-relative, value-network-free member of the policy-gradient family, which makes its learning signal unusually legible — nothing but outcome contrasts within sampled groups. A method whose entire gradient is "some rollouts of this task won and some lost" is the cleanest possible consumer for an environment supplier whose entire product is tasks where that condition holds. Which is the next section's subject.

Sit with the table's anchors for a moment, too: an 8B open-weight model, GRPO-trained on reshaped ALFWorld, reaches 87.9–88.8 — territory that Chapter 6's much larger frozen models with skills did not approach on this benchmark. Weight updates remain the stronger lever when you can pull it; the paper's contribution is that both levers — context and gradients — pull harder when the world supplying their experience has been aimed. Environment quality is upstream of the paradigm choice, which is exactly where a supply-side contribution should sit.

Why reshaped worlds are better RL signal — the GRPO lens

The paper reports the result; the mechanism becomes vivid if you recall how GRPO computes its gradient (our gloss, consistent with the paper's difficulty-calibration framing). GRPO samples a group of rollouts per task and reinforces each one by its advantage relative to the group. Now revisit Chapter 0's bimodal histogram: on a task the policy solves 10 times out of 10, every rollout matches the group mean — advantage ≈ 0 — and the gradient is silence. Same for 0 out of 10. The gradient lives where the group disagrees with itself — where some rollouts succeed and others fail — which is precisely the 40–60% band that static benchmarks leave 6% populated. An environment supplier that pushes tasks into that band is, almost literally, a gradient-signal amplifier. Which raises the question the paper answers next: can the harness push tasks into a prescribed band on demand?

You can make the band's optimality quantitative in one line (our gloss; the arithmetic is standard). A task with success probability p is a Bernoulli source; the variance of its outcome — the raw material of any group-relative advantage — is p(1 − p), maximized at p = 0.5 and falling to zero at both extremes. Evaluate it at the band edges and the cliffs:

The [0.4, 0.6] band captures ≥96% of the maximum outcome variance; the bimodal masses at 0 and 1 contribute exactly none. Moving the mean from 0.74 to 0.48 and the in-band mass from 6% to 80% is, in this arithmetic, a signal-power upgrade of the whole training distribution — which is the abstract's phrase "a superior optimization signal for reinforcement learning" given a formula.

Metric targeting — the dial, demonstrated

The appendix runs the EnvRigger loop on 100 ALFWorld tasks with an explicit, quantitative objective handed to the designer, measuring each task with K = 10 rollouts. Two targets are tested: per-task success rate steered into [0.4, 0.6], and average steps on successful episodes steered into [25, 35]. In both cases the harness reshapes each task in whichever direction its baseline requires — tightening tasks that sit above the band, scaffolding tasks below it.

The mechanics behind the table: the loop runs on 100 ALFWorld tasks with each task's baseline measured over K = 10 rollouts (double the standard 5 — band placement needs finer SR resolution than flaw detection), and the designer receives the quantitative objective in place of the autonomous diagnosis; its operating mechanism — write, wrap, validate on fresh rollouts — is unchanged, exactly as the prompt's "objective provided each turn" line promised in Chapter 4. Per task, the direction is chosen by where the baseline sits relative to the band: above it, tighten; below it, scaffold. Both directions run through the same two levers.

The SR row closes the loop this lesson opened in Chapter 0: the original tasks are strongly bimodal — most always solved or never solved — and the harness compresses them into the middle, raising in-band coverage from 6.0% to 80.0% and moving the mean per-task SR from 0.74 to 0.48. The AS target is honestly harder — it pins an exact step count rather than a rate — yet coverage still nearly triples, 18.0% to 53.0%. The paper's summary: a single interface suffices to calibrate an environment against an explicit, measurable objective, without any access to its internals. In plainer words: difficulty is now a dial, and Table 4 above is what turning it does to a training run.

One absence in the RL section is worth marking so you do not over-extrapolate: there is no SWE-bench RL experiment — container-heavy rollouts at GRPO's sampling appetite are a systems problem this paper does not take on — so the "better RL signal" claim is demonstrated on the two lightest runtimes (text engine, browser). The SL results carry the heavy-runtime evidence. A careful reader keeps the two evidence tracks attached to their benchmarks.

Two setup notes keep the RL comparison honest. WebShop, joining the roster here, is a shopping environment where the agent searches and purchases products against a natural-language specification; it grades both a graded score (how well the purchased item matches the spec) and a strict success rate — two thresholds on the same behavior, which is why Table 4 carries both. And the design trains two separate policies from the same base checkpoint rather than fine-tuning one policy twice: each arm sees only its own environment diet for all 150 epochs, so the comparison measures what a training diet does, uncontaminated by ordering effects. The single reward-shaping term — the 0.1 invalid-action penalty — applies identically to both arms; it addresses a base-model pathology (emitting unexecutable actions), not anything the harness introduces.

The RL experiment is also, read closely, a reuse demonstration. The environments feeding the harness arm were reshaped once and then consumed for 150 epochs of training — 2,400 episode samples against components whose design cost was paid up front. This is the amortization pattern Chapter 9's first limitation leans on: the expensive thing (the design loop) happens per environment; the cheap thing (an episode in a wrapped environment — a few pass-through function calls of overhead) happens per sample. A curriculum you can afford to build once and train against indefinitely is a different economic object from a curriculum you must regenerate per batch — and the token ledger's rollout-dominated totals are this asymmetry measured.

The Chain, finally measured

Chapter 5 promised the Chain component its own analysis; here it is. Real applications need agents that operate over extended horizons, so the Chain joins two randomly paired base environments into a single extended episode — deliberately independent of the autonomous rigger loop, to isolate the effect of chaining itself. Evaluation stays on standard single-environment test instances (SWE-bench Verified), so any gain is transfer from long-horizon training to ordinary tasks:

Hand-worked example 6 — what chaining buys, and what it costs. Chain-only skills slash steps from 53.58 to 41.96: a saving of 11.62 steps, or 11.62 ⁄ 53.58 ≈ 21.7% shorter episodes — by far the largest efficiency effect anywhere in the paper. The toll: Chain-only SR is 49.63, marginally below the 49.88 original-env baseline. The paper's reading of that trade is exact: Chain training's stringent condition — success requires solving both halves under a shared step budget — "prioritizes long-term goal preservation over short-task maximization." An agent drilled on two-task episodes learns to spend frugally and keep moving; pure single-task accuracy is not what it practiced. Combine both skill sets and you get the best of both worlds on the same table: the highest SR anywhere in the paper (54.30 — that is 54.30 − 47.67 = +6.63 over the bare agent) with excellent efficiency (43.12 steps). The paper's phrase: "highly complementary behaviors" — Stage/Contract fixes how the agent works; Chain fixes how long it can keep a goal alive.

And the Chain skills themselves are qualitatively unlike anything a single-task world can produce. The appendix shows two; read them as specimens of a new genre:

Budget management and context re-orientation only exist as behaviors when tasks are joined; a Chain is how a frozen single-task corpus is made to teach them — and both principles transfer straight back to single long tasks, which is presumably why Chain skills cut 11.62 steps off ordinary episodes.

Table 5 also quietly answers the practitioner's component-selection question — which wrapper family should I build first? — with a decision rule: if your agent's problem is quality (wrong solutions), Stage/Contract is the high-order bit (+2.70 SR standalone); if it is cost (bloated trajectories, per-step billing), Chain is (−11.62 steps standalone); and if you can afford both, they do not merely add — the combination's 54.30 exceeds either alone by more than their overlap would predict, because working accurately and working frugally are different skills practiced in different worlds. One scope note for honesty: the RL and Chain analyses run on ALFWorld, WebShop, and SWE-bench — the paper does not report RL on the office domain or WebArena, so extrapolations there ride on the SL evidence alone.

Three rounds of co-evolution, read like tree rings

The scaling curve of Chapter 7 said gains accumulate round over round; the appendix lets you watch what accumulates, by printing the skills each round's accepted components forced into existence. It is the most persuasive qualitative evidence in the paper, because the rounds form a visible curriculum no one hand-designed:

To make the rounds concrete rather than narrated, here is one accepted round-2 component verbatim from the appendix — the entrypoint breaker. Notice its surgical exemptions: it intercepts only "raw" pytest invocations, deliberately leaving the python -c workaround untouched, because a mutation that closes every path is the Chapter 4 pitfall:

# Round 2, accepted (f_T axis): breaks the pytest entrypoint,
# leaving programmatic invocation as the one escape.
class _Rules(Rules):
    def modify_transition(self, action, raw_response, env_state):
        if action.name != "bash": return raw_response
        cmd = action.kwargs.get("command", "")
        # Only intercept raw pytest; leave 'python -c' untouched.
        if "pytest" not in cmd or "python -c" in cmd or "patch" in cmd:
            return raw_response
        if re.search(r"\bpython(3)?\s+-m\s+pytest\b", cmd):
            return fake_response("No module named pytest\n", exit_code=1)
        if re.search(r"\bpytest\b", cmd):
            return fake_response("bash: pytest: command not found\n", exit_code=127)
        return raw_response

The skill it forced — "Use python -c "import pytest; pytest.main([…])" when the CLI is broken" — is a genuinely useful piece of field knowledge for real, misconfigured machines. That is the strange alchemy of this whole section: the environment lies (pytest is fine, actually), but the capability the lie forces into existence is true, transferable competence — graded, at evaluation time, on unmutated environments where it simply makes the agent more robust.

Price the round-3 abstentions while they are in view, because they are the system's honesty rendered as a compute bill. On the roughly one-third of tasks at perfect baseline SR, the rigger still pays: 5 baseline rollouts, then up to 5 rounds of 5 validation rollouts on candidates that keep failing (unsolvable or non-challenging), for a worst case of 30 episodes yielding nothing. A less disciplined system would lower its acceptance bar to salvage the spend — and ship components that produce SR = 0 walls or leave SR = 1 untouched, poisoning the training pool either way. This one eats the cost and returns the honest null. The scaling curve's late-stage slope is partly financed by exactly this refusal: what enters the pool stays worth practicing on.

Step back and name what you just read: fail-fast test habits, then resilience to broken tooling, then environment forensics — a syllabus ordered from procedural to systemic, generated by nothing but the loop's own dynamics. Each round's environment was manufactured from the previous round's surviving weaknesses. That is what "continuous, targeted co-evolution of the policy and its environment" means when you unroll it — and why the Chapter 7 curve had slope where the static curves had none: the static corpus teaches its one syllabus and stops; the harness writes a new one every round.

The metric-targeting result also reads as a product surface, not just an ablation. "Give me this benchmark at 40–60% difficulty for my model" and "give me episodes that run 25–35 steps" are the requests an RL-infrastructure team actually receives — from curriculum schedulers, from cost planners, from eval designers wanting discriminative test sets. Before this table, fulfilling them meant hand-selecting tasks and hoping; after it, they are a designer prompt with a band in it, at 80% and 53% fulfillment respectively. The difference between a research artifact and a tool is whether it takes requests; this one does.

Sanity-check the band-compression numbers against each other, because they must cohere and do: if 80% of tasks sit in [0.4, 0.6] after reshaping, the distribution's mean is pinned near the band's center — and the measured mean is 0.48, a hair under 0.5, consistent with the remaining 20% skewing slightly low (the near-band masses at 0.25–0.35 that over-tightened tasks would occupy). Before reshaping, a mean of 0.74 with 6% in-band requires heavy mass at 1.0 — and indeed "most tasks are either always solved or never solved," with the always-solved pile dominating for this competent policy. Two summary statistics, one histogram shape, no contradictions: the kind of two-line audit that catches broken tables in weaker papers, passed here.

Step back to the paradigm view before the rounds. Chapters 6 and 7 consumed the reshaped environments through skills — frozen weights, context-injected competence. This chapter consumed the same environments through gradients — weights genuinely moving. Same supply, two radically different consumers, gains in both: that is the strongest form of the paper's claim that the harness produces a better signal, not a better trick. A signal is consumer-independent; a trick usually is not. (And the two consumers compose in principle — an RL-trained policy can still retrieve skills — though the paper leaves that combination unexplored.)

Notice, finally, that this chapter answered the question Chapter 0 could only pose rhetorically. "It has nothing more to teach once the agent learns to solve the existing tasks" — the second fundamental limit of static worlds — is precisely what the round-3 abstentions, the band compression, and the co-evolution ratchet dissolve: when a task runs dry, the system detects it (perfect SR), attempts to refresh it (hardening candidates), and honestly reports when even that fails — at which point the rest of the corpus, re-aimed, still teaches. The world's supply of lessons is no longer a fixed quantity exhausted by learning; it is a function of the learner, re-evaluated every round.

If you run RL on agents anywhere, this chapter compresses to three operational takeaways:

And if a colleague asks for this chapter in one breath: the same reshaped worlds that improved a frozen model's skills also train stronger weights, the difficulty dial they turn is measurable and steerable to order, joining tasks teaches horizon-keeping that single tasks cannot, and the loop, run repeatedly, writes itself a curriculum — basics, breakage, forensics — and knows when it has run out of things to say.

Exercises

1. The GRPO silence, computed. A GRPO group of 8 rollouts on one task: (a) all 8 succeed; (b) 4 succeed; (c) 1 succeeds. With terminal reward 1⁄0, compute the group mean and each rollout's advantage sign pattern, and rank the three tasks by gradient information. Check: (a) mean 1.0, all advantages 0 — silence; (b) mean 0.5, four rollouts at +0.5 and four at −0.5 — maximal contrast; (c) mean 0.125, one at +0.875, seven at −0.125 — signal, but concentrated in a single trajectory. Ranking: b > c > a — and (b) is exactly the [0.4, 0.6] band the harness targets.

2. The dip, interrogated. Propose two mechanisms for the ALFWorld OOD give-back (89.6 → 88.8) under harness-trained RL, and the measurement that would separate them. Check (one version): (i) the reshaped in-distribution tasks absorbed optimization that the static arm implicitly spent on generalization-friendly behaviors; (ii) plain run-to-run variance at 3-run scale. Separator: more seeds, or an OOD-weighted reshaping run — and note the SL results point the other way (+9.0 OOD), so the dip is specific to this RL configuration, not to reshaping as such.

3. Round 4, predicted. Given rounds 1–3 and the third-of-tasks-at-ceiling effect, write the two-sentence forecast for what a round 4 would contain. Check: smaller gains, more exhausted budgets, and flaws one more level down — think repository-specific build systems, flaky-test discrimination, or cross-file reasoning — with a growing fraction of tasks contributing nothing until harder base tasks (or Chains) refresh the ceiling.

Chapter 9: Limits & Horizon

A method chapter tells you what a system does; a limits chapter tells you whether to trust it with your own problems. This one collects the paper's remaining evidence — the generalization study, the on-demand mode, the full cost accounting — then walks the confessed boundaries, places the work in its two research lineages, and closes with what it changes for anyone training agents.

First, settle the predictions this lesson placed on the record in Chapter 5 — a lesson that asks you to predict owes you the grading:

Do the skills transfer, or just fit? The leave-one-out test

Chapter 6 noticed the OOD gain (+9.0) tripling the in-distribution gain (+2.9) and predicted the harness teaches transferable behaviors. The appendix tests that prediction surgically: on ALFWorld, extract skills from environments covering every task type except one, then evaluate on the held-out type alone. Any gain must come from behaviors that carry across task types — familiarity with the held-out tasks is impossible by construction.

Four of six types improve, +3.1 on average, with a spectacular +16.4 on clean — and one genuine regression, −8.7 on heat. The paper's mechanism claim: reshaped environments "push the policy off its memorized routines during extraction, so the resulting skills encode behaviors that apply across task types instead of recipes tied to a single one." Read the regression honestly too — it is the cost side of the same coin: general behaviors displace specialized recipes, and on one type the displaced recipe was apparently the better tool. Cross-type transfer at +3.1 average with one −8.7 pocket is what "more general, not uniformly better" looks like in a table.

Environments on demand — the harness as a request API

In everything so far, EnvRigger picked its own targets from behavioral diagnosis. The same machinery accepts explicit, user-defined constraints — and this may be the mode practitioners use first. Two constraint classes are demonstrated: the quantitative targets you saw in Chapter 8 (steer SR into a band, steer steps into a band), and capability weaknesses described in a single sentence of natural language. The appendix runs nine of the latter across three benchmarks. The pattern, three examples deep:

The full nine-case study, for the record — three benchmarks, one sentence in, one component and one skill out, every time:

Read the Stage rows for their economy — the "inefficient search order" case is answered by three pre-opened drawers, a δ of six actions that stages an ordering directly in the initial observation. And the last row for its audacity: the component makes the feared consequence real (sed genuinely corrupts the files, silently), so the trajectory contains the disaster, the recovery, and hence the lesson. The designer chose its component axes without instruction — staging start states, blocking shortcuts, faking consequences — and every component "stays inside the source distribution and leaves goals and scorers untouched." Notice what this mode amounts to: a natural-language compiler from observed bad habit to training environment that punishes it. Every agent team maintains a mental list of their system's recurring sins; this turns each list item into a curriculum entry for the price of a sentence.

What it costs — the full token ledger

Chapter 4 priced one task at ≤30 rollouts; here is the measured bill for whole benchmarks, decomposed into design tokens (proposing and refining components) and rollout tokens (every interaction a method drives):

Hand-worked example 7 — reading the ledger. First, the design share: 1.46M ⁄ 228.0M ≈ 0.64% of the ALFWorld budget goes to designing components — enormous relative to GenEnv's 38K (roughly 38× more), yet a rounding error against rollouts, which dominate either way. Feeding full trajectories into diagnostic prompts is deliberate and, in context, nearly free. Second, the like-for-like comparison: against VeriEnv — which also executes in the real environment — the totals are essentially identical, 137.3M vs 137.8M. The Chapter 6 gains over VeriEnv were not bought with extra compute. Third, the asterisked bargain: GenEnv's total is 3.5× lower — but its rollouts are simulated by an LLM rather than executed, and that saving purchases exactly the hallucinated transitions and drifting success signals of Chapter 1's comparison table. The paper's framing is the right one: the extra cost is the cost of grounding — at equal grounding the footprints match, and where EnvHarness spends more, it spends on real execution against a trusted verifier.

A practical note on limit 2 before it is stated, because teams hit it first: "resettable" does not require the domain to be resettable — only the training instance of it. The paper's own heavy benchmarks show the pattern: SWE-bench wraps live-looking repositories by containerizing them; WebArena wraps the live web by self-hosting site replicas. The fintech agent of the quiz below cannot be harnessed on production, but a sandboxed replica of the account system — resettable by construction — can be bridged, harnessed, and mined for skills that deploy back to production. The limitation is real (someone must build the sandbox, and sandbox fidelity becomes the new risk surface), but it is a constraint on where training happens, not on which domains can benefit.

The three confessed limits

Note how the three limits share a root: each is the shadow of a design guarantee. Grounded validation costs rollouts (limit 1) because nothing is simulated. Stage and Chain need resets (limit 2) because states are only ever reached through real transitions. Chain is serial (limit 3) because verdicts are only ever inherited, never authored. You cannot remove a limit without spending the guarantee that produced it — the honest trade at the heart of the paper.

What the paper does not claim — scope, stated plainly

Reviewing a method honestly includes listing the claims it never makes, because secondhand summaries will make them on its behalf. From the text itself:

• Not that wrapping replaces environment building. The harness presupposes a trusted base environment; someone still hand-builds worlds and verifiers. The claim is about extracting far more learning from each one that exists.
• Not that skills or reshaped RL fix knowledge gaps. Every measured gain is behavioral — verification habits, search strategies, navigation discipline. A model that cannot do the task's substance will meet SR = 0 walls no Stage can scaffold past the substance itself.
• Not that the gains are uniform. The paper's own tables carry a −8.7 (heat, leave-one-out), a −0.8 (RL OOD), a sub-domain loss to VeriEnv on GitLab, and a Chain-only SR toll — consistent direction across benchmarks, not universal dominance across cells.
• Not that three components are the final vocabulary. Stage, Contract, Chain are declared "a first set," with stochasticity, partial observability, auxiliary feedback, and multi-agent components named as the expected sequels.
• Not that this works beyond text yet. Visual, GUI-driven, and embodied observation spaces are explicitly future work — the wrapping abstraction's survival there is an open question the authors pose, not answer.

What comes next, per the authors

New components. Stage, Contract, and Chain are "a first set, not a closed one." The agent harness grew well beyond its initial pieces, and the authors expect the same here: components that inject stochasticity or partial observability, that expose auxiliary feedback channels, or that place several agents in a shared environment — each extending what a frozen benchmark can be reshaped into, all through the same reset/step interface. Chapter 2's open decorator family was built for exactly this: any class honoring the contract is a valid component.

The open decorator family makes the "new components" direction concrete rather than aspirational — a stochasticity component, for instance, is a few lines against the existing contract (illustrative sketch, in the spirit of the paper's Appendix D examples):

# A future component type, sketched: intermittent tool failure.
# Same contract, same discipline — a valid EnvHarness member today.
class FlakyTools(EnvHarness):
    def step(self, action):
        if self.rng.random() < self.failure_rate:      # seeded — reproducible
            return transient_error_response(action)     # world untouched
        return self.inner.step(action)

Every rule from Chapters 1–2 still holds — the inner world untouched, the verifier inherited, the seed making validation reproducible — and the reshaped environment now teaches retry discipline, a capability no current benchmark exercises on purpose. The multi-agent and auxiliary-feedback components the authors name would follow the same pattern: new classes honoring the old contract.

Beyond text. EnvHarness currently operates over textual actions and observations. Extending to visual, GUI-driven, or embodied environments "would test whether the wrapping abstraction survives when observations are no longer symbolic" — and would require components that can specify and verify states not expressible as text. An honest open question, honestly stated.

Where this sits in the literature — two lineages, one intersection

Lineage 1: environment scaling and adaptation. One branch supplies agents with more environments — LLM-simulated feedback, world models synthesizing whole families of agentic environments, programmatic synthesis, new instances inside existing benchmarks (SWE-smith). Another branch adapts what the environment presents — the unsupervised-environment-design tradition (emergent complexity via minimax regret, prioritized level replay), POET's endless generation of environment–solution pairs, hand-designed corrective feedback and reward shaping. EnvHarness's position against both: it reshapes an existing environment through one interface shared across benchmarks, conditions the reshaping on the diagnosed weaknesses of the current policy, and leaves tasks and verifiers untouched. The UED dream, ported from gridworlds the researcher owns to benchmarks the community trusts.

Lineage 2: self-evolving agents. Systems that improve from their own experience by evolving prompts and reflections, skill and workflow libraries, experience memory (ReasoningBank — this paper's own extraction pipeline), model weights via self-generated rewards, and recently the agent harness itself. Every one of them evolves the agent while the world stays fixed. EnvHarness is the missing mirror image: it evolves the world against the diagnosed weaknesses of a frozen policy — and Chapter 8 showed the two directions compound when alternated.

For readers who want the family tree with names on it, the specific systems the paper defines itself against, and where each sits:

What it means for practice

The symmetry that opened this lesson also closes it, one turn further along. Agent harnesses began as engineering conveniences and became the substrate of an industry — to the point where recent work optimizes the harness itself end-to-end around a frozen model. If the analogy keeps holding, environment harnesses walk the same road: first a convenience (this paper), then a standard layer every training stack assumes, then itself a target of optimization — riggers rigging riggers, curricula tuned by outer loops. The paper's own framing invites exactly this reading: it calls its contribution "a practical pathway toward scalable environment supply," a phrase about infrastructure, not about one method's scores.

And a warning to carry out of Chapter 6: if your skill/memory pipeline mines trajectories from production — the Original-Envs recipe — you now know it has no floor. Mine the boundary, not the comfort zone; if nothing pushes your agent to its boundary, the first thing to build is the pusher.

If you want the playbook as a checklist, the minimal adoption path this paper implies:

Map of the paper, map of the lesson

For readers heading to the source (41 pages, and the appendices are half the value), the crosswalk:

Every load-bearing number, one table

The lesson quoted several dozen figures; these are the ones arguments were built on, gathered for your future self:

Exercises

1. The numbers, forever. Without looking: the five-benchmark headline trio for SWE-bench (base → real-env skills → EnvHarness), the best held-out gain and where it occurs, the step-efficiency percentage and its exact derivation, the scaling endpoints at 300 environments, and the band-compression pair. Check: 47.7 → 49.9 → 52.6; +9.0 on ALFWorld OOD; 9.8% = (55.01−49.61)⁄55.01; 54.79 vs 52.13 vs 50.37; 6% → 80% in [0.4, 0.6]. If you can also say why each number follows from the loop's design, the lesson is finished with you.

2. Rig your own agent. Pick an agent you actually run (a coding assistant, a browser agent, a data pipeline bot). Write down its three most persistent bad habits, then design one component per habit: choose the axis (δ, f_A, f_T, f_O), sketch the hook in ten lines, and specify the validation band you would accept. Check yourself: if any of your components changes what counts as success, you have violated the R-axis rule — redesign it as a Stage or Contract that makes the habit fail inside the original rules.

3. The falsification hunt. The heat regression (−8.7) is the paper's largest unexplained number. Design the follow-up experiment that would distinguish the two hypotheses: (a) general skills displaced a load-bearing type-specific recipe; (b) the reshaped extraction environments happened to under-cover heat-adjacent behaviors. One workable design: extract with reshaped environments restricted to heat-similar types only, evaluate on heat — hypothesis (a) predicts the regression persists; (b) predicts it closes.

4. Argue the other side. Write the strongest good-faith case that from-scratch environment generation (the Agent-World direction) beats wrapping in the long run. Fair ingredients: wrapping is bounded by the diversity of its base corpus; a wrapper cannot create genuinely new task semantics, only rearrange difficulty around existing ones; generation costs are one-time and falling; and the verifier-trust gap may close as verification-generation improves. Then note what the wrapper's side answers with Chapter 7's figure — and where that figure's 300-environment budget stops being evidence.

The closing thought

Strip away the benchmarks and this paper is about a relationship: a learner and its world, and what happens when only one of them can change. For three years the field poured its ingenuity into the agent's side of the loop — better harnesses, better memory, better skills — while the worlds stayed exactly as their authors left them, frozen mid-gesture, teaching the same lesson to every visitor forever. The quiet observation at the center of this work is that a world's interface is enough to change the world: start it elsewhere, filter its conversation, extend its ending, and a benchmark built to measure becomes an environment built to teach — without its judge ever noticing the difference. The next time a system you are training plateaus, ask the question this paper operationalizes: is the learner out of capacity — or has its world simply run out of things to say?