Pricing Multi-Turn SWE-Bench and Verifiable RL (RLVR) Environment Contracts

Constructing a hermetic, network-isolated Docker training environment for a real-world C++, Rust, or polyglot Python/TypeScript microservice repository involves two completely distinct cost drivers. First comes **Environment Engineering**: cloning a historical commit graph, pinning broken transitive dependencies from two years ago, compiling native extensions without internet egress, and verifying deterministic test-suite execution in under ninety seconds so RL rollout workers don't bottleneck GPU utilization. That initial containerization legitimately takes a senior DevOps/systems engineer four to ten hours ($550 to $1,400).

Once that base Docker image builds cleanly and caches inside your container registry, authoring twenty-five separate **Task Instances** (each consisting of a realistic issue prompt, a fail-to-pass unit test patch `F2P`, zero pass-to-pass `P2P` regressions, and a human gold patch or multi-turn trajectory) on top of that exact same repository takes forty-five to eighty minutes per task ($85 to $145). Vendors quoting a flat $275 across 500 tasks are charging you that $1,000 container-setup fee over and over on tasks 2 through 25!

To align incentives without starving repository diversity, structure every SWE-bench / RLVR contract as a **Two-Part Tariff**: Line 1 pays a fixed **Per-Repository Base Image Fee** ($650–$1,250) subject to a minimum repo complexity gate (>=25,000 lines of code, >=150 existing tests, permissive MIT/Apache-2.0/BSD license, and <60s container test runtime), and Line 2 pays a **Complexity-Tiered Per-Task Fee** ($80–$165) capped at 25 to 40 tasks maximum per repository so your training distribution spans 40+ distinct architectures rather than overfitting one monorepo.

• Line 1 (Base Repo Dockerization): $650–$1,250 one-time fee | Requires zero-network build and <=60s test harness runtime.
• Line 2 (Task Instance + Gold Patch + F2P Test): $85–$165/task | Capped at <=30 tasks per base repo to prevent repo concentration.
• Compute-Efficiency Bonus/Gate: Reject bloated 18GB Docker images that crash Kubernetes RL rollout nodes.
• Difficulty Floor Gate: Run your current SFT baseline checkpoint pass@8 against submitted tasks; reject trivial tasks scoring >50% pass@8.

During GRPO or PPO reinforcement learning with verifiable rewards, policy models ruthlessly exploit weak unit tests—hardcoding expected dictionary outputs, mocking test assertions, or modifying test files if filesystem permissions are misconfigured. If a vendor delivers a $130 RLVR gym where only a single `assert result == 42` guards the reward signal, your RL run will reward-hack within 200 steps, wasting tens of thousands in GPU hours.

Protect your SOW commercially by adding three automated **Anti-Hacking Acceptance Checks** before releasing Line-2 task fees: (1) `F2P` test diffs are mounted read-only outside the agent's writable workspace during grading; (2) every `F2P` suite contains at least three distinct parameterized input cases including randomized or boundary values; and (3) running `git checkout HEAD` (unpatched repo) exits non-zero on `F2P` while `git apply gold.patch` exits `0` across three consecutive runs with `--random-order` enabled (catching flaky timing tests).

Beyond pure Git repositories, 2026 enterprise teams commission stateful Model Context Protocol (`MCP`) tool-use sandboxes—mocking Jira, Salesforce, Stripe, Postgres, and Slack state databases so agents practice 15-turn enterprise workflows safely. Building a reusable stateful mock service typically prices at $2,500 to $5,000 per mock app ecosystem, plus $45 to $95 per human trajectory + SQL database-state verification assertion.

When contracting for browser-use or OSWorld-style desktop GUI gyms (where agents click DOM elements, manipulate spreadsheets, and reconcile PDFs across multiple tabs), separate **Deterministic State-Snapshot Verification** from fragile pixel-coordinate matching. Vendors who grade browser trajectories using brittle CSS selector strings force you to throw away the dataset six months later; vendors who serialize an offline Dockerized web clone (such as self-hosted GitLab, OwnCloud, Plane, or Odoo instances) and verify final PostgreSQL database state via deterministic SQL queries deliver evergreen RLVR gyms that survive across multiple model generations.

Budgeting for **Rollout Compute Compatibility** is another frequently overlooked commercial detail in RLVR gym SOWs. During online GRPO or PPO reinforcement learning, your training cluster spins up 512 to 4,096 concurrent container instances per training step. If a vendor delivers bloated 22-gigabyte Docker images requiring 16GB of RAM and four minutes of cold-start compilation per episode, your AWS/CoreWeave CPU rollout bill will dwarf the entire vendor SOW! Specify strict resource ceilings inside Exhibit A: maximum compressed image size `<= 5.0 GB`, container cold-start `<= 3.5 seconds`, peak episode memory `<= 4.0 GB RAM`, and `x86_64` + `arm64` multi-arch manifest support.

Finally, negotiate **Perpetual Environment Derivative Rights** explicitly. Unlike static text rows, an RLVR Docker gym is executable software infrastructure: ensure your MSA grants unrestricted perpetual rights to fork the Dockerfiles, synthesize millions of automated policy rollouts, mutate prompt parameters procedurally, and publish sanitized evaluation subsets if your lab open-sources a technical report—with zero per-rollout royalty or container-seat metering.

One subtle legal trap in SWE-bench and RLVR environment procurement involves the SPDX software license of the upstream open-source GitHub repositories your vendor selects for containerization. If a vendor carelessly builds 40% of your paid RLVR gyms out of `GPL-3.0-only` or `AGPL-3.0` repositories—or scrapes private/unlicensed repositories lacking explicit permissive grants—enterprise IP counsel may block your team from redistributing evaluation harnesses or commercializing code-generation checkpoints.

Restrict Line-1 Base Repository eligibility strictly to OSI-approved permissive licenses (`MIT`, `Apache-2.0`, `BSD-2-Clause`, `BSD-3-Clause`, `ISC`, `MPL-2.0`) verified via automated `licensee` / `scancode-toolkit` manifest scans inside every container image, or use your own internal proprietary corporate repositories under VDI.

Additionally, require a **Post-Cutoff Commit Window** (for example, issues and pull requests created after January 2026, or wholly original synthetic feature specifications authored from scratch against existing permissive codebases) so your new RLVR gyms aren't already memorized inside your base pre-training checkpoint. Testing every candidate task against your base model's `pass@8` before approving Line-2 milestone release ensures 100% of your RLVR budget buys genuine frontier gradient signal rather than memorized GitHub history.

For long-horizon agentic debugging gyms spanning 20 to 40 tool invocations (`bash`, `ripgrep`, `gdb`, `pytest`, `git bisect`), structure optional **Trajectory Step-Efficiency Bonuses** on human gold demonstrations: reward senior engineers who demonstrate clean diagnostic localization (isolating the root cause via targeted `rg` and minimal reproduction scripts in 8 turns rather than brute-force reading 35 files). Clean, low-turn diagnostic gold traces warm-start cold-start SFT policies dramatically better before online GRPO rollout training begins, cutting total GPU reinforcement-learning convergence hours on your cluster by 20% to 30%.

Require every delivered container gym to include a machine-readable `environment_manifest.json` documenting exact commit SHA, SPDX license expression, CPU/RAM footprint, median test-execution wall-clock milliseconds, and base-model `pass@8` calibration difficulty so your RL orchestration cluster schedules rollouts deterministically from Day 1.