Hmbown/CodeWhale · error · RuntimeError

Codewhale terminal receipt did not use provider openai

Error message

Codewhale terminal receipt did not use provider openai

What it means

After a run exits 0, launch() parses the codewhale.exec-stream stdout and asserts the terminal metadata receipt reports provider 'openai'. The harness pins this via CODEWHALE_PROVIDER=openai in the environment and --provider openai on the command line so all model traffic traverses the Verifiers interception endpoint. A missing or different provider value means the run is not attributable to the intercepted endpoint, so the rollout is rejected.

Source

Thrown at integrations/verifiers-codewhale/codewhale_harness/harness.py:228

            "--sandbox",
            sandbox,
            "--output-format",
            "stream-json",
        ]
        if self.config.max_turns is not None:
            argv.extend(["--max-turns", str(self.config.max_turns)])
        if self.config.disabled_tools:
            argv.extend(["--disallowed-tools", ",".join(self.config.disabled_tools)])
        if system:
            argv.extend(["--append-system-prompt", system])
        argv.extend(["--", str(prompt or "")])

        result = await runtime.run_program(argv, env)
        if result.exit_code == 0:
            receipt = _parse_stream_receipt(result.stdout)
            terminal = receipt["terminal"]
            if terminal.get("provider") != "openai":
                raise RuntimeError("Codewhale terminal receipt did not use provider openai")
            if terminal.get("model") != ctx.model:
                raise RuntimeError("Codewhale terminal receipt model did not match rollout")
            if terminal.get("approval_posture") != "auto_tools":
                raise RuntimeError("Codewhale terminal receipt did not confirm auto tools")
            if terminal.get("sandbox_posture") != sandbox:
                raise RuntimeError("Codewhale terminal receipt sandbox did not match launch")
            if receipt["events"].get("error", 0) != 0:
                raise RuntimeError("Codewhale successful run contained an error event")
            if terminal.get("status") != "completed":
                raise RuntimeError("Codewhale terminal receipt did not report completion")
            if terminal.get("termination_reason") != "resolved":
                raise RuntimeError("Codewhale terminal receipt was not resolved")
            trace.info["codewhale"] = receipt
        return result


def _has_version(output: str, version: str) -> bool:
    return (

View on GitHub (pinned to 8880682c63)

Solutions

1. Inspect the receipt the harness stores on prior passing runs (trace.info['codewhale']) to see the expected shape
2. Remove any resolved_env entries that override CODEWHALE_PROVIDER or the OPENAI_* variables the harness sets
3. Make facades echo provider='openai' in the terminal metadata event
4. Re-run with the harness-installed binary (drop binary_path) to isolate whether the facade is at fault

Example fix

# before (facade metadata)
meta = {'receipt_kind': 'terminal', 'model': model, 'status': 'completed'}
# after
meta = {'receipt_kind': 'terminal', 'provider': 'openai', 'model': model, 'status': 'completed'}

env = dict(harness.config.resolved_env)
for key in ('CODEWHALE_PROVIDER', 'OPENAI_BASE_URL', 'OPENAI_API_KEY', 'OPENAI_MODEL'):
    env.pop(key, None)  # never let resolved_env override harness-owned interception variables

try:
    result = await harness.launch(ctx, trace, runtime, endpoint, secret, mcp_urls)
except RuntimeError as e:
    if 'did not use provider openai' in str(e):
        mark_rollout_failed(trace, reason='provider_mismatch')  # score 0, keep the run alive
    raise

When it happens

Trigger: A facade binary ignoring --provider or omitting provider from its terminal metadata; resolved_env overriding CODEWHALE_PROVIDER; stub binaries used for dry-runs that emit schema-conformant events without the provider field.

Common situations: Swapping in a mock codewhale for cheap testing; env-merge order changes clobbering harness variables; binary versions that rename the field.

Related errors

• Codewhale terminal receipt model did not match rollout
• Codewhale terminal receipt did not confirm auto tools
• Codewhale terminal receipt sandbox did not match launch
• Codewhale successful run contained an error event
• Codewhale terminal receipt did not report completion

AI-assisted analysis of Hmbown/CodeWhale@8880682c63 (2026-08-16). Data as JSON: /api/errors/a0f4eec5504024d4. Report an issue: GitHub.