Hmbown/CodeWhale · error · RuntimeError

Codewhale terminal receipt model did not match rollout

Error message

Codewhale terminal receipt model did not match rollout

What it means

The terminal receipt's model string must equal ctx.model exactly. The harness exports CODEWHALE_MODEL, DEEPSEEK_MODEL, and OPENAI_MODEL and passes --model, so a mismatch means the binary resolved a different model (its own config, alias expansion) or the facade echoed the wrong identifier — token accounting and rollout attribution would be wrong.

Source

Thrown at integrations/verifiers-codewhale/codewhale_harness/harness.py:230

            "--output-format",
            "stream-json",
        ]
        if self.config.max_turns is not None:
            argv.extend(["--max-turns", str(self.config.max_turns)])
        if self.config.disabled_tools:
            argv.extend(["--disallowed-tools", ",".join(self.config.disabled_tools)])
        if system:
            argv.extend(["--append-system-prompt", system])
        argv.extend(["--", str(prompt or "")])

        result = await runtime.run_program(argv, env)
        if result.exit_code == 0:
            receipt = _parse_stream_receipt(result.stdout)
            terminal = receipt["terminal"]
            if terminal.get("provider") != "openai":
                raise RuntimeError("Codewhale terminal receipt did not use provider openai")
            if terminal.get("model") != ctx.model:
                raise RuntimeError("Codewhale terminal receipt model did not match rollout")
            if terminal.get("approval_posture") != "auto_tools":
                raise RuntimeError("Codewhale terminal receipt did not confirm auto tools")
            if terminal.get("sandbox_posture") != sandbox:
                raise RuntimeError("Codewhale terminal receipt sandbox did not match launch")
            if receipt["events"].get("error", 0) != 0:
                raise RuntimeError("Codewhale successful run contained an error event")
            if terminal.get("status") != "completed":
                raise RuntimeError("Codewhale terminal receipt did not report completion")
            if terminal.get("termination_reason") != "resolved":
                raise RuntimeError("Codewhale terminal receipt was not resolved")
            trace.info["codewhale"] = receipt
        return result


def _has_version(output: str, version: str) -> bool:
    return (
        re.search(
            rf"(?<![0-9A-Za-z.+-]){re.escape(version)}(?![0-9A-Za-z.+-])",

View on GitHub (pinned to 8880682c63)

Solutions

1. Pass the exact model identifier the binary echoes verbatim; avoid aliases
2. Clear model overrides from any config inside the isolated CODEWHALE_HOME the binary reads
3. Fix facades to echo the OPENAI_MODEL environment variable as the model field

Example fix

# before (facade)
meta = {'model': 'gpt-4o-mini'}
# after
import os
meta = {'model': os.environ['OPENAI_MODEL']}

try:
    result = await harness.launch(ctx, trace, runtime, endpoint, secret, mcp_urls)
except RuntimeError as e:
    if 'model did not match' in str(e):
        record_model_drift(ctx.model, e)  # compare requested vs reported, fix config
    raise

When it happens

Trigger: Alias expansion in the receipt (requesting one id, echoing its dated snapshot); a project config inside the isolated home re-selecting a model; facades hard-coding a model name.

Common situations: Serving layers that canonicalize model ids; stale facade stubs after switching eval models; per-model routing configs leaking into the sandbox home.

Related errors

• Codewhale terminal receipt did not use provider openai
• Codewhale terminal receipt did not confirm auto tools
• Codewhale terminal receipt sandbox did not match launch
• Codewhale successful run contained an error event
• Codewhale terminal receipt did not report completion

AI-assisted analysis of Hmbown/CodeWhale@8880682c63 (2026-08-16). Data as JSON: /api/errors/945289a603679330. Report an issue: GitHub.