{"record":{"id":"945289a603679330","repo":"Hmbown/CodeWhale","slug":"codewhale-terminal-receipt-model-did-not-match-rol","errorCode":null,"errorMessage":"Codewhale terminal receipt model did not match rollout","messagePattern":"Codewhale terminal receipt model did not match rollout","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"integrations/verifiers-codewhale/codewhale_harness/harness.py","lineNumber":230,"sourceCode":"            \"--output-format\",\n            \"stream-json\",\n        ]\n        if self.config.max_turns is not None:\n            argv.extend([\"--max-turns\", str(self.config.max_turns)])\n        if self.config.disabled_tools:\n            argv.extend([\"--disallowed-tools\", \",\".join(self.config.disabled_tools)])\n        if system:\n            argv.extend([\"--append-system-prompt\", system])\n        argv.extend([\"--\", str(prompt or \"\")])\n\n        result = await runtime.run_program(argv, env)\n        if result.exit_code == 0:\n            receipt = _parse_stream_receipt(result.stdout)\n            terminal = receipt[\"terminal\"]\n            if terminal.get(\"provider\") != \"openai\":\n                raise RuntimeError(\"Codewhale terminal receipt did not use provider openai\")\n            if terminal.get(\"model\") != ctx.model:\n                raise RuntimeError(\"Codewhale terminal receipt model did not match rollout\")\n            if terminal.get(\"approval_posture\") != \"auto_tools\":\n                raise RuntimeError(\"Codewhale terminal receipt did not confirm auto tools\")\n            if terminal.get(\"sandbox_posture\") != sandbox:\n                raise RuntimeError(\"Codewhale terminal receipt sandbox did not match launch\")\n            if receipt[\"events\"].get(\"error\", 0) != 0:\n                raise RuntimeError(\"Codewhale successful run contained an error event\")\n            if terminal.get(\"status\") != \"completed\":\n                raise RuntimeError(\"Codewhale terminal receipt did not report completion\")\n            if terminal.get(\"termination_reason\") != \"resolved\":\n                raise RuntimeError(\"Codewhale terminal receipt was not resolved\")\n            trace.info[\"codewhale\"] = receipt\n        return result\n\n\ndef _has_version(output: str, version: str) -> bool:\n    return (\n        re.search(\n            rf\"(?<![0-9A-Za-z.+-]){re.escape(version)}(?![0-9A-Za-z.+-])\",","sourceCodeStart":212,"sourceCodeEnd":248,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/8880682c63083a91624de936797efa3ce9e498fd/integrations/verifiers-codewhale/codewhale_harness/harness.py#L212-L248","documentation":"The terminal receipt's model string must equal ctx.model exactly. The harness exports CODEWHALE_MODEL, DEEPSEEK_MODEL, and OPENAI_MODEL and passes --model, so a mismatch means the binary resolved a different model (its own config, alias expansion) or the facade echoed the wrong identifier — token accounting and rollout attribution would be wrong.","triggerScenarios":"Alias expansion in the receipt (requesting one id, echoing its dated snapshot); a project config inside the isolated home re-selecting a model; facades hard-coding a model name.","commonSituations":"Serving layers that canonicalize model ids; stale facade stubs after switching eval models; per-model routing configs leaking into the sandbox home.","solutions":["Pass the exact model identifier the binary echoes verbatim; avoid aliases","Clear model overrides from any config inside the isolated CODEWHALE_HOME the binary reads","Fix facades to echo the OPENAI_MODEL environment variable as the model field"],"exampleFix":"# before (facade)\nmeta = {'model': 'gpt-4o-mini'}\n# after\nimport os\nmeta = {'model': os.environ['OPENAI_MODEL']}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try:\n    result = await harness.launch(ctx, trace, runtime, endpoint, secret, mcp_urls)\nexcept RuntimeError as e:\n    if 'model did not match' in str(e):\n        record_model_drift(ctx.model, e)  # compare requested vs reported, fix config\n    raise","preventionTips":["Pass canonical model ids, not aliases","Keep the isolated home free of model overrides","Echo OPENAI_MODEL verbatim in facades"],"tags":["receipt","integration","model","harness"],"backgroundTag":null,"analyzedSha":"8880682c63083a91624de936797efa3ce9e498fd","analyzedAt":"2026-08-16T11:31:27.956Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}