{"record":{"id":"9c34dbec26cf466b","repo":"Hmbown/CodeWhale","slug":"codewhale-successful-run-contained-an-error-event","errorCode":null,"errorMessage":"Codewhale successful run contained an error event","messagePattern":"Codewhale successful run contained an error event","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"integrations/verifiers-codewhale/codewhale_harness/harness.py","lineNumber":236,"sourceCode":"            argv.extend([\"--disallowed-tools\", \",\".join(self.config.disabled_tools)])\n        if system:\n            argv.extend([\"--append-system-prompt\", system])\n        argv.extend([\"--\", str(prompt or \"\")])\n\n        result = await runtime.run_program(argv, env)\n        if result.exit_code == 0:\n            receipt = _parse_stream_receipt(result.stdout)\n            terminal = receipt[\"terminal\"]\n            if terminal.get(\"provider\") != \"openai\":\n                raise RuntimeError(\"Codewhale terminal receipt did not use provider openai\")\n            if terminal.get(\"model\") != ctx.model:\n                raise RuntimeError(\"Codewhale terminal receipt model did not match rollout\")\n            if terminal.get(\"approval_posture\") != \"auto_tools\":\n                raise RuntimeError(\"Codewhale terminal receipt did not confirm auto tools\")\n            if terminal.get(\"sandbox_posture\") != sandbox:\n                raise RuntimeError(\"Codewhale terminal receipt sandbox did not match launch\")\n            if receipt[\"events\"].get(\"error\", 0) != 0:\n                raise RuntimeError(\"Codewhale successful run contained an error event\")\n            if terminal.get(\"status\") != \"completed\":\n                raise RuntimeError(\"Codewhale terminal receipt did not report completion\")\n            if terminal.get(\"termination_reason\") != \"resolved\":\n                raise RuntimeError(\"Codewhale terminal receipt was not resolved\")\n            trace.info[\"codewhale\"] = receipt\n        return result\n\n\ndef _has_version(output: str, version: str) -> bool:\n    return (\n        re.search(\n            rf\"(?<![0-9A-Za-z.+-]){re.escape(version)}(?![0-9A-Za-z.+-])\",\n            output,\n        )\n        is not None\n    )\n\n","sourceCodeStart":218,"sourceCodeEnd":254,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/8880682c63083a91624de936797efa3ce9e498fd/integrations/verifiers-codewhale/codewhale_harness/harness.py#L218-L254","documentation":"The parsed event counter must show zero 'error' events for a run that exited 0. Codewhale finished cleanly but emitted at least one error event mid-run, so the harness invalidates the rollout: the trajectory is no longer clean even though the exit code says success.","triggerScenarios":"A tool call fails once and the run recovers; the interception endpoint returns a transient 5xx that the binary logs as an error event before retrying; internal failures classified as error events by the binary.","commonSituations":"Flaky MCP servers behind mcp_urls; unstable local interception endpoints; network blips during long rollouts.","solutions":["Check the events summary the harness attaches to the trace (trace.info['codewhale']['events']) and rerun; transient causes usually clear","Stabilize the failing dependency seen in the error events (restart the flaky MCP server, fix endpoint reliability)","Filter expected noise at its source so the binary never classifies it as an error event"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"for attempt in range(2):\n    try:\n        result = await harness.launch(ctx, trace, runtime, endpoint, secret, mcp_urls)\n        break\n    except RuntimeError as e:\n        if 'contained an error event' not in str(e) or attempt == 1:\n            raise\n        restart_flaky_dependency()  # e.g. the MCP server behind mcp_urls","preventionTips":["Health-check MCP servers and the interception endpoint before rollouts","Treat any error event as a signal to stabilize the environment, not relax the check","Watch trace.info['codewhale']['events'] trends across rollouts"],"tags":["receipt","integration","events","harness"],"backgroundTag":null,"analyzedSha":"8880682c63083a91624de936797efa3ce9e498fd","analyzedAt":"2026-08-16T11:31:27.956Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}