{"record":{"id":"00495ad03a17480b","repo":"abhigyanpatwari/GitNexus","slug":"candidate-overlay-is-byte-identical-to-the-incumbe","errorCode":null,"errorMessage":"candidate overlay is byte-identical to the incumbent skills","messagePattern":"candidate overlay is byte-identical to the incumbent skills","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"eval/workflow_bench/evolution.py","lineNumber":404,"sourceCode":"\n    mkdir_command = [\"/bin/mkdir\", \"-p\", f\"{SANDBOX_TMP}/wfbench-empty-hooks\"]\n    mkdir_result = sandbox.run(\n        mkdir_command,\n        timeout=60,\n        env=build_sandbox_environment(),\n    )\n    if not mkdir_result.ok:\n        raise ManagedProcessError(mkdir_command, mkdir_result)\n\n    command, added = _sandbox_overlay_git(sandbox, [\"add\", \"--\", *relative_paths])\n    if not added.ok:\n        raise ManagedProcessError(command, added)\n    command, changed = _sandbox_overlay_git(\n        sandbox,\n        [\"diff\", \"--cached\", \"--quiet\", \"--no-ext-diff\", \"--no-textconv\", \"--\"],\n    )\n    if changed.returncode == 0:\n        raise ValueError(\"candidate overlay is byte-identical to the incumbent skills\")\n    if changed.returncode != 1:\n        raise ManagedProcessError(command, changed)\n\n    command, committed = _sandbox_overlay_git(\n        sandbox,\n        [\n            \"commit\",\n            \"--quiet\",\n            \"--no-verify\",\n            \"-m\",\n            \"benchmark candidate skill overlay\",\n        ],\n        extra_config=(\n            \"user.name=workflow-bench\",\n            \"user.email=workflow-bench@invalid\",\n        ),\n    )\n    if not committed.ok:","sourceCodeStart":386,"sourceCodeEnd":422,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/d540b00184d71a896261ee02670da9a92d59d8f7/eval/workflow_bench/evolution.py#L386-L422","documentation":"Raised by commit_candidate_overlay after it stages the proposer's skill overlay into the sandbox git clone and runs `git diff --cached --quiet`. Exit code 0 means the staged tree is byte-identical to HEAD, so the candidate introduced no change. The harness refuses to spend a benchmark run on a no-op candidate, since promotion evidence would be pure noise.","triggerScenarios":"The proposer copied the incumbent skill verbatim into the overlay; its only edit was a sed/heredoc that matched nothing; whitespace/encoding that already matched; or the overlay only touches skills no benchmarked arm loads (so the bytes round-trip). Any of these leaves `git add` with nothing to diff against HEAD.","commonSituations":"First-generation conservative proposers; an overlay built with `cp <incumbent> <overlay>` where the follow-up edit step was skipped or failed silently inside the no-Write-tool proposer session; re-running the same candidate twice.","solutions":["Before staging, diff the overlay against the incumbent: `diff -r <incumbent>/.claude/skills/<skill> <overlay>/.claude/skills/<skill>` and confirm at least one changed byte.","Re-read the proposer's proposal file — if it claims a change but the bytes match, the proposer's in-session edit failed; regenerate the candidate.","If the no-op is intentional (e.g. testing the gate), catch the ValueError and treat the generation as 'no candidate produced'."],"exampleFix":"# before\ncommit_candidate_overlay(overlay, worktree=clone, sandbox=session)\n\n# after: skip no-op overlays before they reach git\nfrom eval.workflow_bench.evolution import candidate_overlay_payload\nimport pathlib\n\ndef _incumbent_bytes(clone, rel):\n    p = pathlib.Path(clone) / rel\n    return p.read_bytes() if p.exists() else b\"\"\n\ndigest, payload = candidate_overlay_payload(overlay)\nif all(_incumbent_bytes(clone, rel.as_posix()) == content for rel, content in payload):\n    raise SystemExit(\"no-op candidate; refusing to stage\")\ncommit_candidate_overlay(overlay, worktree=clone, sandbox=session)","handlingStrategy":"validation","validationCode":"from pathlib import Path\nfrom eval.workflow_bench.evolution import candidate_overlay_payload\n\ndef overlay_changes_anything(overlay: Path, clone: Path) -> bool:\n    \"\"\"True if at least one overlay byte differs from the incumbent clone.\"\"\"\n    _, payload = candidate_overlay_payload(overlay)\n    for rel, content in payload:\n        incumbent = clone / Path(*rel.parts)\n        if not incumbent.exists() or incumbent.read_bytes() != content:\n            return True\n    return False\n\n# gate the call\nif not overlay_changes_anything(overlay, clone):\n    raise SystemExit(\"no-op candidate; skip commit_candidate_overlay\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always diff overlay vs incumbent before staging; a zero diff is a logic error in the proposer.","Have the proposer write a proposal file whose stated change you can verify byte-for-byte.","Treat a no-op candidate as 'no generation', not as a failure to retry unchanged."],"tags":["workflow-bench","evolution","git","validation","candidate-overlay"],"backgroundTag":null,"analyzedSha":"d540b00184d71a896261ee02670da9a92d59d8f7","analyzedAt":"2026-08-12T19:50:25.132Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}