{"record":{"id":"b2834ffa9fdc54fb","repo":"abhigyanpatwari/GitNexus","slug":"planning-must-create-or-modify-exactly-one-plan-ar","errorCode":null,"errorMessage":"planning must create or modify exactly one plan artifact; observed {len(changed)}","messagePattern":"planning must create or modify exactly one plan artifact; observed (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"eval/workflow_bench/runner_artifacts.py","lineNumber":296,"sourceCode":"                snapshot[path] = hashlib.file_digest(handle, \"sha256\").hexdigest()\n            after = os.fstat(descriptor)\n            if (opened.st_size, opened.st_mtime_ns) != (after.st_size, after.st_mtime_ns):\n                raise ValueError(f\"plan artifact changed while hashing: {path}\")\n        finally:\n            os.close(descriptor)\n    return snapshot\n\n\ndef new_plan_doc(worktree: Path, before: dict[Path, str]) -> Path:\n    \"\"\"Return the sole new or modified plan, rejecting ambiguous evidence.\"\"\"\n\n    after = snapshot_plan_docs(worktree)\n    deleted = sorted(path for path in before if path not in after)\n    if deleted:\n        raise ValueError(\"planning deleted existing plan artifact(s): \" + \", \".join(str(path) for path in deleted))\n    changed = sorted(path for path, digest in after.items() if before.get(path) != digest)\n    if len(changed) != 1:\n        raise ValueError(f\"planning must create or modify exactly one plan artifact; observed {len(changed)}\")\n    return changed[0]\n\n\ndef make_worktree(repo: Path, ref: str, parent: Path) -> Path:\n    \"\"\"Create a self-contained clone per benchmark arm.\"\"\"\n\n    target = Path(tempfile.mkdtemp(prefix=\"wfbench-\", dir=parent))\n    target.rmdir()\n    try:\n        run_checked(\n            [\n                \"git\",\n                \"clone\",\n                \"--no-local\",\n                \"--no-hardlinks\",\n                \"--no-tags\",\n                \"--quiet\",\n                str(repo),","sourceCodeStart":278,"sourceCodeEnd":314,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/d540b00184d71a896261ee02670da9a92d59d8f7/eval/workflow_bench/runner_artifacts.py#L278-L314","documentation":"Thrown by new_plan_doc when the diff of before/after plan digests does not contain exactly one changed path. The harness pins each planning step to produce a single, unambiguous plan artifact; zero changed (planning did nothing observable) or more than one (planning scattered changes) both fail the evidence contract.","triggerScenarios":"`changed = sorted(path for path, digest in after.items() if before.get(path) != digest)` has length != 1. Either no plan was created/modified, or several were.","commonSituations":"Agent wrote plans to a directory other than docs/plans (so zero observed); agent emitted multiple plan files; agent only touched non-plan files; the plan landed as .txt instead of .md/.html and was filtered out by snapshot_plan_docs.","solutions":["Confirm the plan is written under the worktree's docs/plans/ as a .md or .html file — other extensions and locations are filtered out by snapshot_plan_docs.","Ensure the planning step produces exactly one new or modified plan file.","If the legitimately created plan is not detected, check it is a regular file (not a symlink) and has a .md/.html suffix.","Merge multiple plan fragments into a single document before the snapshot is taken."],"exampleFix":"// before — agent writes two plan fragments\n(docs/plans / 'a.md').write_text(part_a)\n(docs/plans / 'b.md').write_text(part_b)\n\n// after — single plan document\n(docs/plans / 'plan.md').write_text(part_a + '\\n' + part_b)","handlingStrategy":"validation","validationCode":"from pathlib import Path\nimport re\n\ndef exactly_one_plan_changed(before, after) -> bool:\n    changed = [p for p, d in after.items() if before.get(p) != d]\n    return len(changed) == 1\n\ndef is_acceptable_plan_path(path: Path) -> bool:\n    return path.suffix.lower() in {'.md', '.html'} and path.lstat().st_mode and 0o170000 == 0o100000","typeGuard":"null","tryCatchPattern":"try:\n    plan = new_plan_doc(worktree, before)\nexcept ValueError as e:\n    if 'exactly one plan artifact' in str(e):\n        print('planning produced', e)  # inspect 0 vs N, adjust prompt/paths\n        raise\n    raise","preventionTips":["Write the single plan under docs/plans/ with a .md or .html suffix.","Do not scatter planning output across multiple files or directories.","Snapshot before/after yourself first to confirm exactly one path differs.","Ensure the plan is a regular file (not a symlink) so snapshot_plan_docs includes it."],"tags":["plan-artifacts","validation","workflow-bench","evidence-contract"],"backgroundTag":null,"analyzedSha":"d540b00184d71a896261ee02670da9a92d59d8f7","analyzedAt":"2026-08-12T19:50:25.132Z","schemaVersion":2},"datasetVersion":"2026-08-13T04:17:16.726Z"}