{"record":{"id":"b893d40a14a59334","repo":"abhigyanpatwari/GitNexus","slug":"phase-did-not-create-or-change-its-required-artifa","errorCode":null,"errorMessage":"phase did not create or change its required artifact: {relative}","messagePattern":"phase did not create or change its required artifact: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"eval/workflow_bench/runner_artifacts.py","lineNumber":207,"sourceCode":"    worktree: Path,\n    before: dict[str, str],\n    *,\n    allowed_artifact: Path,\n) -> None:\n    \"\"\"Require a phase to change only its one explicit workspace artifact.\"\"\"\n\n    root = worktree.expanduser().absolute()\n    artifact = allowed_artifact.expanduser().absolute()\n    try:\n        relative = PurePosixPath(artifact.relative_to(root).as_posix())\n    except ValueError as exc:\n        raise ValueError(f\"phase artifact escapes the workspace: {allowed_artifact}\") from exc\n    after = workspace_snapshot(root)\n    changed = {path for path in before.keys() | after.keys() if before.get(path) != after.get(path)}\n    artifact_key = relative.as_posix()\n    artifact_state = after.get(artifact_key)\n    if before.get(artifact_key) == artifact_state:\n        raise ValueError(f\"phase did not create or change its required artifact: {relative}\")\n    if artifact_state is None or not artifact_state.startswith(\"f:\"):\n        raise ValueError(f\"phase artifact must be a regular non-symlink file: {relative}\")\n\n    try:\n        metadata = artifact.lstat()\n        descriptor = os.open(artifact, os.O_RDONLY | getattr(os, \"O_NOFOLLOW\", 0))\n    except OSError as exc:\n        raise ValueError(f\"phase artifact must be a readable regular non-symlink file: {relative}\") from exc\n    try:\n        opened = os.fstat(descriptor)\n        if (\n            stat.S_ISLNK(metadata.st_mode)\n            or not stat.S_ISREG(metadata.st_mode)\n            or not stat.S_ISREG(opened.st_mode)\n            or metadata.st_dev != opened.st_dev\n            or metadata.st_ino != opened.st_ino\n        ):\n            raise ValueError(f\"phase artifact must be a regular non-symlink file: {relative}\")","sourceCodeStart":189,"sourceCodeEnd":225,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/d540b00184d71a896261ee02670da9a92d59d8f7/eval/workflow_bench/runner_artifacts.py#L189-L225","documentation":"Raised by enforce_phase_workspace (runner_artifacts.py:207) when the allowed artifact's state in the 'after' snapshot is identical to its state in the 'before' snapshot. The phase contract says a planning/work phase must create or modify its one declared artifact; an unchanged artifact means the phase produced no evidence and the result is unverifiable.","triggerScenarios":"before.get(artifact_key) == artifact_state after the phase ran. The model did not write to the declared artifact path, wrote to a different path, or wrote then reverted it; the artifact path was wrong (typo / wrong directory).","commonSituations":"The planning prompt told the model to write docs/plans/foo.md but it wrote docs/plans/bar.md; the work phase's artifact is mis-specified relative to what the model actually edits; the model failed silently and produced nothing; the model edited the file but then ran git checkout to revert.","solutions":["Check the run transcript to see what path the model actually wrote to; align allowed_artifact with that path or fix the prompt.","Confirm the artifact_key (relative posix path) matches what the model writes — mismatches in casing or trailing slashes cause this.","If the model genuinely produced nothing, treat it as a failed run (record resolved=False) rather than retry with the same prompt."],"exampleFix":"# before: artifact path does not match what the model writes\nenforce_phase_workspace(wt, before, allowed_artifact=wt/'docs/plans/plan-v1.md')\n# model writes docs/plans/plan-v2.md\n\n# after\nenforce_phase_workspace(wt, before, allowed_artifact=wt/'docs/plans/plan-v2.md')","handlingStrategy":"validation","validationCode":"from pathlib import Path\nimport os\n\nartifact = Path(allowed_artifact).expanduser().absolute()\nassert artifact.exists(), f\"artifact not created: {artifact}\"\nassert artifact.is_file() and not artifact.is_symlink(), (\n    f\"artifact is not a regular file: {artifact}\")\n# The phase must have changed it relative to `before` — caller checks the\n# snapshot entry, but a quick existence/type check catches the common case.","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Align the allowed_artifact path with what the model prompt instructs it to write.","Inspect the run transcript when this fires to find the path the model actually used.","Do not retry blindly — a phase that produced nothing is a failed run."],"tags":["phase-boundary","artifact","evidence"],"backgroundTag":null,"analyzedSha":"d540b00184d71a896261ee02670da9a92d59d8f7","analyzedAt":"2026-08-12T19:50:25.132Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}