{"record":{"id":"1f835a868972804c","repo":"abhigyanpatwari/GitNexus","slug":"untracked-benchmark-harness-data-blocks-oracle-san","errorCode":null,"errorMessage":"untracked benchmark harness data blocks oracle sanitization","messagePattern":"untracked benchmark harness data blocks oracle sanitization","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"eval/workflow_bench/oracle_assets.py","lineNumber":309,"sourceCode":"    hidden_tree = hidden_tree_result.stdout_tail.strip()\n    if hidden_tree and (\n        len(hidden_tree) not in {40, 64} or any(character not in \"0123456789abcdefABCDEF\" for character in hidden_tree)\n    ):\n        raise ValueError(\"committed benchmark harness is not a single bounded tree\")\n\n    current = root.joinpath(*HIDDEN_HARNESS_PATH.parts)\n    if hidden_tree:\n        parent = root\n        for part in HIDDEN_HARNESS_PATH.parts:\n            parent /= part\n            try:\n                metadata = parent.lstat()\n            except OSError as exc:\n                raise ValueError(\"committed benchmark harness is missing from the clone checkout\") from exc\n            if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISDIR(metadata.st_mode):\n                raise ValueError(\"benchmark harness checkout must contain only real directories\")\n    elif current.exists() or current.is_symlink():\n        raise ValueError(\"untracked benchmark harness data blocks oracle sanitization\")\n\n    _git_checked(\n        root,\n        [\n            \"rm\",\n            \"-r\",\n            \"--force\",\n            \"--quiet\",\n            \"--ignore-unmatch\",\n            \"--\",\n            HIDDEN_HARNESS_PATH.as_posix(),\n        ],\n        timeout=120,\n    )\n    sanitized_tree = _git_checked(root, [\"write-tree\"], timeout=60)\n    deterministic_git_env = {\n        \"PATH\": os.environ.get(\"PATH\", \"/usr/local/bin:/usr/bin:/bin\"),\n        \"LANG\": \"C.UTF-8\",","sourceCodeStart":291,"sourceCodeEnd":327,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/d540b00184d71a896261ee02670da9a92d59d8f7/eval/workflow_bench/oracle_assets.py#L291-L327","documentation":"The elif branch: ls-tree -d reported NO committed harness tree (hidden_tree empty), yet the worktree path eval/workflow_bench exists (or is a symlink). Because git rm only removes tracked paths, untracked oracle data would survive sanitization and be readable by the model, so the harness refuses to proceed.","triggerScenarios":"Called when the clone's HEAD does not track eval/workflow_bench, but untracked data exists at that path in the worktree (a prior run's artifacts, a copied-in directory, or a symlink).","commonSituations":"Reusing a clone from a previous benchmark run without cleaning; a setup step that writes into eval/workflow_bench; copying oracle fixtures into the wrong clone.","solutions":["Remove the untracked data: `git -C <clone> clean -fdx -- eval/workflow_bench` or `rm -rf <clone>/eval/workflow_bench`.","Re-clone into an empty directory so no stale untracked files are present.","Audit setup commands to ensure none write under eval/workflow_bench."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"import subprocess\nfrom pathlib import Path\n\ndef harness_path_clean_when_untracked(clone: Path, rel=\"eval/workflow_bench\") -> bool:\n    p = clone / rel\n    if p.exists() or p.is_symlink():\n        tracked = subprocess.run(\n            [\"git\", \"-C\", str(clone), \"ls-tree\", \"-d\", \"--format=%(objectname)\", \"HEAD\", \"--\", rel],\n            capture_output=True, text=True,\n        ).stdout.strip()\n        return bool(tracked)\n    return True\n","typeGuard":"def is_untracked_harness_block(exc: BaseException) -> bool:\n    return isinstance(exc, ValueError) and \"untracked benchmark harness data\" in str(exc)\n","tryCatchPattern":"try:\n    oracle_assets.sanitize_clone_for_hidden_oracles(clone)\nexcept ValueError as exc:\n    quarantine(clone)\n    raise AbortTask(str(exc)) from exc\n","preventionTips":["Always sanitize a freshly cloned, never-reused directory.","Run `git clean -fdx` before sanitizing if clone reuse is unavoidable."],"tags":["git","untracked","worktree","oracle","sanitization","invariant"],"backgroundTag":null,"analyzedSha":"d540b00184d71a896261ee02670da9a92d59d8f7","analyzedAt":"2026-08-12T19:50:25.132Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}