abhigyanpatwari/GitNexus · error · ValueError

untracked benchmark harness data blocks oracle sanitization

Error message

untracked benchmark harness data blocks oracle sanitization

What it means

The elif branch: ls-tree -d reported NO committed harness tree (hidden_tree empty), yet the worktree path eval/workflow_bench exists (or is a symlink). Because git rm only removes tracked paths, untracked oracle data would survive sanitization and be readable by the model, so the harness refuses to proceed.

Source

Thrown at eval/workflow_bench/oracle_assets.py:309

    hidden_tree = hidden_tree_result.stdout_tail.strip()
    if hidden_tree and (
        len(hidden_tree) not in {40, 64} or any(character not in "0123456789abcdefABCDEF" for character in hidden_tree)
    ):
        raise ValueError("committed benchmark harness is not a single bounded tree")

    current = root.joinpath(*HIDDEN_HARNESS_PATH.parts)
    if hidden_tree:
        parent = root
        for part in HIDDEN_HARNESS_PATH.parts:
            parent /= part
            try:
                metadata = parent.lstat()
            except OSError as exc:
                raise ValueError("committed benchmark harness is missing from the clone checkout") from exc
            if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISDIR(metadata.st_mode):
                raise ValueError("benchmark harness checkout must contain only real directories")
    elif current.exists() or current.is_symlink():
        raise ValueError("untracked benchmark harness data blocks oracle sanitization")

    _git_checked(
        root,
        [
            "rm",
            "-r",
            "--force",
            "--quiet",
            "--ignore-unmatch",
            "--",
            HIDDEN_HARNESS_PATH.as_posix(),
        ],
        timeout=120,
    )
    sanitized_tree = _git_checked(root, ["write-tree"], timeout=60)
    deterministic_git_env = {
        "PATH": os.environ.get("PATH", "/usr/local/bin:/usr/bin:/bin"),
        "LANG": "C.UTF-8",

View on GitHub (pinned to d540b00184)

Solutions

  1. Remove the untracked data: `git -C <clone> clean -fdx -- eval/workflow_bench` or `rm -rf <clone>/eval/workflow_bench`.
  2. Re-clone into an empty directory so no stale untracked files are present.
  3. Audit setup commands to ensure none write under eval/workflow_bench.
Defensive patterns

Strategy: validation

Validate before calling

import subprocess
from pathlib import Path

def harness_path_clean_when_untracked(clone: Path, rel="eval/workflow_bench") -> bool:
    p = clone / rel
    if p.exists() or p.is_symlink():
        tracked = subprocess.run(
            ["git", "-C", str(clone), "ls-tree", "-d", "--format=%(objectname)", "HEAD", "--", rel],
            capture_output=True, text=True,
        ).stdout.strip()
        return bool(tracked)
    return True

Type guard

def is_untracked_harness_block(exc: BaseException) -> bool:
    return isinstance(exc, ValueError) and "untracked benchmark harness data" in str(exc)

Try / catch

try:
    oracle_assets.sanitize_clone_for_hidden_oracles(clone)
except ValueError as exc:
    quarantine(clone)
    raise AbortTask(str(exc)) from exc

Prevention

When it happens

Trigger: Called when the clone's HEAD does not track eval/workflow_bench, but untracked data exists at that path in the worktree (a prior run's artifacts, a copied-in directory, or a symlink).

Common situations: Reusing a clone from a previous benchmark run without cleaning; a setup step that writes into eval/workflow_bench; copying oracle fixtures into the wrong clone.

Related errors

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/1f835a868972804c. Report an issue: GitHub.