abhigyanpatwari/GitNexus · error · ValueError

committed benchmark harness is missing from the clone checko

Error message

committed benchmark harness is missing from the clone checkout

What it means

Raised inside sanitize_clone_for_hidden_oracles while walking each component of the HIDDEN_HARNESS_PATH ('eval/workflow_bench') in the worktree. HEAD's ls-tree reported the harness as a committed tree (hidden_tree was non-empty), so the directory must exist in the checkout; lstat() on a path component raised OSError, meaning the worktree disagrees with HEAD.

Source

Thrown at eval/workflow_bench/oracle_assets.py:305

        tail_bytes=1024,
    )
    if not hidden_tree_result.ok:
        raise ValueError("cannot inspect the clone for committed benchmark harness data")
    hidden_tree = hidden_tree_result.stdout_tail.strip()
    if hidden_tree and (
        len(hidden_tree) not in {40, 64} or any(character not in "0123456789abcdefABCDEF" for character in hidden_tree)
    ):
        raise ValueError("committed benchmark harness is not a single bounded tree")

    current = root.joinpath(*HIDDEN_HARNESS_PATH.parts)
    if hidden_tree:
        parent = root
        for part in HIDDEN_HARNESS_PATH.parts:
            parent /= part
            try:
                metadata = parent.lstat()
            except OSError as exc:
                raise ValueError("committed benchmark harness is missing from the clone checkout") from exc
            if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISDIR(metadata.st_mode):
                raise ValueError("benchmark harness checkout must contain only real directories")
    elif current.exists() or current.is_symlink():
        raise ValueError("untracked benchmark harness data blocks oracle sanitization")

    _git_checked(
        root,
        [
            "rm",
            "-r",
            "--force",
            "--quiet",
            "--ignore-unmatch",
            "--",
            HIDDEN_HARNESS_PATH.as_posix(),
        ],
        timeout=120,
    )

View on GitHub (pinned to d540b00184)

Solutions

Re-clone with a full checkout: avoid --no-checkout and disable sparse-checkout for the benchmark clone.
Restore the path from HEAD before sanitizing: `git -C <clone> checkout HEAD -- eval/workflow_bench`.
If using a sparse cone, add eval/workflow_bench to the sparse paths so it is materialized.
Verify with `git -C <clone> ls-files -- eval/workflow_bench | head` shows the committed files.

Defensive patterns

Strategy: validation

Validate before calling

import subprocess, stat
from pathlib import Path

def checkout_has_harness(clone: Path, parts=("eval", "workflow_bench")) -> bool:
    cur = clone
    for part in parts:
        cur = cur / part
        try:
            st = cur.lstat()
        except OSError:
            return False
        if stat.S_ISLNK(st.st_mode) or not stat.S_ISDIR(st.st_mode):
            return False
    return True

Type guard

def is_harness_missing_from_checkout(exc: BaseException) -> bool:
    return isinstance(exc, ValueError) and "missing from the clone checkout" in str(exc)

Try / catch

from eval.workflow_bench import oracle_assets
try:
    oracle_assets.sanitize_clone_for_hidden_oracles(clone)
except ValueError as exc:
    # The clone is untrusted now; never fall back to a partially sanitized clone.
    quarantine(clone)
    raise AbortTask(f"oracle sanitization failed: {exc}") from exc

Prevention

Use a standard full clone for benchmark runs; do not combine --no-checkout or sparse-checkout with sanitize_clone_for_hidden_oracles.
Run sanitization before any setup command that could mutate the worktree.
Treat this as a hard abort: a mismatch between HEAD and the worktree means the clone state is undefined.

When it happens

Trigger: Called when git HEAD lists eval/workflow_bench as a tree but one of the eval/ or eval/workflow_bench/ directories cannot be lstat'd in the clone's working tree (sparse-checkout excluded it, a partial/--no-checkout clone, or the dir was deleted after checkout).

Common situations: Cloning with --no-checkout or a sparse-checkout cone that omits eval/; a prior task or setup command that rm'd eval/workflow_bench; an interrupted checkout.

Related errors

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/dfd71aaf3a0bc233. Report an issue: GitHub.