abhigyanpatwari/GitNexus · error · ValueError

plan artifact changed while opening: {path}

Error message

plan artifact changed while opening: {path}

What it means

Thrown by snapshot_plan_docs during its TOCTOU hardening of each plan artifact. The code lstat's the path, then re-opens it with O_NOFOLLOW and fstat's the resulting file descriptor; if the device id (st_dev) or inode (st_ino) differ between the two stats, the file at that path was swapped between the check and the open. The harness treats plan evidence as untrusted input, so a mismatch (race, symlink swap, concurrent rewrite) aborts the snapshot rather than hashing attacker-controlled bytes.

Source

Thrown at eval/workflow_bench/runner_artifacts.py:276

    if not plans.exists():
        return {}
    if plans.is_symlink() or not plans.is_dir():
        raise ValueError(f"plan directory must be a real directory: {plans}")

    snapshot: dict[Path, str] = {}
    for path in sorted(plans.iterdir()):
        if path.suffix.lower() not in {".md", ".html"}:
            continue
        metadata = path.lstat()
        if stat.S_ISLNK(metadata.st_mode):
            raise ValueError(f"plan artifact cannot be a symlink: {path}")
        if not stat.S_ISREG(metadata.st_mode):
            raise ValueError(f"plan artifact must be a regular file: {path}")
        descriptor = os.open(path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0))
        try:
            opened = os.fstat(descriptor)
            if not stat.S_ISREG(opened.st_mode) or opened.st_dev != metadata.st_dev or opened.st_ino != metadata.st_ino:
                raise ValueError(f"plan artifact changed while opening: {path}")
            with os.fdopen(descriptor, "rb", closefd=False) as handle:
                snapshot[path] = hashlib.file_digest(handle, "sha256").hexdigest()
            after = os.fstat(descriptor)
            if (opened.st_size, opened.st_mtime_ns) != (after.st_size, after.st_mtime_ns):
                raise ValueError(f"plan artifact changed while hashing: {path}")
        finally:
            os.close(descriptor)
    return snapshot


def new_plan_doc(worktree: Path, before: dict[Path, str]) -> Path:
    """Return the sole new or modified plan, rejecting ambiguous evidence."""

    after = snapshot_plan_docs(worktree)
    deleted = sorted(path for path in before if path not in after)
    if deleted:
        raise ValueError("planning deleted existing plan artifact(s): " + ", ".join(str(path) for path in deleted))
    changed = sorted(path for path, digest in after.items() if before.get(path) != digest)

View on GitHub (pinned to d540b00184)

Solutions

  1. Stop any concurrent writer of docs/plans/ during the run; the harness snapshots serially and assumes quiescence.
  2. Run on a local filesystem (ext4/xfs/apfs) that guarantees stable st_dev/st_ino across the open window.
  3. If the swap is legitimate (the agent legitimately rewrites the plan), ensure the snapshot is taken only after the agent process has exited, not while it is still running.
  4. Re-run the benchmark arm; transient races against a finishing agent usually clear once the agent is joined.

Example fix

// before: snapshot while agent still running
snapshot = snapshot_plan_docs(worktree)  # agent may rewrite plan concurrently

// after: join the agent first, then snapshot a quiescent tree
agent_proc.wait()
snapshot = snapshot_plan_docs(worktree)
Defensive patterns

Strategy: validation

Validate before calling

import os, stat
from pathlib import Path

def is_quiescent(path: Path) -> bool:
    """True if the path's stat is stable across two reads (no concurrent writer)."""
    a = path.lstat()
    import time; time.sleep(0.01)
    b = path.lstat()
    return a.st_dev == b.st_dev and a.st_ino == b.st_ino and (a.st_size, a.st_mtime_ns) == (b.st_size, b.st_mtime_ns)

Type guard

from pathlib import Path

def is_safe_plan_path(path: Path) -> bool:
    """A plan path safe to snapshot: regular file, not a symlink."""
    import stat
    try:
        st = path.lstat()
    except OSError:
        return False
    return stat.S_ISREG(st.st_mode)

Try / catch

from eval.workflow_bench.runner_artifacts import snapshot_plan_docs
try:
    snap = snapshot_plan_docs(worktree)
except ValueError as e:
    if 'changed while opening' in str(e):
        # TOCTOU race: re-run after confirming no writer is active
        raise SystemExit('plan file swapped mid-snapshot; quiesce agents and retry')
    raise

Prevention

When it happens

Trigger: A concurrent process replaces a docs/plans/*.md or *.html file in the small window between path.lstat() and os.fstat(descriptor). Concretely: metadata.st_dev != opened.st_dev or metadata.st_ino != opened.st_ino after os.open(path, O_RDONLY | O_NOFOLLOW).

Common situations: An agent under evaluation rewrites its plan file mid-snapshot; a filesystem (some network/FUSE mounts) that does not report stable inodes; two benchmark arms sharing a worktree; an external watcher regenerating plans on save.

Related errors

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/d1d41d1b116f6290. Report an issue: GitHub.