{"record":{"id":"1d9dc6ab71f30722","repo":"abhigyanpatwari/GitNexus","slug":"oracle-stage-root-changed-during-verification","errorCode":null,"errorMessage":"oracle stage root changed during verification","messagePattern":"oracle stage root changed during verification","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"eval/workflow_bench/oracle_assets.py","lineNumber":493,"sourceCode":"        0o400,\n    )\n    try:\n        view = memoryview(item.payload)\n        while view:\n            written = os.write(descriptor, view)\n            if written <= 0:\n                raise OSError(\"short write while staging oracle\")\n            view = view[written:]\n        os.fchmod(descriptor, 0o400)\n        os.fsync(descriptor)\n    finally:\n        os.close(descriptor)\n\n\ndef _verify_staged_oracle(stage_root: Path, snapshot: TaskOracleSnapshot) -> None:\n    root_metadata = stage_root.lstat()\n    if stat.S_ISLNK(root_metadata.st_mode) or not stat.S_ISDIR(root_metadata.st_mode):\n        raise ValueError(\"oracle stage root changed during verification\")\n    for item in snapshot.files:\n        relative = PurePosixPath(item.target)\n        current = stage_root\n        for part in relative.parts[:-1]:\n            current /= part\n            metadata = current.lstat()\n            if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISDIR(metadata.st_mode):\n                raise ValueError(f\"oracle stage parent changed during verification: {item.target}\")\n        observed = _read_oracle_file(stage_root, relative)\n        if observed != item.payload:\n            raise ValueError(f\"oracle file changed during verification: {item.target}\")\n\n\n@contextmanager\ndef staged_task_oracle(worktree: Path, snapshot: TaskOracleSnapshot) -> Iterator[Path]:\n    \"\"\"Materialize a private random oracle root only after the model exits.\"\"\"\n\n    root = worktree.expanduser().absolute()","sourceCodeStart":475,"sourceCodeEnd":511,"githubUrl":"https://github.com/abhigyanpatwari/GitNexus/blob/d540b00184d71a896261ee02670da9a92d59d8f7/eval/workflow_bench/oracle_assets.py#L475-L511","documentation":"Raised by _verify_staged_oracle after the benchmark surrenders control to the model: the staged oracle root is re-stat()ed and must still be a real (non-symlink) directory that this harness created. A mismatch means something — the model under test, a concurrent process, or a filesystem race — replaced the stage root. This is a deliberate integrity violation rather than a normal control-flow error.","triggerScenarios":"Called via staged_task_oracle() after the `yield` returns, when stage_root.lstat() shows S_ISLNK or not S_ISDIR. Happens if the model (or any sibling process) renamed/replaced/unlinked the .wfbench-oracle-<hex> directory while it had access.","commonSituations":"An agent under benchmark tries to tamper with oracle files to cheat; a misconfigured cleanup task (cron, IDE watcher, `make clean`) sweeps the worktree; running the benchmark on a network filesystem where directory entries are not stable; a leftover foreground `rm -rf` from a prior run.","solutions":["Inspect stage_root path in the traceback and confirm no external process is mutating the worktree (lsof / ps for sweepers).","Re-run the benchmark against a clean worktree on a local filesystem (tmpfs or ext4/xfs), not NFS or a synced folder.","Audit the agent under test for code that writes inside .wfbench-oracle-* — that is forbidden tampering and must be fixed in the agent.","If running concurrently, serialize staged_task_oracle invocations so each owns its worktree exclusively."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"# Pre-flight: ensure nothing else can write the worktree before staging.\nimport os, stat\nfrom pathlib import Path\n\ndef assert_quiet_real_dir(worktree: Path) -> None:\n    st = worktree.lstat()\n    assert stat.S_ISDIR(st.st_mode) and not stat.S_ISLNK(st.st_mode), \\\n        f'worktree {worktree} is not a real directory'\n    # Optional: take an exclusive lock so concurrent runs cannot interfere.\n    fd = os.open(worktree, os.O_RDONLY)\n    import fcntl\n    fcntl.flock(fd, fcntl.LOCK_EX | fcntl.LOCK_NB)  # raises if contended","typeGuard":null,"tryCatchPattern":"try:\n    with staged_task_oracle(worktree, snapshot) as stage:\n        run_model(stage)\nexcept ValueError as exc:\n    if 'changed during verification' in str(exc):\n        log.critical('oracle tampering detected: %s', exc)\n        mark_run_invalid()\n    else:\n        raise","preventionTips":["Run the benchmark on a local filesystem with no sync daemon or editor watching the worktree.","Serialize concurrent benchmark runs so each owns its worktree exclusively.","Treat any 'changed during verification' ValueError as a security incident, not a transient failure."],"tags":["oracle","toctou","integrity","benchmark","filesystem"],"backgroundTag":null,"analyzedSha":"d540b00184d71a896261ee02670da9a92d59d8f7","analyzedAt":"2026-08-12T19:50:25.132Z","schemaVersion":2},"datasetVersion":"2026-08-12T23:17:12.415Z"}