abhigyanpatwari/GitNexus · error · SandboxError

sanitized graph source exceeds the scrub byte limit

Error message

sanitized graph source exceeds the scrub byte limit

What it means

Raised when the running total of scanned file bytes exceeds MAX_GRAPH_SCRUB_TOTAL_BYTES (2 GiB). Only files no larger than MAX_GRAPH_SCRUB_FILE_BYTES (512 KiB) each are read for marker matching; if their cumulative size tops 2 GiB the harness aborts rather than read unbounded data.

Source

Thrown at eval/workflow_bench/sanitized_graph.py:196

            path_matches = any(marker in relative_text for marker in GRAPH_MARKERS)
            if path_matches:
                path = Path(entry.path)
                if stat.S_ISDIR(metadata.st_mode) and not stat.S_ISLNK(metadata.st_mode):
                    shutil.rmtree(path)
                else:
                    path.unlink()
                removed.append(relative_text)
                continue
            if stat.S_ISDIR(metadata.st_mode):
                pending.append((Path(entry.path), relative))
                continue
            if stat.S_ISLNK(metadata.st_mode) or not stat.S_ISREG(metadata.st_mode):
                continue
            if metadata.st_size > MAX_GRAPH_SCRUB_FILE_BYTES:
                continue
            scanned_bytes += metadata.st_size
            if scanned_bytes > MAX_GRAPH_SCRUB_TOTAL_BYTES:
                raise SandboxError("sanitized graph source exceeds the scrub byte limit")
            descriptor = os.open(entry.path, os.O_RDONLY | getattr(os, "O_NOFOLLOW", 0))
            try:
                opened = os.fstat(descriptor)
                if not stat.S_ISREG(opened.st_mode) or (opened.st_dev, opened.st_ino, opened.st_size) != (
                    metadata.st_dev,
                    metadata.st_ino,
                    metadata.st_size,
                ):
                    raise SandboxError(f"sanitized graph source changed while opening: {relative}")
                chunks: list[bytes] = []
                remaining = MAX_GRAPH_SCRUB_FILE_BYTES + 1
                while remaining > 0:
                    chunk = os.read(descriptor, min(64 * 1024, remaining))
                    if not chunk:
                        break
                    chunks.append(chunk)
                    remaining -= len(chunk)
                payload = b"".join(chunks)

View on GitHub (pinned to d540b00184)

Solutions

1. Exclude large corpora (docs, fixtures, generated code) from the task snapshot before graph preparation.
2. Declare bulky read-only data as sandbox_dependencies instead of sandbox_copy so it bypasses the scrubbed seed.
3. Raise MAX_GRAPH_SCRUB_TOTAL_BYTES only if the scan budget is genuinely too small and the marker set is still fully checked.

import os
from workflow_bench.sanitized_graph import MAX_GRAPH_SCRUB_FILE_BYTES, MAX_GRAPH_SCRUB_TOTAL_BYTES

def estimate_scrub_bytes(root):
    total = 0
    for dirpath, dirnames, files in os.walk(root):
        parts = os.path.relpath(dirpath, root).split(os.sep)
        if parts and parts[0] in {".git", ".gitnexus"}:
            dirnames[:] = []
            continue
        for name in files:
            try:
                size = os.stat(os.path.join(dirpath, name)).st_size
            except OSError:
                continue
            if size <= MAX_GRAPH_SCRUB_FILE_BYTES:
                total += size
            if total > MAX_GRAPH_SCRUB_TOTAL_BYTES:
                raise RuntimeError(f"scrub byte budget exceeded: ~{total} bytes")
    return total

When it happens

Trigger: A seed containing many small files (each <= 512 KiB) whose sizes sum past 2 GiB is fed to the scrubber: e.g. huge docs corpora, thousands of small generated source files, vendored test fixtures.

Common situations: Vendored docs or test corpora in the snapshot; a task repo bundling many subprojects; generated/minified trees that survived history pruning.

Related errors

• sanitized graph source exceeds the scrub entry limit
• transcript_artifacts exceeds the per-row session limit of {M
• transcript_artifacts exceeds the global evidence limit of {M
• {label} is unavailable: {path}: {exc}
• {label} must be a real non-symlink directory: {path}

AI-assisted analysis of abhigyanpatwari/GitNexus@d540b00184 (2026-08-12). Data as JSON: /api/errors/aadb6fcb04784901. Report an issue: GitHub.