unclecode/crawl4ai · error · ArtifactTooLarge

artifact exceeds {MAX_ARTIFACT_BYTES} bytes

Error message

artifact exceeds {MAX_ARTIFACT_BYTES} bytes

What it means

Error "artifact exceeds {MAX_ARTIFACT_BYTES} bytes" thrown in unclecode/crawl4ai.

Source

Thrown at deploy/docker/artifacts.py:81

    try:
        with os.scandir(ARTIFACT_DIR) as it:
            for entry in it:
                try:
                    st = entry.stat(follow_symlinks=False)
                    if os.path.stat.S_ISREG(st.st_mode):
                        total += st.st_size
                except OSError:
                    continue
    except FileNotFoundError:
        return 0
    return total


def write_artifact(kind: str, data: bytes) -> Dict:
    """Write bytes under a server-generated opaque id. Returns metadata."""
    ext, mime = _KIND.get(kind, (".bin", "application/octet-stream"))
    if len(data) > MAX_ARTIFACT_BYTES:
        raise ArtifactTooLarge(f"artifact exceeds {MAX_ARTIFACT_BYTES} bytes")
    init_store()
    if _dir_size() + len(data) > ARTIFACT_QUOTA_BYTES:
        raise QuotaExceeded("artifact storage quota exceeded")

    artifact_id = uuid.uuid4().hex  # 32 hex chars, unguessable
    path = os.path.join(ARTIFACT_DIR, artifact_id + ext)
    # O_EXCL: never follow/overwrite an existing entry; O_NOFOLLOW: refuse a
    # symlink at the final component. Server-chosen name => no traversal.
    flags = os.O_WRONLY | os.O_CREAT | os.O_EXCL | getattr(os, "O_NOFOLLOW", 0)
    fd = os.open(path, flags, 0o600)
    with os.fdopen(fd, "wb") as f:
        f.write(data)
    return {"artifact_id": artifact_id, "mime": mime, "size": len(data)}


def resolve_artifact(artifact_id: str) -> Tuple[str, str]:
    """Map an opaque id to (path, mime), enforcing hex-id, no-symlink, TTL.

View on GitHub (pinned to 7e80152142)

Solutions

1. Reduce the size of the uploaded artifact below MAX_ARTIFACT_BYTES.
2. Compress or split large artifacts before upload.

Example fix

# Keep artifact <= MAX_ARTIFACT_BYTES

When it happens

Trigger: Thrown at deploy/docker/artifacts.py:81 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of unclecode/crawl4ai@7e80152142 (2026-08-14). Data as JSON: /api/errors/4d1949a1ddb7fe14. Report an issue: GitHub.