sgl-project/sglang · error · ValueError

{len(extents)} extents exceed BUMPARENA_MAX_EXTENTS ({BumpAr

Error message

{len(extents)} extents exceed BUMPARENA_MAX_EXTENTS ({BumpArenaStub.MAX_EXTENTS})

What it means

BumpArenaStub.set_extents raises this when the number of (base, nbytes) extents passed in exceeds BumpArenaStub.MAX_EXTENTS, a fixed capacity limit for the bump-arena's extent array. The stub keeps a bounded ctypes array of extents for first-fit allocation, so more extents than MAX_EXTENTS cannot be registered.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_utils.py:477

            ctypes.c_size_t,
        ]
        self._fn_set_extents.restype = None
        self._fn_set_align = lib[f"bumparena_set_align_{self.sfx}"]
        self._fn_set_align.argtypes = [ctypes.c_size_t]
        self._fn_set_align.restype = None
        self._fn_cursor = lib[f"bumparena_cursor_{self.sfx}"]
        self._fn_cursor.argtypes = []
        self._fn_cursor.restype = ctypes.c_size_t
        self._fn_freed = lib[f"bumparena_freed_{self.sfx}"]
        self._fn_freed.argtypes = []
        self._fn_freed.restype = ctypes.c_size_t
        return lib

    def set_extents(self, extents: List[tuple]) -> None:
        """Register ``(base, nbytes)`` extents (first-fit order) and reset
        every bump cursor."""
        if len(extents) > BumpArenaStub.MAX_EXTENTS:
            raise ValueError(
                f"{len(extents)} extents exceed BUMPARENA_MAX_EXTENTS "
                f"({BumpArenaStub.MAX_EXTENTS})"
            )
        n = len(extents)
        bases = (ctypes.c_void_p * n)(*(base for base, _ in extents))
        sizes = (ctypes.c_size_t * n)(*(nbytes for _, nbytes in extents))
        self._fn_set_extents(bases, sizes, ctypes.c_size_t(n))

    def set_align(self, nbytes: int) -> None:
        self._fn_set_align(ctypes.c_size_t(nbytes))

    @property
    def cursor_bytes(self) -> int:
        return int(self._fn_cursor())

    @property
    def freed_bytes(self) -> int:
        """Bytes handed back through ``free`` since the last ``set_extents``

View on GitHub (pinned to 0132848349)

Solutions

1. Reduce the number of extents by coalescing contiguous/adjacent reservations into fewer larger extents before calling set_extents.
2. Increase BUMPARENA_MAX_EXTENTS (env/config) if the workload legitimately needs more segments.
3. Check for memory fragmentation sources (many small cudaMalloc / graph pool re-borrows) and consolidate pool allocations.

Example fix

// before
arena.set_extents([(b, s) for b, s in raw_extents])

// after
raw_extents.sort()
merged = []
for base, size in raw_extents:
    if merged and merged[-1][0] + merged[-1][1] == base:
        merged[-1] = (merged[-1][0], merged[-1][1] + size)
    else:
        merged.append((base, size))
arena.set_extents(merged)

from python.sglang.srt.utils.cuda_vmm_utils import BumpArenaStub
merged = coalesce_contiguous(extents)
assert len(merged) <= BumpArenaStub.MAX_EXTENTS, f"{len(merged)} > {BumpArenaStub.MAX_EXTENTS}; raise BUMPARENA_MAX_EXTENTS"

try:
    arena.set_extents(extents)
except ValueError as e:
    if "BUMPARENA_MAX_EXTENTS" in str(e):
        arena.set_extents(coalesce_contiguous(extents))
    else:
        raise

When it happens

Trigger: Calling set_extents (directly or via __init__ / borrow_graph_pool) with a list of extents longer than BumpArenaStub.MAX_EXTENTS (controlled by the BUMPARENA_MAX_EXTENTS constant).

Common situations: Fragmented allocations (many small memaps/graph pool segments) after a change in graph capture batch sizes, mem-fraction, or TP size; or overriding BUMPARENA_MAX_EXTENTS to a smaller value than the workload produces.

Related errors

• CUDA VMM POSIX FD broker failed
• CUDA VMM POSIX FD broker returned no file descriptor
• memory_size must be positive
• consumer_count must be positive
• recycle_interval must be positive

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/67e15f0f82c5fad1. Report an issue: GitHub.