sgl-project/sglang · error · RuntimeError

graph capture input at {ptr} is outside VMM allocation [base

Error message

graph capture input at {ptr} is outside VMM allocation [base={base}, size={size}]

What it means

After querying an allocation's base/size, the code verifies the cursor address actually lies within that allocation range; if the computed byte offset is outside [0, size) the pointer is outside the VMM allocation and the invariant is broken. This catches inconsistent address-range results or corrupted pointer arithmetic before chunk indices are built.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_utils.py:135

    input_chunk_indices: List[List[int]] = []
    input_offsets: List[int] = []
    for ptr, nbytes in graph_inputs:
        ptr, remaining = int(ptr), int(nbytes)
        if remaining <= 0:
            raise RuntimeError(f"Invalid graph capture input size: {nbytes}")
        cursor = ptr
        first_base = None
        chunks: List[int] = []
        while remaining > 0:
            err, base, size = drv.cuMemGetAddressRange(cursor)
            if err != drv.CUresult.CUDA_SUCCESS:
                raise RuntimeError(f"cuMemGetAddressRange: {err}")
            base, size = int(base), int(size)
            if first_base is None:
                first_base = base
            byte_offset = cursor - base
            if not 0 <= byte_offset < size:
                raise RuntimeError(
                    f"graph capture input at {ptr} is outside VMM allocation "
                    f"[base={base}, size={size}]"
                )
            idx = base_to_idx.setdefault(base, len(bases_info))
            if idx == len(bases_info):
                bases_info.append((base, size))
            chunks.append(idx)
            advance = min(remaining, size - byte_offset)
            assert advance > 0, "Failed to advance VMM graph capture span"
            remaining -= advance
            cursor += advance
        input_chunk_indices.append(chunks)
        input_offsets.append(ptr - first_base)
    return bases_info, input_chunk_indices, input_offsets


def make_rw_access_desc(device_id: int):
    """A read-write, device-local ``CUmemAccessDesc`` for ``device_id``."""

View on GitHub (pinned to 0132848349)

Solutions

Recompute the graph input list from live tensors and retry capture
Ensure inputs are allocated from the VMM pool (not foreign allocations) when using capture bases
Check for 32-bit/64-bit pointer truncation in code building the ptr/nbytes pairs

Defensive patterns

Strategy: validation

Validate before calling

err, base, size = drv.cuMemGetAddressRange(ptr)
if not (base <= ptr < base + size):
    raise ValueError("pointer outside its VMM allocation")

Type guard

def pointer_in_allocation(ptr: int, base: int, size: int) -> bool:
    return base <= ptr < base + size

Prevention

Allocate capture inputs from the VMM pool
Recompute capture bases after any reallocation

When it happens

Trigger: A graph input pointer that doesn't fall inside any cuMem allocation returned by the driver; integer/pointer arithmetic bugs producing a cursor past the region end; non-contiguous spanning inputs where the walk goes past the last chunk.

Common situations: Corrupted capture input lists; buffers straddling allocations freed mid-walk; pointer truncation on 32-bit builds.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/de9a5ff2192cedcc. Report an issue: GitHub.