sgl-project/sglang · error · RuntimeError

cuMemGetAddressRange: {err}

Error message

cuMemGetAddressRange: {err}

What it means

While chunking a graph capture input across VMM allocations, cuMemGetAddressRange failed for the current address. This driver query returns the base and size of the allocation containing an address; failure typically means the address is not a valid device pointer in the current context (freed, host memory, or from another context).

Source

Thrown at python/sglang/srt/utils/cuda_vmm_utils.py:129

      - ``input_chunk_indices[j]`` = indices of allocations covering input j
      - ``input_offsets[j]`` = byte offset of input j from its first base
    """
    drv = _get_cuda_driver()
    base_to_idx = {}
    bases_info: List[tuple] = []
    input_chunk_indices: List[List[int]] = []
    input_offsets: List[int] = []
    for ptr, nbytes in graph_inputs:
        ptr, remaining = int(ptr), int(nbytes)
        if remaining <= 0:
            raise RuntimeError(f"Invalid graph capture input size: {nbytes}")
        cursor = ptr
        first_base = None
        chunks: List[int] = []
        while remaining > 0:
            err, base, size = drv.cuMemGetAddressRange(cursor)
            if err != drv.CUresult.CUDA_SUCCESS:
                raise RuntimeError(f"cuMemGetAddressRange: {err}")
            base, size = int(base), int(size)
            if first_base is None:
                first_base = base
            byte_offset = cursor - base
            if not 0 <= byte_offset < size:
                raise RuntimeError(
                    f"graph capture input at {ptr} is outside VMM allocation "
                    f"[base={base}, size={size}]"
                )
            idx = base_to_idx.setdefault(base, len(bases_info))
            if idx == len(bases_info):
                bases_info.append((base, size))
            chunks.append(idx)
            advance = min(remaining, size - byte_offset)
            assert advance > 0, "Failed to advance VMM graph capture span"
            remaining -= advance
            cursor += advance
        input_chunk_indices.append(chunks)

View on GitHub (pinned to 0132848349)

Solutions

1. Verify every ptr in graph_inputs is a live device allocation (check tensor validity at build time)
2. Rebuild capture inputs at capture time, not from cached pointers from a prior capture
3. For non-device buffers, route them outside the VMM capture-base mechanism

err, base, size = drv.cuMemGetAddress_range(ptr)
assert err == drv.CUresult.CUDA_SUCCESS  # ptr is a valid device address

def is_live_device_pointer(ptr: int) -> bool:
    drv = _get_cuda_driver()
    err, _, _ = drv.cuMemGetAddressRange(ptr)
    return err == drv.CUresult.CUDA_SUCCESS

When it happens

Trigger: Passing a host/CPU pointer or an already-freed tensor's data_ptr in graph_inputs; tensors allocated in a different CUDA context; stale pointers captured from a previous run.

Common situations: Graph capture lists built after tensors were freed; mixing pinned-host buffers into VMM capture inputs; multi-context bugs in multiprocess inference.

Related errors

• Invalid graph capture input size: {nbytes}
• graph capture input at {ptr} is outside VMM allocation [base
• CUDA error {int(result[0])}({_cudaGetErrorString(result[0])}
• CUDA VMM POSIX FD broker failed
• CUDA VMM POSIX FD broker returned no file descriptor

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e06f7f167c5aaeec. Report an issue: GitHub.