sgl-project/sglang · error · RuntimeError

Invalid graph capture input size: {nbytes}

Error message

Invalid graph capture input size: {nbytes}

What it means

compute_graph_capture_bases validates each graph capture input buffer: its byte size must be positive. A zero or negative nbytes means the caller passed a degenerate/empty buffer description, and address-range chunking over it is meaningless, so it fails fast.

Source

Thrown at python/sglang/srt/utils/cuda_vmm_utils.py:122

    ``graph_inputs`` is a list of ``(device_ptr, nbytes)`` pairs. A captured
    tensor can cross expandable-segment allocation boundaries, so each input
    is walked with ``cuMemGetAddressRange`` until its byte span is covered.

    Returns ``(bases_info, input_chunk_indices, input_offsets)``:
      - ``bases_info[i] = (base_ptr, alloc_size)`` per unique allocation
      - ``input_chunk_indices[j]`` = indices of allocations covering input j
      - ``input_offsets[j]`` = byte offset of input j from its first base
    """
    drv = _get_cuda_driver()
    base_to_idx = {}
    bases_info: List[tuple] = []
    input_chunk_indices: List[List[int]] = []
    input_offsets: List[int] = []
    for ptr, nbytes in graph_inputs:
        ptr, remaining = int(ptr), int(nbytes)
        if remaining <= 0:
            raise RuntimeError(f"Invalid graph capture input size: {nbytes}")
        cursor = ptr
        first_base = None
        chunks: List[int] = []
        while remaining > 0:
            err, base, size = drv.cuMemGetAddressRange(cursor)
            if err != drv.CUresult.CUDA_SUCCESS:
                raise RuntimeError(f"cuMemGetAddressRange: {err}")
            base, size = int(base), int(size)
            if first_base is None:
                first_base = base
            byte_offset = cursor - base
            if not 0 <= byte_offset < size:
                raise RuntimeError(
                    f"graph capture input at {ptr} is outside VMM allocation "
                    f"[base={base}, size={size}]"
                )
            idx = base_to_idx.setdefault(base, len(bases_info))
            if idx == len(bases_info):

View on GitHub (pinned to 0132848349)

Solutions

1. Filter out empty tensors before building the graph_inputs list
2. Fix upstream sizing logic so capture inputs always have positive nbytes
3. During warmup, use a real (non-zero-length) dummy batch for capture

Example fix

# before
graph_inputs = [(t.data_ptr(), t.nbytes) for t in tensors]

# after
graph_inputs = [(t.data_ptr(), t.nbytes) for t in tensors if t.numel() > 0]

graph_inputs = [(p, n) for p, n in graph_inputs if int(n) > 0]

def valid_capture_inputs(inputs) -> bool:
    return all(int(n) > 0 for _, n in inputs)

When it happens

Trigger: Passing a graph input tensor with numel 0 (e.g. empty batch slot) to get_graph_capture_bases; a bug producing tensor.nbytes == 0; negative sizes from arithmetic underflow.

Common situations: CUDA graph capture paths with empty dummy inputs; capture rehearsal during warmup with zero-length sequences; incorrect pointer/size bookkeeping in the capture list.

Related errors

• memory_size must be positive
• consumer_count must be positive
• recycle_interval must be positive
• consumer_count must be 1, the attention TP size, or the full
• CUDA VMM feature transport requires each feature field to co

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/e0bb54a9a96bdac9. Report an issue: GitHub.