{"record":{"id":"e0bb54a9a96bdac9","repo":"sgl-project/sglang","slug":"invalid-graph-capture-input-size-nbytes","errorCode":null,"errorMessage":"Invalid graph capture input size: {nbytes}","messagePattern":"Invalid graph capture input size: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/cuda_vmm_utils.py","lineNumber":122,"sourceCode":"\n    ``graph_inputs`` is a list of ``(device_ptr, nbytes)`` pairs. A captured\n    tensor can cross expandable-segment allocation boundaries, so each input\n    is walked with ``cuMemGetAddressRange`` until its byte span is covered.\n\n    Returns ``(bases_info, input_chunk_indices, input_offsets)``:\n      - ``bases_info[i] = (base_ptr, alloc_size)`` per unique allocation\n      - ``input_chunk_indices[j]`` = indices of allocations covering input j\n      - ``input_offsets[j]`` = byte offset of input j from its first base\n    \"\"\"\n    drv = _get_cuda_driver()\n    base_to_idx = {}\n    bases_info: List[tuple] = []\n    input_chunk_indices: List[List[int]] = []\n    input_offsets: List[int] = []\n    for ptr, nbytes in graph_inputs:\n        ptr, remaining = int(ptr), int(nbytes)\n        if remaining <= 0:\n            raise RuntimeError(f\"Invalid graph capture input size: {nbytes}\")\n        cursor = ptr\n        first_base = None\n        chunks: List[int] = []\n        while remaining > 0:\n            err, base, size = drv.cuMemGetAddressRange(cursor)\n            if err != drv.CUresult.CUDA_SUCCESS:\n                raise RuntimeError(f\"cuMemGetAddressRange: {err}\")\n            base, size = int(base), int(size)\n            if first_base is None:\n                first_base = base\n            byte_offset = cursor - base\n            if not 0 <= byte_offset < size:\n                raise RuntimeError(\n                    f\"graph capture input at {ptr} is outside VMM allocation \"\n                    f\"[base={base}, size={size}]\"\n                )\n            idx = base_to_idx.setdefault(base, len(bases_info))\n            if idx == len(bases_info):","sourceCodeStart":104,"sourceCodeEnd":140,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/cuda_vmm_utils.py#L104-L140","documentation":"compute_graph_capture_bases validates each graph capture input buffer: its byte size must be positive. A zero or negative nbytes means the caller passed a degenerate/empty buffer description, and address-range chunking over it is meaningless, so it fails fast.","triggerScenarios":"Passing a graph input tensor with numel 0 (e.g. empty batch slot) to get_graph_capture_bases; a bug producing tensor.nbytes == 0; negative sizes from arithmetic underflow.","commonSituations":"CUDA graph capture paths with empty dummy inputs; capture rehearsal during warmup with zero-length sequences; incorrect pointer/size bookkeeping in the capture list.","solutions":["Filter out empty tensors before building the graph_inputs list","Fix upstream sizing logic so capture inputs always have positive nbytes","During warmup, use a real (non-zero-length) dummy batch for capture"],"exampleFix":"# before\ngraph_inputs = [(t.data_ptr(), t.nbytes) for t in tensors]\n\n# after\ngraph_inputs = [(t.data_ptr(), t.nbytes) for t in tensors if t.numel() > 0]","handlingStrategy":"validation","validationCode":"graph_inputs = [(p, n) for p, n in graph_inputs if int(n) > 0]","typeGuard":"def valid_capture_inputs(inputs) -> bool:\n    return all(int(n) > 0 for _, n in inputs)","tryCatchPattern":null,"preventionTips":["Filter empty tensors before capture","Use non-zero dummy batches for capture warmup"],"tags":["cuda","vmm","cuda-graph","validation"],"backgroundTag":"invalid-buffer-size","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}