{"record":{"id":"e06f7f167c5aaeec","repo":"sgl-project/sglang","slug":"cumemgetaddressrange-err","errorCode":null,"errorMessage":"cuMemGetAddressRange: {err}","messagePattern":"cuMemGetAddressRange: (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/cuda_vmm_utils.py","lineNumber":129,"sourceCode":"      - ``input_chunk_indices[j]`` = indices of allocations covering input j\n      - ``input_offsets[j]`` = byte offset of input j from its first base\n    \"\"\"\n    drv = _get_cuda_driver()\n    base_to_idx = {}\n    bases_info: List[tuple] = []\n    input_chunk_indices: List[List[int]] = []\n    input_offsets: List[int] = []\n    for ptr, nbytes in graph_inputs:\n        ptr, remaining = int(ptr), int(nbytes)\n        if remaining <= 0:\n            raise RuntimeError(f\"Invalid graph capture input size: {nbytes}\")\n        cursor = ptr\n        first_base = None\n        chunks: List[int] = []\n        while remaining > 0:\n            err, base, size = drv.cuMemGetAddressRange(cursor)\n            if err != drv.CUresult.CUDA_SUCCESS:\n                raise RuntimeError(f\"cuMemGetAddressRange: {err}\")\n            base, size = int(base), int(size)\n            if first_base is None:\n                first_base = base\n            byte_offset = cursor - base\n            if not 0 <= byte_offset < size:\n                raise RuntimeError(\n                    f\"graph capture input at {ptr} is outside VMM allocation \"\n                    f\"[base={base}, size={size}]\"\n                )\n            idx = base_to_idx.setdefault(base, len(bases_info))\n            if idx == len(bases_info):\n                bases_info.append((base, size))\n            chunks.append(idx)\n            advance = min(remaining, size - byte_offset)\n            assert advance > 0, \"Failed to advance VMM graph capture span\"\n            remaining -= advance\n            cursor += advance\n        input_chunk_indices.append(chunks)","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/cuda_vmm_utils.py#L111-L147","documentation":"While chunking a graph capture input across VMM allocations, cuMemGetAddressRange failed for the current address. This driver query returns the base and size of the allocation containing an address; failure typically means the address is not a valid device pointer in the current context (freed, host memory, or from another context).","triggerScenarios":"Passing a host/CPU pointer or an already-freed tensor's data_ptr in graph_inputs; tensors allocated in a different CUDA context; stale pointers captured from a previous run.","commonSituations":"Graph capture lists built after tensors were freed; mixing pinned-host buffers into VMM capture inputs; multi-context bugs in multiprocess inference.","solutions":["Verify every ptr in graph_inputs is a live device allocation (check tensor validity at build time)","Rebuild capture inputs at capture time, not from cached pointers from a prior capture","For non-device buffers, route them outside the VMM capture-base mechanism"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"err, base, size = drv.cuMemGetAddress_range(ptr)\nassert err == drv.CUresult.CUDA_SUCCESS  # ptr is a valid device address","typeGuard":"def is_live_device_pointer(ptr: int) -> bool:\n    drv = _get_cuda_driver()\n    err, _, _ = drv.cuMemGetAddressRange(ptr)\n    return err == drv.CUresult.CUDA_SUCCESS","tryCatchPattern":null,"preventionTips":["Build capture input lists from live tensors at capture time","Don't cache raw pointers across allocation lifetimes"],"tags":["cuda","vmm","cuda-graph","invalid-pointer"],"backgroundTag":"invalid-device-pointer","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}