{"record":{"id":"a232bac1d99d90ab","repo":"sgl-project/sglang","slug":"name-must-live-on-cuda","errorCode":null,"errorMessage":"{name} must live on CUDA","messagePattern":"(.+?) must live on CUDA","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py","lineNumber":285,"sourceCode":"    return expanded_cnt, expanded_idx\n\n\ndef _check_and_expand_metadata_tensor(\n    name: str,\n    tensor: torch.Tensor | None,\n    expected_shape: Tuple[int, ...],\n    context: str | None,\n    hint: str | Callable[[], str] | None,\n    device: torch.device,\n) -> torch.Tensor | None:\n    if tensor is None:\n        return None\n    if tensor.dtype != torch.int32:\n        raise ValueError(f\"{name} must have dtype torch.int32\")\n    if tensor.device != device:\n        raise ValueError(f\"{name} must be on the same device as block sparse tensors\")\n    if not tensor.is_cuda:\n        raise ValueError(f\"{name} must live on CUDA\")\n    return _expand_sparsity_tensor(tensor, expected_shape, name, context, hint)\n\n\ndef get_block_sparse_expected_shapes(\n    batch_size: int,\n    num_head: int,\n    seqlen_q: int,\n    seqlen_k: int,\n    m_block_size: int,\n    n_block_size: int,\n    q_stage: int,\n) -> Tuple[Tuple[int, int, int], Tuple[int, int, int, int]]:\n    \"\"\"Return (expected_count_shape, expected_index_shape) for block sparse normalization.\"\"\"\n    m_block_size_effective = q_stage * m_block_size\n    expected_m_blocks = ceildiv(seqlen_q, m_block_size_effective)\n    expected_n_blocks = ceildiv(seqlen_k, n_block_size)\n    expected_count_shape = (batch_size, num_head, expected_m_blocks)\n    expected_index_shape = (batch_size, num_head, expected_m_blocks, expected_n_blocks)","sourceCodeStart":267,"sourceCodeEnd":303,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py#L267-L303","documentation":"Raised when an auxiliary metadata tensor (e.g. dq_write_order) is not on CUDA while the rest of the block-sparse bundle is. The kernel reads it from GPU memory.","triggerScenarios":"Passing a CPU dq_write_order tensor (e.g. from torch.arange on CPU) into normalize_block_sparse_tensors.","commonSituations":"Generating dq_write_order during CPU-side mask preparation and forgetting the .cuda() transfer.","solutions":["dq_write_order = dq_write_order.cuda()","Create it directly on the target CUDA device with device='cuda'"],"exampleFix":"// before\norder = torch.arange(M, dtype=torch.int32)\n// after\norder = torch.arange(M, dtype=torch.int32, device='cuda')","handlingStrategy":"validation","validationCode":"assert dq_write_order is None or dq_write_order.is_cuda","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never build metadata on CPU then forget the transfer","Add an is_cuda assert in test fixtures"],"tags":["block-sparse","cuda","metadata","cpu-tensor"],"backgroundTag":"tensor-not-on-gpu","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}