{"record":{"id":"e95a22080694a9b0","repo":"sgl-project/sglang","slug":"name-must-be-on-the-same-device-as-block-sparse","errorCode":null,"errorMessage":"{name} must be on the same device as block sparse tensors","messagePattern":"(.+?) must be on the same device as block sparse tensors","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py","lineNumber":283,"sourceCode":"        idx, expected_index_shape, f\"{name}_block_idx\", context, hint\n    )\n    return expanded_cnt, expanded_idx\n\n\ndef _check_and_expand_metadata_tensor(\n    name: str,\n    tensor: torch.Tensor | None,\n    expected_shape: Tuple[int, ...],\n    context: str | None,\n    hint: str | Callable[[], str] | None,\n    device: torch.device,\n) -> torch.Tensor | None:\n    if tensor is None:\n        return None\n    if tensor.dtype != torch.int32:\n        raise ValueError(f\"{name} must have dtype torch.int32\")\n    if tensor.device != device:\n        raise ValueError(f\"{name} must be on the same device as block sparse tensors\")\n    if not tensor.is_cuda:\n        raise ValueError(f\"{name} must live on CUDA\")\n    return _expand_sparsity_tensor(tensor, expected_shape, name, context, hint)\n\n\ndef get_block_sparse_expected_shapes(\n    batch_size: int,\n    num_head: int,\n    seqlen_q: int,\n    seqlen_k: int,\n    m_block_size: int,\n    n_block_size: int,\n    q_stage: int,\n) -> Tuple[Tuple[int, int, int], Tuple[int, int, int, int]]:\n    \"\"\"Return (expected_count_shape, expected_index_shape) for block sparse normalization.\"\"\"\n    m_block_size_effective = q_stage * m_block_size\n    expected_m_blocks = ceildiv(seqlen_q, m_block_size_effective)\n    expected_n_blocks = ceildiv(seqlen_k, n_block_size)","sourceCodeStart":265,"sourceCodeEnd":301,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py#L265-L301","documentation":"Raised when an auxiliary metadata tensor (e.g. dq_write_order) lives on a different device than the mask block count tensor. All block-sparse tensors must be co-located because the kernel treats them as one bundle.","triggerScenarios":"dq_write_order on cuda:1 (or CPU) while mask_block_cnt is on cuda:0 when calling normalize_block_sparse_tensors.","commonSituations":"Multi-GPU (TP/EP) runs where metadata was generated on a different rank's device, or CPU-cached metadata reused after moving the mask tensors.","solutions":["Move metadata to mask_cnt.device: dq_write_order = dq_write_order.to(mask_cnt.device)","Standardize all tensors to torch.cuda.current_device() before building BlockSparseTensorsTorch"],"exampleFix":"// before\norder = order.to('cuda:0'); mask = mask.to('cuda:1')\n// after\norder = order.to(mask_cnt.device)","handlingStrategy":"validation","validationCode":"if dq_write_order is not None:\n    dq_write_order = dq_write_order.to(mask_block_cnt.device)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Normalize every tensor to mask_cnt.device in one place","In TP runs, use torch.cuda.current_device() consistently"],"tags":["block-sparse","device-mismatch","metadata"],"backgroundTag":"tensor-device-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}