{"record":{"id":"997ebe52a27599c8","repo":"xai-org/x-algorithm","slug":"name-must-be-on-the-same-device-as-block-sparse","errorCode":null,"errorMessage":"{name} must be on the same device as block sparse tensors","messagePattern":"(.+?) must be on the same device as block sparse tensors","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py","lineNumber":269,"sourceCode":"        idx, expected_index_shape, f\"{name}_block_idx\", context, hint\n    )\n    return expanded_cnt, expanded_idx\n\n\ndef _check_and_expand_metadata_tensor(\n    name: str,\n    tensor: torch.Tensor | None,\n    expected_shape: Tuple[int, ...],\n    context: str | None,\n    hint: str | Callable[[], str] | None,\n    device: torch.device,\n) -> torch.Tensor | None:\n    if tensor is None:\n        return None\n    if tensor.dtype != torch.int32:\n        raise ValueError(f\"{name} must have dtype torch.int32\")\n    if tensor.device != device:\n        raise ValueError(f\"{name} must be on the same device as block sparse tensors\")\n    if not tensor.is_cuda:\n        raise ValueError(f\"{name} must live on CUDA\")\n    return _expand_sparsity_tensor(tensor, expected_shape, name, context, hint)\n\n\ndef get_block_sparse_expected_shapes(\n    batch_size: int,\n    num_head: int,\n    seqlen_q: int,\n    seqlen_k: int,\n    m_block_size: int,\n    n_block_size: int,\n    q_stage: int,\n) -> Tuple[Tuple[int, int, int], Tuple[int, int, int, int]]:\n    m_block_size_effective = q_stage * m_block_size\n    expected_m_blocks = ceildiv(seqlen_q, m_block_size_effective)\n    expected_n_blocks = ceildiv(seqlen_k, n_block_size)\n    expected_count_shape = (batch_size, num_head, expected_m_blocks)","sourceCodeStart":251,"sourceCodeEnd":287,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py#L251-L287","documentation":"Raised by _check_and_expand_metadata_tensor when a metadata tensor's device differs from the device of the block-sparse tensors. Everything must be co-located on the same CUDA device for the kernel to dereference them safely.","triggerScenarios":"Block tensors on 'cuda:0' but the metadata tensor on CPU or on 'cuda:1'; typically after moving q/k/v with .to(model.device) but leaving metadata behind.","commonSituations":"Multi-GPU training with device_per_rank; heterogeneous data pipelines where metadata is computed on CPU; moving only some inputs in .to() calls.","solutions":["Move the metadata tensor to the block tensors' device: t = t.to(cnt.device)","Centralize a single device variable and use it for all inputs","Assert t.device == cnt.device before the call"],"exampleFix":"# before\nmeta = compute_meta().cuda(0)\nout = ranker(q.to('cuda:1'), ..., meta=meta)\n\n# after\ndev = q.device\nmeta = compute_meta().to(dev)\nout = ranker(q.to(dev), ..., meta=meta)","handlingStrategy":"validation","validationCode":"device = mask_block_cnt.device\nassert meta.device == device, f\"meta on {meta.device}, blocks on {device}\"","typeGuard":"def matches_device(t: torch.Tensor, dev: torch.device) -> bool:\n    return t.device == dev","tryCatchPattern":null,"preventionTips":["Derive one device from the primary tensors and normalize everything to it"],"tags":["pytorch","device-mismatch","block-sparsity"],"backgroundTag":"pytorch-device-mismatch","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}