{"record":{"id":"54cb51857c63fdf3","repo":"xai-org/x-algorithm","slug":"name-block-cnt-and-name-block-idx-must-be-on-t","errorCode":null,"errorMessage":"{name}_block_cnt and {name}_block_idx must be on the same device","messagePattern":"(.+?)_block_cnt and (.+?)_block_idx must be on the same device","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py","lineNumber":242,"sourceCode":"def _check_and_expand_block(\n    name: str,\n    cnt: torch.Tensor | None,\n    idx: torch.Tensor | None,\n    expected_count_shape: Tuple[int, ...],\n    expected_index_shape: Tuple[int, ...],\n    context: str | None,\n    hint: str | Callable[[], str] | None,\n) -> Tuple[torch.Tensor | None, torch.Tensor | None]:\n    if (cnt is None) != (idx is None):\n        raise ValueError(\n            f\"{name}_block_cnt and {name}_block_idx must both be provided or both be None\"\n        )\n    if cnt is None or idx is None:\n        return None, None\n    if cnt.dtype != torch.int32 or idx.dtype != torch.int32:\n        raise ValueError(f\"{name}_block tensors must have dtype torch.int32\")\n    if cnt.device != idx.device:\n        raise ValueError(f\"{name}_block_cnt and {name}_block_idx must be on the same device\")\n    if not cnt.is_cuda or not idx.is_cuda:\n        raise ValueError(f\"{name}_block tensors must live on CUDA\")\n    expanded_cnt = _expand_sparsity_tensor(\n        cnt, expected_count_shape, f\"{name}_block_cnt\", context, hint\n    )\n    if idx.ndim == 4 and idx.shape[3] <= expected_index_shape[3]:\n        expected_index_shape = (*expected_index_shape[:3], idx.shape[3])\n    expanded_idx = _expand_sparsity_tensor(\n        idx, expected_index_shape, f\"{name}_block_idx\", context, hint\n    )\n    return expanded_cnt, expanded_idx\n\n\ndef _check_and_expand_metadata_tensor(\n    name: str,\n    tensor: torch.Tensor | None,\n    expected_shape: Tuple[int, ...],\n    context: str | None,","sourceCodeStart":224,"sourceCodeEnd":260,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py#L224-L260","documentation":"Raised by _check_and_expand_block (called from normalize_block_sparse_tensors) when the block-sparsity count tensor and index tensor for the same named pair (e.g. mask_block_cnt / mask_block_idx) are on different torch devices. The library requires both tensors of a pair to be co-located because the CUDA kernels index them together.","triggerScenarios":"Calling the ranker/attention API with mask_block_cnt on 'cuda:0' but mask_block_idx on 'cpu' (or on a different GPU, e.g. 'cuda:1'). Also happens when one tensor is created with device=... and the other comes from a cached/preloaded tensor that was never moved.","commonSituations":"Loading sparse metadata from disk (numpy/CPU tensors) and only moving one of the two tensors to GPU; multi-GPU pipelines where tensors are pinned to different devices; mixing tensors produced by different pipeline stages.","solutions":["Move both tensors to the same device, e.g. cnt = cnt.to(idx.device) before calling the API","Ensure cnt.device == idx.device in a pre-flight assert","Check for accidental .cuda() on only one tensor in data loading code"],"exampleFix":"# before\nmask_block_cnt = torch.from_numpy(cnt_np).cuda()\nmask_block_idx = torch.from_numpy(idx_np)  # still on CPU\n\n# after\nmask_block_idx = torch.from_numpy(idx_np).to(mask_block_cnt.device)","handlingStrategy":"validation","validationCode":"assert cnt is None or idx is None or cnt.device == idx.device, f\"device mismatch: {cnt.device} vs {idx.device}\"","typeGuard":"def same_device(a: torch.Tensor, b: torch.Tensor) -> bool:\n    return a.device == b.device","tryCatchPattern":"try:\n    out = api(...)\nexcept ValueError as e:\n    if \"same device\" in str(e):\n        cnt = cnt.to(idx.device)\n        out = api(...)\n    else:\n        raise","preventionTips":["Keep a single `device = q.device` variable and .to(device) every sparse tensor","Assert pairwise device equality in your data loader"],"tags":["pytorch","device-mismatch","block-sparsity","validation"],"backgroundTag":"pytorch-device-mismatch","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}