{"record":{"id":"503f190bb254ae04","repo":"xai-org/x-algorithm","slug":"name-block-tensors-must-live-on-cuda","errorCode":null,"errorMessage":"{name}_block tensors must live on CUDA","messagePattern":"(.+?)_block tensors must live on CUDA","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py","lineNumber":244,"sourceCode":"    cnt: torch.Tensor | None,\n    idx: torch.Tensor | None,\n    expected_count_shape: Tuple[int, ...],\n    expected_index_shape: Tuple[int, ...],\n    context: str | None,\n    hint: str | Callable[[], str] | None,\n) -> Tuple[torch.Tensor | None, torch.Tensor | None]:\n    if (cnt is None) != (idx is None):\n        raise ValueError(\n            f\"{name}_block_cnt and {name}_block_idx must both be provided or both be None\"\n        )\n    if cnt is None or idx is None:\n        return None, None\n    if cnt.dtype != torch.int32 or idx.dtype != torch.int32:\n        raise ValueError(f\"{name}_block tensors must have dtype torch.int32\")\n    if cnt.device != idx.device:\n        raise ValueError(f\"{name}_block_cnt and {name}_block_idx must be on the same device\")\n    if not cnt.is_cuda or not idx.is_cuda:\n        raise ValueError(f\"{name}_block tensors must live on CUDA\")\n    expanded_cnt = _expand_sparsity_tensor(\n        cnt, expected_count_shape, f\"{name}_block_cnt\", context, hint\n    )\n    if idx.ndim == 4 and idx.shape[3] <= expected_index_shape[3]:\n        expected_index_shape = (*expected_index_shape[:3], idx.shape[3])\n    expanded_idx = _expand_sparsity_tensor(\n        idx, expected_index_shape, f\"{name}_block_idx\", context, hint\n    )\n    return expanded_cnt, expanded_idx\n\n\ndef _check_and_expand_metadata_tensor(\n    name: str,\n    tensor: torch.Tensor | None,\n    expected_shape: Tuple[int, ...],\n    context: str | None,\n    hint: str | Callable[[], str] | None,\n    device: torch.device,","sourceCodeStart":226,"sourceCodeEnd":262,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py#L226-L262","documentation":"Raised by _check_and_expand_block when either the block count or index tensor of a block-sparsity pair is not on CUDA. The downstream cute-DSL kernels are GPU-only, so CPU metadata is rejected before kernel launch.","triggerScenarios":"Passing mask_block_cnt and/or mask_block_idx that are CPU tensors (e.g. freshly created via torch.zeros on CPU or loaded from a file without .to('cuda')) to normalize_block_sparse_tensors / the ranker API.","commonSituations":"Prototyping on CPU then switching to the CUDA path without moving metadata tensors; loading pickled sparse schedules from disk; debugging scripts that build indices with numpy and forget the device transfer.","solutions":["Move both block tensors to CUDA: cnt = cnt.cuda(); idx = idx.cuda()","Verify a CUDA device is available before calling (torch.cuda.is_available())","Keep all block-sparsity metadata on the same GPU as q/k/v"],"exampleFix":"# before\nmask_block_cnt = torch.zeros(B, H, M, dtype=torch.int32)\n\n# after\nmask_block_cnt = torch.zeros(B, H, M, dtype=torch.int32, device='cuda')","handlingStrategy":"validation","validationCode":"assert cnt.is_cuda and idx.is_cuda, \"block tensors must be on CUDA\"","typeGuard":"def on_cuda(*ts) -> bool:\n    return all(t is None or t.is_cuda for t in ts)","tryCatchPattern":null,"preventionTips":["Move sparse metadata to GPU in the same place q/k/v are moved","Use device='cuda' at tensor creation time"],"tags":["pytorch","cuda","block-sparsity","cpu-tensor"],"backgroundTag":"pytorch-tensor-not-on-cuda","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}