{"record":{"id":"ea6374dcb921f6a1","repo":"xai-org/x-algorithm","slug":"name-must-live-on-cuda","errorCode":null,"errorMessage":"{name} must live on CUDA","messagePattern":"(.+?) must live on CUDA","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py","lineNumber":271,"sourceCode":"    return expanded_cnt, expanded_idx\n\n\ndef _check_and_expand_metadata_tensor(\n    name: str,\n    tensor: torch.Tensor | None,\n    expected_shape: Tuple[int, ...],\n    context: str | None,\n    hint: str | Callable[[], str] | None,\n    device: torch.device,\n) -> torch.Tensor | None:\n    if tensor is None:\n        return None\n    if tensor.dtype != torch.int32:\n        raise ValueError(f\"{name} must have dtype torch.int32\")\n    if tensor.device != device:\n        raise ValueError(f\"{name} must be on the same device as block sparse tensors\")\n    if not tensor.is_cuda:\n        raise ValueError(f\"{name} must live on CUDA\")\n    return _expand_sparsity_tensor(tensor, expected_shape, name, context, hint)\n\n\ndef get_block_sparse_expected_shapes(\n    batch_size: int,\n    num_head: int,\n    seqlen_q: int,\n    seqlen_k: int,\n    m_block_size: int,\n    n_block_size: int,\n    q_stage: int,\n) -> Tuple[Tuple[int, int, int], Tuple[int, int, int, int]]:\n    m_block_size_effective = q_stage * m_block_size\n    expected_m_blocks = ceildiv(seqlen_q, m_block_size_effective)\n    expected_n_blocks = ceildiv(seqlen_k, n_block_size)\n    expected_count_shape = (batch_size, num_head, expected_m_blocks)\n    expected_index_shape = (batch_size, num_head, expected_m_blocks, expected_n_blocks)\n    return expected_count_shape, expected_index_shape","sourceCodeStart":253,"sourceCodeEnd":289,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py#L253-L289","documentation":"Raised by _check_and_expand_metadata_tensor when a metadata tensor passed the device check equality but is still not on CUDA — practically hit when the shared device itself is CPU (all tensors are co-located but on CPU). The kernels are CUDA-only.","triggerScenarios":"Running the whole pipeline on CPU: block tensors and metadata all on 'cpu', so tensor.device == device passes, but tensor.is_cuda is False.","commonSituations":"CI or unit tests without GPUs; environments where CUDA_VISIBLE_DEVICES is empty; falling back to CPU execution for a CUDA-only library.","solutions":["Move the whole workload (and all tensors) to a CUDA device","If no GPU is available, use a non-block-sparse or CPU-compatible code path instead","Check torch.cuda.is_available() before entering the block-sparse path"],"exampleFix":"# before\ndevice = torch.device('cpu')\ncnt = cnt.to(device); idx = idx.to(device); meta = meta.to(device)\n\n# after\ndevice = torch.device('cuda')\ncnt = cnt.to(device); idx = idx.to(device); meta = meta.to(device)","handlingStrategy":"validation","validationCode":"assert torch.cuda.is_available() and meta.is_cuda and mask_block_cnt.is_cuda","typeGuard":"def cuda_pipeline_ready(*ts) -> bool:\n    return torch.cuda.is_available() and all(t is None or t.is_cuda for t in ts)","tryCatchPattern":null,"preventionTips":["Gate the block-sparse path behind torch.cuda.is_available()","Skip CI runs without GPU or mock with a CPU-compatible path"],"tags":["pytorch","cuda","block-sparsity","cpu-tensor"],"backgroundTag":"pytorch-tensor-not-on-cuda","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}