{"record":{"id":"688e48ae49610eaa","repo":"sgl-project/sglang","slug":"name-block-tensors-must-have-dtype-torch-int32","errorCode":null,"errorMessage":"{name}_block tensors must have dtype torch.int32","messagePattern":"(.+?)_block tensors must have dtype torch\\.int32","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py","lineNumber":248,"sourceCode":"\n\ndef _check_and_expand_block(\n    name: str,\n    cnt: torch.Tensor | None,\n    idx: torch.Tensor | None,\n    expected_count_shape: Tuple[int, ...],\n    expected_index_shape: Tuple[int, ...],\n    context: str | None,\n    hint: str | Callable[[], str] | None,\n) -> Tuple[torch.Tensor | None, torch.Tensor | None]:\n    if (cnt is None) != (idx is None):\n        raise ValueError(\n            f\"{name}_block_cnt and {name}_block_idx must both be provided or both be None\"\n        )\n    if cnt is None or idx is None:\n        return None, None\n    if cnt.dtype != torch.int32 or idx.dtype != torch.int32:\n        raise ValueError(f\"{name}_block tensors must have dtype torch.int32\")\n    if cnt.device != idx.device:\n        raise ValueError(\n            f\"{name}_block_cnt and {name}_block_idx must be on the same device\"\n        )\n    if not cnt.is_cuda or not idx.is_cuda:\n        raise ValueError(f\"{name}_block tensors must live on CUDA\")\n    expanded_cnt = _expand_sparsity_tensor(\n        cnt, expected_count_shape, f\"{name}_block_cnt\", context, hint\n    )\n    # [Note] Allow Compact block sparse indices\n    # Allow the last dimension (n_blocks) of idx to be <= expected, since\n    # FA4 only accesses indices 0..cnt-1 per query tile. This enables compact\n    # index tensors that avoid O(N^2) memory at long sequence lengths.\n    if idx.ndim == 4 and idx.shape[3] <= expected_index_shape[3]:\n        expected_index_shape = (*expected_index_shape[:3], idx.shape[3])\n    expanded_idx = _expand_sparsity_tensor(\n        idx, expected_index_shape, f\"{name}_block_idx\", context, hint\n    )","sourceCodeStart":230,"sourceCodeEnd":266,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py#L230-L266","documentation":"Block-sparsity *_block_cnt/*_block_idx tensors must be torch.int32. Any other dtype (int64 from default arange/range ops, int16, etc.) is rejected with ValueError.","triggerScenarios":"Passing metadata tensors created as torch.long (default for arange/tensor(list)) or other dtypes to normalize_block_sparse_tensors.","commonSituations":"Metadata built with torch.arange(...) (defaults to int64) or from numpy int64 arrays converted to torch; porting from kernels that accept int64.","solutions":["Cast both tensors: cnt.to(torch.int32), idx.to(torch.int32)","Create with explicit dtype: torch.arange(..., dtype=torch.int32, device='cuda')"],"exampleFix":"# before\nidx = torch.arange(nblocks)  # int64\n# after\nidx = torch.arange(nblocks, dtype=torch.int32, device='cuda')","handlingStrategy":"validation","validationCode":"if cnt is not None:\\n    cnt = cnt.to(torch.int32); idx = idx.to(torch.int32)\\nassert cnt is None or (cnt.is_cuda and idx.is_cuda and cnt.device == idx.device)","typeGuard":"def valid_block_dtype(t: torch.Tensor) -> bool:\\n    return t.dtype == torch.int32","tryCatchPattern":null,"preventionTips":["Create metadata with dtype=torch.int32, device='cuda'","Wrap metadata construction in one helper that enforces dtype/device"],"tags":["block-sparse","attention","dtype","int32-required"],"backgroundTag":"tensor-dtype-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}