sgl-project/sglang · error · ValueError

{tensor_name}{context_clause} with shape {tensor.shape} cann

Error message

{tensor_name}{context_clause} with shape {tensor.shape} cannot be expanded to expected shape {expected_shape}.{hint_clause}

What it means

_expand_sparsity_tensor validates that a block-sparsity metadata tensor can be broadcast (expand) to the expected shape: every dim must match or be 1. Otherwise it raises ValueError with tensor name, context, shapes, and an optional hint.

Source

Thrown at python/sglang/kernels/ops/attention/flash_attn/cute/block_sparsity.py:225

def _expand_sparsity_tensor(
    tensor: torch.Tensor,
    expected_shape: Tuple[int, ...],
    tensor_name: str,
    context: str | None,
    hint: str | Callable[[], str] | None,
) -> torch.Tensor:
    """Check if we need to expand the tensor to expected shape, and do so if possible."""
    needs_expand = tensor.shape != expected_shape
    if not needs_expand:
        return tensor
    can_expand = all(
        map(lambda cur, tgt: cur == tgt or cur == 1, tensor.shape, expected_shape)
    )
    if not can_expand:
        context_clause = f" ({context})" if context else ""
        resolved_hint = hint() if callable(hint) else hint
        hint_clause = f" Hint: {resolved_hint}" if resolved_hint else ""
        raise ValueError(
            f"{tensor_name}{context_clause} with shape {tensor.shape} cannot be expanded to expected shape {expected_shape}."
            f"{hint_clause}"
        )
    return tensor.expand(*expected_shape)


def _check_and_expand_block(
    name: str,
    cnt: torch.Tensor | None,
    idx: torch.Tensor | None,
    expected_count_shape: Tuple[int, ...],
    expected_index_shape: Tuple[int, ...],
    context: str | None,
    hint: str | Callable[[], str] | None,
) -> Tuple[torch.Tensor | None, torch.Tensor | None]:
    if (cnt is None) != (idx is None):
        raise ValueError(
            f"{name}_block_cnt and {name}_block_idx must both be provided or both be None"

View on GitHub (pinned to 0132848349)

Solutions

1. Fix the metadata tensor shape so each dim equals the expected dim or is 1 (broadcastable)
2. Expand leading dims explicitly: tensor.expand(B, H, ...) before passing
3. Print expected_shape from the error and reshape/permute the metadata to match

Example fix

# before
cnt = torch.zeros(num_blocks, dtype=torch.int32, device='cuda')  # expected [B, H, ...]
# after
cnt = cnt.expand(B, H, *cnt.shape).contiguous()  # or build with full shape

def broadcastable(t, expected):\n    return all(c == e or c == 1 for c, e in zip(t.shape, expected))\nassert broadcastable(cnt, expected_cnt_shape) and broadcastable(idx, expected_idx_shape)

def is_expandable(tensor: torch.Tensor, expected_shape: tuple) -> bool:\n    return all(c == e or c == 1 for c, e in zip(tensor.shape, expected_shape))

When it happens

Trigger: Passing a *_block_cnt/*_block_idx tensor to normalize_block_sparse_tensors whose shape has a dim that neither equals the expected dim nor is 1 (e.g. batch of 1 when 8 heads expected, or swapped head/level dims).

Common situations: Configuring FlashAttention block-sparsity (e.g. MoBA/block-sparse attention) with per-request metadata that doesn't broadcast across batch or heads; wrong tensor layout from a sparsity planner.

Related errors

• {name}_block_cnt and {name}_block_idx must both be provided
• {name}_block tensors must have dtype torch.int32
• {name}_block_cnt and {name}_block_idx must be on the same de
• {name}_block tensors must live on CUDA
• Block sparse tensors{context} require BLOCK_SIZE_KV={base_n_

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/166f6d0120e8b808. Report an issue: GitHub.