xai-org/x-algorithm · error · ValueError

{tensor_name}{context_clause} with shape {tensor.shape} cann

Error message

{tensor_name}{context_clause} with shape {tensor.shape} cannot be expanded to expected shape {expected_shape}.{hint_clause}

What it means

Sparsity metadata tensors may be broadcast-expanded to the expected full shape only when each dimension either already matches or is 1. _expand_sparsity_tensor checks this and raises with the tensor name, optional context, shapes, and an optional hint when expansion is impossible (e.g. a dim of 3 where 4 is expected).

Source

Thrown at phoenix/xrex/cutedsl/ranker_fa4/block_sparsity.py:217

    return min_block_size


def _expand_sparsity_tensor(
    tensor: torch.Tensor,
    expected_shape: Tuple[int, ...],
    tensor_name: str,
    context: str | None,
    hint: str | Callable[[], str] | None,
) -> torch.Tensor:
    needs_expand = tensor.shape != expected_shape
    if not needs_expand:
        return tensor
    can_expand = all(map(lambda cur, tgt: cur == tgt or cur == 1, tensor.shape, expected_shape))
    if not can_expand:
        context_clause = f" ({context})" if context else ""
        resolved_hint = hint() if callable(hint) else hint
        hint_clause = f" Hint: {resolved_hint}" if resolved_hint else ""
        raise ValueError(
            f"{tensor_name}{context_clause} with shape {tensor.shape} cannot be expanded to expected shape {expected_shape}."
            f"{hint_clause}"
        )
    return tensor.expand(*expected_shape)


def _check_and_expand_block(
    name: str,
    cnt: torch.Tensor | None,
    idx: torch.Tensor | None,
    expected_count_shape: Tuple[int, ...],
    expected_index_shape: Tuple[int, ...],
    context: str | None,
    hint: str | Callable[[], str] | None,
) -> Tuple[torch.Tensor | None, torch.Tensor | None]:
    if (cnt is None) != (idx is None):
        raise ValueError(
            f"{name}_block_cnt and {name}_block_idx must both be provided or both be None"

View on GitHub (pinned to 24c60942c5)

Solutions

Print tensor.shape vs the expected_shape named in the message and fix the generating code so they match
Regenerate sparsity metadata with the current config (heads, experts, block counts)
Only rely on size-1 dims for broadcasting; never expect arbitrary dims to expand

Example fix

# before
cnt = torch.ones((2, 12, 31))          # expected (2, 12, 32)
res = _expand_sparsity_tensor(cnt, (2, 12, 32), "k_block_cnt", ctx, None)
# after
cnt = torch.ones((2, 12, 32))
res = _expand_sparsity_tensor(cnt, (2, 12, 32), "k_block_cnt", ctx, None)

Defensive patterns

Strategy: validation

Validate before calling

def can_expand_to(shape, expected):
    return all(c == t or c == 1 for c, t in zip(shape, expected))
assert can_expand_to(tuple(cnt.shape), tuple(expected_shape)), \
    f"{cnt.shape} cannot expand to {expected_shape}"

Type guard

def is_expandable(t: torch.Tensor, expected_shape) -> bool:
    return all(c == t or c == 1 for c, t in zip(t.shape, expected_shape))

Prevention

Generate sparsity metadata programmatically from the model config rather than by hand
Log expected vs actual shapes in data prep
Add shape assertions next to metadata creation

When it happens

Trigger: Passing a block_sparsity cnt/idx/metadata tensor whose shape disagrees with the expected shape in any dimension that is not 1, via _check_and_expand_block or _check_and_expert_metadata_tensor into normalize_block_sparse_tensors.

Common situations: Changing num_experts, batch, or head counts without regenerating sparsity metadata; hand-built layout tensors with a wrong tile count; broadcasting expectations from a different model config.

Related errors

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/4986a033372e5dbb. Report an issue: GitHub.