xai-org/x-algorithm · error · ValueError

Expected bound to have shape (4,), got {bound_arr.shape}

Error message

Expected bound to have shape (4,), got {bound_arr.shape}

What it means

_normalize_bound converts the attention bound (segment boundary spec) to an int32 array and requires exactly shape (4,) — the four bounds (start, end, etc.) per batch — before broadcasting to (batch_size, 4). None defaults to (0, seq_len+1, seq_len+1, seq_len+1).

Source

Thrown at phoenix/xrex/pallas/ranker_attention_fa3.py:411

        grid_names=("heads", "q_seq", "batch"),
        num_threads=3,
        thread_name="wg",
        compiler_params=plgpu.CompilerParams(approx_math=True),
    )(q, k, v, bound)

    if save_residuals:
        assert lse is not None
        return out, (lse,)

    return out


def _normalize_bound(bound, batch_size: int, seq_len: int):
    if bound is None:
        bound = (0, seq_len + 1, seq_len + 1, seq_len + 1)
    bound_arr = jnp.asarray(bound, dtype=jnp.int32)
    if bound_arr.shape != (4,):
        raise ValueError(f"Expected bound to have shape (4,), got {bound_arr.shape}")
    return jnp.broadcast_to(bound_arr, (batch_size, 4))


@functools.partial(jax.custom_vjp, nondiff_argnums=(3, 4, 5, 6, 7, 8, 9))
@functools.partial(
    jax.jit,
    static_argnames=[
        "config",
        "save_residuals",
        "bound",
        "sm_scale",
        "cap",
        "cap_method",
        "z_loss_weight",
    ],
)
def attention(
    q,

View on GitHub (pinned to 24c60942c5)

Solutions

1. Pass bound as a length-4 sequence, e.g. [0, end, end, end]
2. Or pass None to use the default full-range bound
3. Keep elements ints (they are cast to int32)

Example fix

# before
attn = sharded_mha(q, k, v, bound=jnp.array([[0, 128, 128, 128]] * batch))
# after
attn = sharded_mha(q, k, v, bound=(0, 128, 128, 128))

null

def is_valid_bound(b) -> bool:
    return b is None or (hasattr(b, "__len__") and len(b) == 4)

When it happens

Trigger: Passing bound as a scalar, a (batch_size, 4) array (double-broadcast attempt), a 3-element list, or nested lists of wrong shape.

Common situations: Users pre-broadcasting bounds themselves; passing per-query bounds or Python ints; shape confusion about whether bound is per-batch or global.

Understand the failure class

Background: Tensor shape mismatch errors ("must have shape", "expected shape ... got ..."): when tensor dimensions disagree with what an op or layer was told to expect — this error's family across 6 libraries.

Related errors

• q, k, and v should all be 4D, got: {q.ndim=}, {k.ndim=}, {v.
• {kv_seq_len=} must be a multiple of {block_kv=}
• {q_seq_len=} must be a multiple of {block_q * 2=}
• {q_seq_len=} must be a multiple of {config.block_q_dq=} * {c
• Invalid backward pass implementation: {backward_pass_impl}

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/1752f4bed250da5c. Report an issue: GitHub.