xai-org/x-algorithm · error · ValueError

fixed_total_len={fixed_total_len} is smaller than packed len

Error message

fixed_total_len={fixed_total_len} is smaller than packed length {packed_len}

What it means

pack_inputs flattens variable-length sequences, drops padding, and optionally re-pads to fixed_total_len. If the actual packed (valid token) length exceeds fixed_total_len, there is not enough room and the function raises this error. It typically means the caller's static bucket size is smaller than the real total sequence length in the batch.

Source

Thrown at phoenix/xrex/pallas/ranker_attention_varlen.py:164

    flat_segment_ids = segment_ids.reshape(total_len)
    valid = flat_segment_ids != PADDING_SEGMENT_ID

    flat_q = q.reshape(total_len, num_q_heads, head_dim)
    flat_k = k.reshape(total_len, num_kv_heads, head_dim)
    flat_v = v.reshape(total_len, num_kv_heads, head_dim)
    flat_temp = temp.reshape(total_len)

    q_packed = flat_q[valid]
    k_packed = flat_k[valid]
    v_packed = flat_v[valid]
    temp_packed = flat_temp[valid]
    segment_ids_packed = flat_segment_ids[valid]

    packed_len = int(q_packed.shape[0])
    if fixed_total_len is not None:
        if fixed_total_len < packed_len:
            raise ValueError(
                f"fixed_total_len={fixed_total_len} is smaller than packed length {packed_len}"
            )
        pad_len = fixed_total_len - packed_len
        if pad_len > 0:
            q_packed = jnp.pad(q_packed, ((0, pad_len), (0, 0), (0, 0)))
            k_packed = jnp.pad(k_packed, ((0, pad_len), (0, 0), (0, 0)))
            v_packed = jnp.pad(v_packed, ((0, pad_len), (0, 0), (0, 0)))
            temp_packed = jnp.pad(temp_packed, ((0, pad_len),), constant_values=0)
            segment_ids_packed = jnp.pad(
                segment_ids_packed, ((0, pad_len),), constant_values=PADDING_SEGMENT_ID
            )

    seqlens = jnp.sum(segment_ids != PADDING_SEGMENT_ID, axis=1).astype(jnp.int32)
    cu_seqlens = jnp.concatenate([jnp.array([0], dtype=seqlens.dtype), jnp.cumsum(seqlens)])
    max_seqlen = int(jnp.max(seqlens))

    return (
        q_packed,

View on GitHub (pinned to 24c60942c5)

Solutions

1. Increase fixed_total_len to >= the true packed length (e.g. next bucket size).
2. Compute fixed_total_len from the batch: int(valid.sum()) or sum of lengths before calling, with headroom.
3. Filter/clip batches whose total length exceeds the bucket during data loading.
4. Check whether padding tokens are being counted as valid due to a wrong mask, inflating packed_len.

Example fix

# before
out = attn(q, k, v, lens, fixed_total_len=8192)  # batch packs 9000 tokens
# after
fixed = int(sum(lens))  # or next bucket >= packed len
out = attn(q, k, v, lens, fixed_total_len=16384)

packed = int(sum(lens))  # or int(valid_mask.sum())
assert fixed_total_len is None or fixed_total_len >= packed

When it happens

Trigger: Calling the varlen attention wrapper with fixed_total_len set to a value smaller than sum of valid sequence lengths in the batch (packed_len), e.g. bucket 8192 while the batch packs 10000 tokens.

Common situations: Static-shape bucketing for TPU compilation where batch content grew beyond the bucket; mixed datasets with occasional longer batches after shuffling; incorrect computation of fixed_total_len from batch metadata.

Related errors

• Invalid backward pass implementation: {backward_pass_impl}
• cap_method must be in [tanh, soft_sign], got {cap_method}
• Invalid backward pass implementation: {backward_pass_impl}
• cap_method must be in [tanh, soft_sign, none], got {cap_meth
• {self.block_q=} must be a multiple of 64

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/f12dd0b9652c752d. Report an issue: GitHub.