xai-org/x-algorithm · error · ValueError

{self.block_q=} must be a multiple of 64

Error message

{self.block_q=} must be a multiple of 64

What it means

TuningConfig.__post_init__ validates that the forward query tile size block_q is a multiple of 64, a hardware alignment requirement of the warp-specialized TMA kernels in this FA3-style pallas attention.

Source

Thrown at phoenix/xrex/pallas/ranker_attention_fa3.py:59

@dataclasses.dataclass(frozen=True)
class TuningConfig:
    block_q: int
    block_kv: int
    max_concurrent_steps: int
    use_schedule_barrier: bool = True
    causal: bool = False
    compute_wgs_bwd: int = 1

    block_q_dkv: int | None = None
    block_kv_dkv: int | None = None
    block_q_dq: int | None = None
    block_kv_dq: int | None = None

    def __post_init__(self):
        if self.block_q % 64:
            raise ValueError(f"{self.block_q=} must be a multiple of 64")
        if self.block_kv % 64:
            raise ValueError(f"{self.block_kv=} must be a multiple of 64")
        if self.max_concurrent_steps < 2:
            raise ValueError(f"{self.max_concurrent_steps=} must be at least 2")

        backward_blocks = [self.block_q_dkv, self.block_kv_dkv, self.block_q_dq, self.block_kv_dq]
        block_is_set = [blk is not None for blk in backward_blocks]
        if any(block_is_set) and not all(block_is_set):
            raise ValueError(
                "Backward block sizes (block_q_dkv, block_kv_dkv, block_q_dq, "
                "block_kv_dq) must either all be specified or all be None."
            )

    @property
    def has_backward_blocks(self) -> bool:
        return self.block_q_dkv is not None

View on GitHub (pinned to 24c60942c5)

Solutions

1. Set block_q to a multiple of 64 (128 is the typical default)
2. If you need smaller tiles, check whether ranker_attention.py (non-FA3) supports them

Example fix

# before
cfg = TuningConfig(block_q=96, ...)
# after
cfg = TuningConfig(block_q=128, ...)

assert block_q % 64 == 0, "block_q must be a multiple of 64"

null

When it happens

Trigger: Constructing TuningConfig with block_q such as 32, 48, or 96 — any value not divisible by 64.

Common situations: Copying tuning configs from non-FA2 pallas flash attention (which allows 32/16 tiles), or hand-tuning block sizes for small models without knowing the 64-alignment constraint.

Related errors

• {self.block_kv=} must be a multiple of 64
• {self.max_concurrent_steps=} must be at least 2
• Backward block sizes (block_q_dkv, block_kv_dkv, block_q_dq,
• {head_dim=} must be divisible by 64
• Need to specify backward blocks.

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/ad63407ef304ebeb. Report an issue: GitHub.