xai-org/x-algorithm · error · ValueError

{self.block_kv=} must be a multiple of 64

Error message

{self.block_kv=} must be a multiple of 64

What it means

TuningConfig.__post_init__ requires the forward key/value tile size block_kv to be a multiple of 64, matching the TMA shared-memory alignment requirements of the FA3-style kernels.

Source

Thrown at phoenix/xrex/pallas/ranker_attention_fa3.py:61

@dataclasses.dataclass(frozen=True)
class TuningConfig:
    block_q: int
    block_kv: int
    max_concurrent_steps: int
    use_schedule_barrier: bool = True
    causal: bool = False
    compute_wgs_bwd: int = 1

    block_q_dkv: int | None = None
    block_kv_dkv: int | None = None
    block_q_dq: int | None = None
    block_kv_dq: int | None = None

    def __post_init__(self):
        if self.block_q % 64:
            raise ValueError(f"{self.block_q=} must be a multiple of 64")
        if self.block_kv % 64:
            raise ValueError(f"{self.block_kv=} must be a multiple of 64")
        if self.max_concurrent_steps < 2:
            raise ValueError(f"{self.max_concurrent_steps=} must be at least 2")

        backward_blocks = [self.block_q_dkv, self.block_kv_dkv, self.block_q_dq, self.block_kv_dq]
        block_is_set = [blk is not None for blk in backward_blocks]
        if any(block_is_set) and not all(block_is_set):
            raise ValueError(
                "Backward block sizes (block_q_dkv, block_kv_dkv, block_q_dq, "
                "block_kv_dq) must either all be specified or all be None."
            )

    @property
    def has_backward_blocks(self) -> bool:
        return self.block_q_dkv is not None


def _attention_forward(
    q,

View on GitHub (pinned to 24c60942c5)

Solutions

1. Set block_kv to a multiple of 64 (e.g. 64 or 128)
2. Reduce num_stages or max_concurrent_steps instead if shared memory is the constraint

Example fix

# before
cfg = TuningConfig(block_q=128, block_kv=50, ...)
# after
cfg = TuningConfig(block_q=128, block_kv=64, ...)

assert block_kv % 64 == 0, "block_kv must be a multiple of 64"

null

When it happens

Trigger: Constructing TuningConfig with block_kv like 32, 48, or 100 — any value not divisible by 64.

Common situations: Reducing block_kv to fit long-context sequences in shared memory without preserving 64-alignment; porting configs from FA2-style pallas kernels.

Related errors

• {self.block_q=} must be a multiple of 64
• {self.max_concurrent_steps=} must be at least 2
• Backward block sizes (block_q_dkv, block_kv_dkv, block_q_dq,
• {head_dim=} must be divisible by 64
• Need to specify backward blocks.

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/dfdca760813b38dd. Report an issue: GitHub.