sgl-project/sglang · critical · ValueError

MiniMax H3 packed sequence alignment {MINIMAX_H3_PACKED_SEQU

Error message

MiniMax H3 packed sequence alignment {MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT} must be divisible by the combined sequence-parallel size {sp_size} (ulysses={ulysses_size} x ring={ring_size}). Choose degrees whose product divides both the TP-local attention heads and the packed sequence alignment.

What it means

The packed-sequence alignment constant (MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT) must be divisible by the combined sequence-parallel size sp_size = ulysses_size * ring_size, because ring adds an outer row split on top of Ulysses's inner split and every local chunk must keep the alignment padding valid. Both degrees' product must divide the constant (and the TP-local heads).

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py:1855

    ) -> None:
        if ulysses_size <= 0:
            raise ValueError("MiniMax H3 Ulysses size must be positive.")
        if ring_size <= 0:
            raise ValueError("MiniMax H3 ring size must be positive.")
        local_heads = arch.num_attention_heads // tp_size
        if local_heads % ulysses_size:
            raise ValueError(
                f"MiniMax H3 TP-local heads {local_heads} must be divisible by "
                f"Ulysses size {ulysses_size} (total heads="
                f"{arch.num_attention_heads}, TP={tp_size})."
            )
        # ring never shards heads (only rows), so it has no head-divisibility
        # constraint; the packed sequence alignment constant must still
        # divide the *combined* sequence-parallel size, since ring adds an
        # outer row split on top of Ulysses's inner one (see forward()).
        sp_size = ulysses_size * ring_size
        if MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT % sp_size:
            raise ValueError(
                "MiniMax H3 packed sequence alignment "
                f"{MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT} must be divisible by "
                f"the combined sequence-parallel size {sp_size} "
                f"(ulysses={ulysses_size} x ring={ring_size}). Choose degrees "
                "whose product divides both the TP-local attention heads and "
                "the packed sequence alignment."
            )

    def __init__(
        self,
        config: MiniMaxH3DiTConfig,
        hf_config: dict[str, Any],
        quant_config: QuantizationConfig | None = None,
        adaln_cache_path: str | None = None,
        adaln_cache_model_variant: str | None = None,
        adaln_weight_files: list[str] | None = None,
        adaln_plan_width: int = MINIMAX_H3_ADALN_MAX_PLAN_WIDTH,
    ) -> None:

View on GitHub (pinned to 0132848349)

Solutions

1. Pick (ulysses, ring) whose product divides MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT (typically a power of two <= the constant, e.g. product in {1,2,4,...,128})
2. Shift capacity into Ulysses rather than ring if the product is too large (ring doesn't need head divisibility, but the product limit still applies)
3. Recompute: sp = ulysses * ring; assert MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT % sp == 0 before launch

Example fix

# before
ulysses_size, ring_size = 8, 16  # sp=128? if alignment smaller -> fail
# after
ulysses_size, ring_size = 8, 4  # product must divide alignment constant
assert ALIGN % (ulysses * ring) == 0

from sglang.multimodal_gen.runtime.models.dits.minimax_h3 import MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT as ALIGN
sp = ulysses_size * ring_size
assert ALIGN % sp == 0 and (arch.num_attention_heads // tp_size) % ulysses_size == 0

def sp_ok(align: int, u: int, r: int) -> bool:
    return u >= 1 and r >= 1 and align % (u * r) == 0

When it happens

Trigger: e.g. alignment=128 with ulysses=8, ring=4 → sp=32 divides 128 OK, but ulysses=9 or sp=256 fails; any sp_size that is not a divisor of the alignment constant.

Common situations: Turning on ring attention on top of Ulysses so the product exceeds the alignment constant (e.g. 16*16=256 > 128); tuning sequence-parallel degrees independently without checking the product.

Related errors

• packed seq_len {seq_len} not divisible by the combined seque
• MiniMax H3 TP-local heads {local_heads} must be divisible by
• world_size must be positive and divide global_heads
• MiniMax H3 Ulysses size must be positive.
• MiniMax H3 ring size must be positive.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4379a16e9de265e5. Report an issue: GitHub.