{"record":{"id":"4379a16e9de265e5","repo":"sgl-project/sglang","slug":"minimax-h3-packed-sequence-alignment-minimax-h3-p","errorCode":null,"errorMessage":"MiniMax H3 packed sequence alignment {MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT} must be divisible by the combined sequence-parallel size {sp_size} (ulysses={ulysses_size} x ring={ring_size}). Choose degrees whose product divides both the TP-local attention heads and the packed sequence alignment.","messagePattern":"MiniMax H3 packed sequence alignment (.+?) must be divisible by the combined sequence-parallel size (.+?) \\(ulysses=(.+?) x ring=(.+?)\\)\\. Choose degrees whose product divides both the TP-local attention heads and the packed sequence alignment\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py","lineNumber":1855,"sourceCode":"    ) -> None:\n        if ulysses_size <= 0:\n            raise ValueError(\"MiniMax H3 Ulysses size must be positive.\")\n        if ring_size <= 0:\n            raise ValueError(\"MiniMax H3 ring size must be positive.\")\n        local_heads = arch.num_attention_heads // tp_size\n        if local_heads % ulysses_size:\n            raise ValueError(\n                f\"MiniMax H3 TP-local heads {local_heads} must be divisible by \"\n                f\"Ulysses size {ulysses_size} (total heads=\"\n                f\"{arch.num_attention_heads}, TP={tp_size}).\"\n            )\n        # ring never shards heads (only rows), so it has no head-divisibility\n        # constraint; the packed sequence alignment constant must still\n        # divide the *combined* sequence-parallel size, since ring adds an\n        # outer row split on top of Ulysses's inner one (see forward()).\n        sp_size = ulysses_size * ring_size\n        if MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT % sp_size:\n            raise ValueError(\n                \"MiniMax H3 packed sequence alignment \"\n                f\"{MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT} must be divisible by \"\n                f\"the combined sequence-parallel size {sp_size} \"\n                f\"(ulysses={ulysses_size} x ring={ring_size}). Choose degrees \"\n                \"whose product divides both the TP-local attention heads and \"\n                \"the packed sequence alignment.\"\n            )\n\n    def __init__(\n        self,\n        config: MiniMaxH3DiTConfig,\n        hf_config: dict[str, Any],\n        quant_config: QuantizationConfig | None = None,\n        adaln_cache_path: str | None = None,\n        adaln_cache_model_variant: str | None = None,\n        adaln_weight_files: list[str] | None = None,\n        adaln_plan_width: int = MINIMAX_H3_ADALN_MAX_PLAN_WIDTH,\n    ) -> None:","sourceCodeStart":1837,"sourceCodeEnd":1873,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py#L1837-L1873","documentation":"The packed-sequence alignment constant (MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT) must be divisible by the combined sequence-parallel size sp_size = ulysses_size * ring_size, because ring adds an outer row split on top of Ulysses's inner split and every local chunk must keep the alignment padding valid. Both degrees' product must divide the constant (and the TP-local heads).","triggerScenarios":"e.g. alignment=128 with ulysses=8, ring=4 → sp=32 divides 128 OK, but ulysses=9 or sp=256 fails; any sp_size that is not a divisor of the alignment constant.","commonSituations":"Turning on ring attention on top of Ulysses so the product exceeds the alignment constant (e.g. 16*16=256 > 128); tuning sequence-parallel degrees independently without checking the product.","solutions":["Pick (ulysses, ring) whose product divides MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT (typically a power of two <= the constant, e.g. product in {1,2,4,...,128})","Shift capacity into Ulysses rather than ring if the product is too large (ring doesn't need head divisibility, but the product limit still applies)","Recompute: sp = ulysses * ring; assert MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT % sp == 0 before launch"],"exampleFix":"# before\nulysses_size, ring_size = 8, 16  # sp=128? if alignment smaller -> fail\n# after\nulysses_size, ring_size = 8, 4  # product must divide alignment constant\nassert ALIGN % (ulysses * ring) == 0","handlingStrategy":"validation","validationCode":"from sglang.multimodal_gen.runtime.models.dits.minimax_h3 import MINIMAX_H3_PACKED_SEQUENCE_ALIGNMENT as ALIGN\nsp = ulysses_size * ring_size\nassert ALIGN % sp == 0 and (arch.num_attention_heads // tp_size) % ulysses_size == 0","typeGuard":"def sp_ok(align: int, u: int, r: int) -> bool:\n    return u >= 1 and r >= 1 and align % (u * r) == 0","tryCatchPattern":null,"preventionTips":["Keep ulysses*ring a power of two <= the alignment constant","Validate the combined SP product in a launch-time smoke check"],"tags":["sequence-parallel","ulysses","ring-attention","alignment","divisibility"],"backgroundTag":"invalid-parallelism-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}