{"record":{"id":"28233cc83f937436","repo":"sgl-project/sglang","slug":"num-attention-heads-must-be-positive","errorCode":null,"errorMessage":"num_attention_heads must be positive.","messagePattern":"num_attention_heads must be positive\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py","lineNumber":1807,"sourceCode":"\n    def _can_batch_block_adaln(self) -> bool:\n        return (\n            self.adaln_cache is None\n            and get_tp_world_size() > 1\n            and not torch.compiler.is_compiling()\n            and not envs.SGLANG_CACHE_DIT_ENABLED\n            and not hasattr(self, \"_sglang_cache_dit_adapter\")\n            and not is_layerwise_offloaded_module(self)\n            and all(type(block) is MiniMaxH3DiTBlock for block in self.blocks)\n        )\n\n    def _validate_tp_config(\n        self, *, arch: MiniMaxH3DiTArchConfig, tp_size: int\n    ) -> None:\n        if tp_size <= 0:\n            raise ValueError(\"TP size must be positive.\")\n        if arch.num_attention_heads <= 0:\n            raise ValueError(\"num_attention_heads must be positive.\")\n        if arch.hidden_size <= 0:\n            raise ValueError(\"hidden_size must be positive.\")\n        if arch.attention_head_dim <= 0:\n            raise ValueError(\"attention_head_dim must be positive.\")\n        if arch.ffn_hidden_size <= 0:\n            raise ValueError(\"ffn_hidden_size must be positive.\")\n        for name, value in (\n            (\"num_attention_heads\", arch.num_attention_heads),\n            (\"hidden_size\", arch.hidden_size),\n            (\"ffn_hidden_size\", arch.ffn_hidden_size),\n            (\"time_embed_hidden_size\", arch.time_embed_hidden_size),\n            (\"adaln_out_features\", arch.adaln_out_features),\n            (\"final_adaln_out_features\", arch.final_adaln_out_features),\n            (\"video_patch_output_dim\", arch.latents_dim * math.prod(arch.patch_size)),\n            (\"audio_patch_output_dim\", arch.audio_latents_dim),\n        ):\n            if value % tp_size:\n                raise ValueError(","sourceCodeStart":1789,"sourceCodeEnd":1825,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py#L1789-L1825","documentation":"The MiniMax H3 DiT constructor validates that the architecture config's num_attention_heads is positive, because TP shards heads across ranks (num_attention_heads // tp_size) and attention projection weights are shaped by head count. A zero/negative value means a malformed or partially-loaded model config.","triggerScenarios":"Building the model from a hand-written MiniMaxH3DiTArchConfig or a JSON checkpoint config where num_attention_heads is 0, missing and defaulted to 0, or negative.","commonSituations":"Typos in a custom arch config; converting a checkpoint with a renamed field (n_heads vs num_attention_heads) so the field silently defaults to 0; editing a config template and dropping the field.","solutions":["Inspect the arch config object right before constructing the model and print num_attention_heads","Fix the source config: set num_attention_heads to the checkpoint's true head count (e.g. 24/48/64)","If loading from JSON, verify the key name matches MiniMaxH3DiTArchConfig's field names exactly"],"exampleFix":"# before\narch = MiniMaxH3DiTArchConfig(num_attention_heads=0, ...)\n# after\narch = MiniMaxH3DiTArchConfig(num_attention_heads=48, ...)","handlingStrategy":"validation","validationCode":"assert arch.num_attention_heads > 0, f\"bad num_attention_heads={arch.num_attention_heads}\"","typeGuard":"def heads_ok(arch) -> bool:\n    return getattr(arch, \"num_attention_heads\", 0) > 0","tryCatchPattern":null,"preventionTips":["Validate arch config fields before model construction","Log the deserialized config after checkpoint conversion"],"tags":["config","model-architecture","validation","minimax-h3"],"backgroundTag":"invalid-model-config","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}