{"record":{"id":"b5229bea72376449","repo":"sgl-project/sglang","slug":"unsupported-qk-norm-qk-norm","errorCode":null,"errorMessage":"Unsupported qk_norm: {qk_norm}","messagePattern":"Unsupported qk_norm: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py","lineNumber":420,"sourceCode":"        self.tp_rmsnorm = qk_norm == \"rms_norm_across_heads\" and tp_size > 1\n\n        self.attn1 = USPAttention(\n            num_heads=self.local_num_heads,\n            head_size=self.dim_head,\n            causal=False,\n            supported_attention_backends=supported_attention_backends,\n            prefix=add_prefix(\"attn1\", prefix),\n            quant_config=quant_config,\n            is_cross_attention=False,\n        )\n        if qk_norm == \"rms_norm\":\n            self.norm_q = RMSNorm(self.dim_head, eps=eps)\n            self.norm_k = RMSNorm(self.dim_head, eps=eps)\n        elif qk_norm == \"rms_norm_across_heads\":\n            self.norm_q = RMSNorm(dim, eps=eps)\n            self.norm_k = RMSNorm(dim, eps=eps)\n        else:\n            raise ValueError(f\"Unsupported qk_norm: {qk_norm}\")\n        if not cross_attn_norm:\n            raise ValueError(\"LingBotWorld requires cross_attn_norm=True\")\n        self.self_attn_residual_norm = ScaleResidualLayerNormScaleShift(\n            dim,\n            eps=eps,\n            elementwise_affine=True,\n            dtype=torch.float32,\n        )\n\n        cross_attn_backends = {\n            b for b in supported_attention_backends if not b.is_sparse\n        }\n        if added_kv_proj_dim is not None:\n            self.attn2 = WanI2VCrossAttention(\n                dim,\n                num_heads,\n                qk_norm=qk_norm,\n                eps=eps,","sourceCodeStart":402,"sourceCodeEnd":438,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/lingbot_world.py#L402-L438","documentation":"LingBotWorld attention only implements two QK-norm modes: per-head RMSNorm ('rms_norm'-style on dim_head) and 'rms_norm_across_heads' (RMSNorm over the full dim). Any other qk_norm string in the config is rejected at init because no norm layers could be built.","triggerScenarios":"Constructing the attention with config.qk_norm set to something like 'layer_norm', 'qk_norm', None, or a typo such as 'rms-norm-across-heads'.","commonSituations":"Adapting a config from another DiT family that uses different norm names; hand-editing config.json; case or dash/slash differences in the string.","solutions":["Set qk_norm to one of the supported strings: per-head mode or 'rms_norm_across_heads'","If your checkpoint truly uses a different norm, implement the branch in LingBotWorld attention and map the name","Validate the config value before model construction"],"exampleFix":"// before\nconfig.qk_norm = \"layer_norm\"\n\n// after\nconfig.qk_norm = \"rms_norm_across_heads\"   // or per-head RMSNorm mode","handlingStrategy":"validation","validationCode":"SUPPORTED_QK_NORM = {'rms_norm', 'rms_norm_across_heads'}\\nassert config.qk_norm in SUPPORTED_QK_NORM, f'qk_norm={config.qk_norm!r} not in {SUPPORTED_QK_NORM}'","typeGuard":"def is_supported_qk_norm(v: str) -> bool:\\n    return v in ('rms_norm', 'rms_norm_across_heads')","tryCatchPattern":null,"preventionTips":["Validate strings against the supported set before init","Copy exact strings from the checkpoint's reference config","Add alias mapping in your config loader"],"tags":["config-validation","qk-norm","init-time","lingbot"],"backgroundTag":"unsupported-config-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}