{"record":{"id":"a0a1a0eda096c3f3","repo":"sgl-project/sglang","slug":"qk-norm-type-not-supported","errorCode":null,"errorMessage":"QK Norm type not supported","messagePattern":"QK Norm type not supported","errorType":"exception","errorClass":"Exception","httpStatus":null,"severity":"critical","filePath":"python/sglang/multimodal_gen/runtime/models/dits/wanvideo.py","lineNumber":519,"sourceCode":"                supported_attention_backends=self_attn_backends,\n                prefix=add_prefix(\"attn1\", prefix),\n                quant_config=quant_config,\n                is_cross_attention=False,\n            )\n\n        self.hidden_dim = dim\n        self.num_attention_heads = num_heads\n        self.dim_head = dim // num_heads\n        if qk_norm == \"rms_norm\":\n            self.norm_q = RMSNorm(self.dim_head, eps=eps)\n            self.norm_k = RMSNorm(self.dim_head, eps=eps)\n        elif qk_norm == \"rms_norm_across_heads\":\n            # LTX applies qk norm across all heads\n            self.norm_q = RMSNorm(dim, eps=eps)\n            self.norm_k = RMSNorm(dim, eps=eps)\n        else:\n            logger.error(\"QK Norm type not supported\")\n            raise Exception\n        assert cross_attn_norm is True\n        self.qk_norm = qk_norm\n        self.tp_rmsnorm = qk_norm == \"rms_norm_across_heads\" and tp_size > 1\n        self.self_attn_residual_norm = ScaleResidualLayerNormScaleShift(\n            dim,\n            eps=eps,\n            elementwise_affine=True,\n            dtype=torch.float32,\n        )\n\n        # 2. Cross-attention\n        cross_attn_backends = {\n            b for b in supported_attention_backends if not b.is_sparse\n        }\n        if added_kv_proj_dim is not None:\n            # I2V\n            self.attn2 = WanI2VCrossAttention(\n                dim,","sourceCodeStart":501,"sourceCodeEnd":537,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/wanvideo.py#L501-L537","documentation":"Raised in WanVideo attention __init__ when the qk_norm string matches none of the supported options (only 'rms_norm_across_heads' and other listed branches are supported). The exception is raised bare after an error log, so the traceback carries no message.","triggerScenarios":"Constructing the attention block with an unrecognized qk_norm value such as 'layer_norm', 'knorm', a typo, or None.","commonSituations":"Adding a new Wan/LTX variant with a different QK normalization; config carrying an empty qk_norm string.","solutions":["Check supported qk_norm values in the if/elif chain in wanvideo.py around line 519 and use one of them","Set qk_norm = 'rms_norm_across_heads' for LTX-style models","Extend the chain with a new branch (plus norm_q/norm_k modules) if your model genuinely needs a new norm type"],"exampleFix":"# before\nqk_norm = \"layer_norm\"\n# after\nqk_norm = \"rms_norm_across_heads\"","handlingStrategy":"validation","validationCode":"SUPPORTED_QK_NORMS = {\"rms_norm_across_heads\", ...}  # mirror if/elif chain\nassert qk_norm in SUPPORTED_QK_NORMS, f\"unsupported qk_norm {qk_norm!r}\"","typeGuard":"def is_supported_qk_norm(s: str) -> bool:\n    return s in {\"rms_norm_across_heads\", \"layer_norm_across_heads\"}","tryCatchPattern":"except Exception as e:\n    logger.error(\"QK norm %r unsupported during attention init\", qk_norm)\n    raise","preventionTips":["Validate config strings at load time before constructing modules","Note: this raise is a bare Exception with no message — check the preceding log line"],"tags":["wanvideo","qk-norm","config-validation","init"],"backgroundTag":"unsupported-config-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}