{"record":{"id":"e9776462d5288867","repo":"sgl-project/sglang","slug":"unknown-qk-norm-qk-norm-should-be-one-of-none","errorCode":null,"errorMessage":"unknown qk_norm: {qk_norm}. Should be one of None, 'layer_norm', 'fp32_layer_norm', 'layer_norm_across_heads', 'rms_norm', 'rms_norm_across_heads', 'l2'.","messagePattern":"unknown qk_norm: (.+?)\\. Should be one of None, 'layer_norm', 'fp32_layer_norm', 'layer_norm_across_heads', 'rms_norm', 'rms_norm_across_heads', 'l2'\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/dits/glm_image.py","lineNumber":590,"sourceCode":"                    input_is_parallel=True,\n                    quant_config=quant_config,\n                    prefix=f\"{prefix}.to_out.0\" if prefix else \"to_out.0\",\n                )\n            ]\n        )\n\n        if qk_norm is None:\n            self.norm_q = None\n            self.norm_k = None\n        elif qk_norm == \"layer_norm\":\n            self.norm_q = nn.LayerNorm(\n                dim_head, eps=eps, elementwise_affine=elementwise_affine\n            )\n            self.norm_k = nn.LayerNorm(\n                dim_head, eps=eps, elementwise_affine=elementwise_affine\n            )\n        else:\n            raise ValueError(\n                f\"unknown qk_norm: {qk_norm}. Should be one of None, 'layer_norm', 'fp32_layer_norm', 'layer_norm_across_heads', 'rms_norm', 'rms_norm_across_heads', 'l2'.\"\n            )\n\n        self.attn = USPAttention(\n            num_heads=self.num_local_heads,\n            head_size=dim_head,\n            num_kv_heads=self.num_local_kv_heads,\n            dropout_rate=0,\n            softmax_scale=None,\n            causal=False,\n        )\n\n    def forward(\n        self,\n        hidden_states: torch.Tensor,\n        encoder_hidden_states: torch.Tensor,\n        attention_mask: Optional[torch.Tensor] = None,\n        image_rotary_emb: Optional[Tuple[torch.Tensor, torch.Tensor]] = None,","sourceCodeStart":572,"sourceCodeEnd":608,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/dits/glm_image.py#L572-L608","documentation":"Raised in the GLM image DiT attention block's __init__ when the qk_norm argument is not one of the supported normalization schemes (None, 'layer_norm', 'fp32_layer_norm', 'layer_norm_across_heads', 'rms_norm', 'rms_norm_across_heads', 'l2'). The constructor dispatches on this string to build norm_q/norm_k modules, so any unrecognized spelling falls through to this ValueError. It is purely a configuration-string validation error at model construction time.","triggerScenarios":"Constructing the GLM image model (or its attention module) with a qk_norm string outside the allowed set — e.g. 'rmsnorm', 'layernorm', 'LayerNorm', 'none' (string instead of None), or a value read from a JSON config with a typo.","commonSituations":"Hand-editing a model config JSON and mistyping the qk_norm field; porting a config from another DiT repo that uses different norm names; passing the string 'none' instead of the Python None.","solutions":["Set qk_norm to one of the exact allowed values: None, 'layer_norm', 'fp32_layer_norm', 'layer_norm_across_heads', 'rms_norm', 'rms_norm_across_heads', 'l2'","Check the model's official config file for the correct qk_norm value and use it verbatim","If loading from a custom config, add a normalization/mapping step that translates your naming to the supported names before constructing the model"],"exampleFix":"# before\nmodel = GLMImageModel(config={\"qk_norm\": \"rmsnorm\"})\n\n# after\nmodel = GLMImageModel(config={\"qk_norm\": \"rms_norm\"})","handlingStrategy":"validation","validationCode":"ALLOWED_QK_NORM = {None, 'layer_norm', 'fp32_layer_norm', 'layer_norm_across_heads', 'rms_norm', 'rms_norm_across_heads', 'l2'}\nassert config.get('qk_norm') in ALLOWED_QK_NORM, f\"qk_norm must be one of {ALLOWED_QK_NORM}, got {config.get('qk_norm')!r}\"","typeGuard":"def is_valid_qk_norm(v) -> bool:\n    return v in {None, 'layer_norm', 'fp32_layer_norm', 'layer_norm_across_heads', 'rms_norm', 'rms_norm_across_heads', 'l2'}","tryCatchPattern":null,"preventionTips":["Validate config strings against the model's allowed set before construction","Load configs from the checkpoint's original files rather than hand-editing","Add unit tests for config parsing that assert unknown values fail fast"],"tags":["config","validation","diffusion","qk-norm"],"backgroundTag":"invalid-config-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}