{"record":{"id":"43c5606edf088ca0","repo":"sgl-project/sglang","slug":"only-rms-norm-across-heads-is-supported-as-a-val","errorCode":null,"errorMessage":"Only 'rms_norm_across_heads' is supported as a valid value for `qk_norm`.","messagePattern":"Only 'rms_norm_across_heads' is supported as a valid value for `qk_norm`\\.","errorType":"validation","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/adapter/ltx_2_connector.py","lineNumber":113,"sourceCode":"        self,\n        query_dim: int,\n        heads: int = 8,\n        kv_heads: int = 8,\n        dim_head: int = 64,\n        dropout: float = 0.0,\n        bias: bool = True,\n        cross_attention_dim: Optional[int] = None,\n        out_bias: bool = True,\n        qk_norm: str = \"rms_norm_across_heads\",\n        norm_eps: float = 1e-6,\n        norm_elementwise_affine: bool = True,\n        rope_type: str = \"interleaved\",\n        apply_gated_attention: bool = False,\n        processor=None,\n    ):\n        super().__init__()\n        if qk_norm != \"rms_norm_across_heads\":\n            raise NotImplementedError(\n                \"Only 'rms_norm_across_heads' is supported as a valid value for `qk_norm`.\"\n            )\n\n        self.head_dim = dim_head\n        self.inner_dim = dim_head * heads\n        self.inner_kv_dim = self.inner_dim if kv_heads is None else dim_head * kv_heads\n        self.query_dim = query_dim\n        self.cross_attention_dim = (\n            cross_attention_dim if cross_attention_dim is not None else query_dim\n        )\n        self.use_bias = bias\n        self.dropout = dropout\n        self.out_dim = query_dim\n        self.heads = heads\n        self.rope_type = rope_type\n\n        self.norm_q = torch.nn.RMSNorm(\n            dim_head * heads, eps=norm_eps, elementwise_affine=norm_elementwise_affine","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/adapter/ltx_2_connector.py#L95-L131","documentation":"NotImplementedError from the LTX-2 attention connector __init__: only qk_norm='rms_norm_across_heads' is implemented. Any other qk_norm value (or None) from the model config is rejected.","triggerScenarios":"Constructing the connector with qk_norm set to anything other than 'rms_norm_across_heads' (e.g. 'rmsnorm', 'layer_norm', None) — commonly from a checkpoint config that uses a different normalization name.","commonSituations":"New LTX-2 checkpoint variants with different qk-norm configs; typos or case differences in config ('RMS_Norm_across_heads'); ported configs from other codebases that use None or 'rms_norm' per-head.","solutions":["Set qk_norm='rms_norm_across_heads' exactly (lowercase, underscores)","Check the checkpoint config for the exact string it uses and remap it before constructing the connector","If you truly need another norm, implement it in the connector — it is deliberately unimplemented"],"exampleFix":"// before\nqk_norm=\"rms_norm\"\n// after\nqk_norm=\"rms_norm_across_heads\"","handlingStrategy":"validation","validationCode":"qk_norm = qk_norm or \"rms_norm_across_heads\"\nif qk_norm != \"rms_norm_across_heads\":\n    qk_norm = {\"rms_norm\": \"rms_norm_across_heads\"}.get(qk_norm, qk_norm)\nassert qk_norm == \"rms_norm_across_heads\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Normalize qk_norm string (strip().lower()) from checkpoint configs","Hard-code 'rms_norm_across_heads' unless implementing a new norm"],"tags":["qk-norm","unsupported-feature","ltx-2"],"backgroundTag":"unsupported-argument-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}