{"record":{"id":"749ef8d66d944ce7","repo":"sgl-project/sglang","slug":"fuse-qkv-projections-is-not-supported-for-mode","errorCode":null,"errorMessage":"`fuse_qkv_projections()` is not supported for models having added KV projections.","messagePattern":"`fuse_qkv_projections\\(\\)` is not supported for models having added KV projections\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/models/vaes/autoencoder.py","lineNumber":589,"sourceCode":"        else:\n            z = posterior.mode()\n        dec = self.decode(z).sample\n\n        return dec\n\n    # Copied from diffusers.models.unets.unet_2d_condition.UNet2DConditionModel.fuse_qkv_projections\n    def fuse_qkv_projections(self):\n        \"\"\"\n        Enables fused QKV projections. For self-attention modules, all projection matrices (i.e., query, key, value)\n        are fused. For cross-attention modules, key and value projection matrices are fused.\n\n        > [!WARNING] > This API is 🧪 experimental.\n        \"\"\"\n        self.original_attn_processors = None\n\n        for _, attn_processor in self.attn_processors.items():\n            if \"Added\" in str(attn_processor.__class__.__name__):\n                raise ValueError(\n                    \"`fuse_qkv_projections()` is not supported for models having added KV projections.\"\n                )\n\n        self.original_attn_processors = self.attn_processors\n\n        for module in self.modules():\n            if isinstance(module, Attention):\n                module.fuse_projections(fuse=True)\n\n        self.set_attn_processor(FusedAttnProcessor2_0())\n\n\nEntryClass = AutoencoderKL\n","sourceCodeStart":571,"sourceCodeEnd":603,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/models/vaes/autoencoder.py#L571-L603","documentation":"fuse_qkv_projections merges the Q, K, and V projection matrices into single kernels for speed. Models whose attention processors add extra KV projections (class name contains 'Added', e.g. AddedKVProcessor) cannot be fused because the added projections have no place in the fused layout, so the method raises before mutating state.","triggerScenarios":"Calling fuse_qkv_projections() on an autoencoder/attention model that uses an 'Added'-type attention processor (installed explicitly or by default in architectures with added KV layers).","commonSituations":"Applying a generic 'speed up inference by fusing QKV' recipe to a model architecture that uses added-KV attention; enabling fusion after a config change added KV projections; copying optimization code from a text-to-image model to a video/added-KV model.","solutions":["Skip fusion for this model — it's unsupported by design; remove the fuse_qkv_projections() call","If a custom processor with 'Added' in the name was set but not actually needed, reset to a standard processor first: set_attn_processor(AttnProcessor()), then fuse","Use torch.compile or other optimization that tolerates added KV projections"],"exampleFix":"# before\nmodel.fuse_qkv_projections()  # raises on added-KV models\n# after\n# don't fuse; or reset processor first\nmodel.set_attn_processor(AttnProcessor())\nmodel.fuse_qkv_projections()","handlingStrategy":"validation","validationCode":"has_added_kv = any(\"Added\" in type(p).__name__ for p in model.attn_processors.values())\nif not has_added_kv:\n    model.fuse_qkv_projections()","typeGuard":"def can_fuse_qkv(model) -> bool:\n    return not any(\"Added\" in type(p).__name__ for p in model.attn_processors.values())","tryCatchPattern":"try:\n    model.fuse_qkv_projections()\nexcept ValueError:\n    logger.warning(\"QKV fusion unsupported for this model; skipping\")","preventionTips":["Gate fusion behind an architecture capability check","Keep original_attn_processors around to unfuse instead of re-defaulting"],"tags":["fusion","attention","optimization","value-error"],"backgroundTag":"unsupported-operation-for-model-type","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}