{"record":{"id":"6434624650a6e26b","repo":"hpcaitech/Open-Sora","slug":"fuse-qkv-projections-is-not-supported-for-mode","errorCode":null,"errorMessage":"`fuse_qkv_projections()` is not supported for models having added KV projections.","messagePattern":"`fuse_qkv_projections\\(\\)` is not supported for models having added KV projections\\.","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"opensora/models/hunyuan_vae/autoencoder_kl_causal_3d.py","lineNumber":590,"sourceCode":"        return (dec, posterior, z)\n\n    # Copied from diffusers.models.unet_2d_condition.UNet2DConditionModel.fuse_qkv_projections\n    def fuse_qkv_projections(self):\n        \"\"\"\n        Enables fused QKV projections. For self-attention modules, all projection matrices (i.e., query,\n        key, value) are fused. For cross-attention modules, key and value projection matrices are fused.\n\n        <Tip warning={true}>\n\n        This API is 🧪 experimental.\n\n        </Tip>\n        \"\"\"\n        self.original_attn_processors = None\n\n        for _, attn_processor in self.attn_processors.items():\n            if \"Added\" in str(attn_processor.__class__.__name__):\n                raise ValueError(\"`fuse_qkv_projections()` is not supported for models having added KV projections.\")\n\n        self.original_attn_processors = self.attn_processors\n\n        for module in self.modules():\n            if isinstance(module, Attention):\n                module.fuse_projections(fuse=True)\n\n    # Copied from diffusers.models.unet_2d_condition.UNet2DConditionModel.unfuse_qkv_projections\n    def unfuse_qkv_projections(self):\n        \"\"\"Disables the fused QKV projection if enabled.\n\n        <Tip warning={true}>\n\n        This API is 🧪 experimental.\n\n        </Tip>\n\n        \"\"\"","sourceCodeStart":572,"sourceCodeEnd":608,"githubUrl":"https://github.com/hpcaitech/Open-Sora/blob/7ad6a96a135feb81f755c84fb391818718f6beb2/opensora/models/hunyuan_vae/autoencoder_kl_causal_3d.py#L572-L608","documentation":"fuse_qkv_projections merges each Attention module's query/key/value projections into a single fused matrix for faster inference. This optimization is impossible when any attention processor is an 'Added-KV' variant (extra key/value projections), so the method scans self.attn_processors and aborts if any class name contains 'Added'.","triggerScenarios":"Calling model.fuse_qkv_projections() on an autoencoder whose attention layers use AttnAddedKVProcessor (or any processor class with 'Added' in the name), e.g. after set_attn_processor(AttnAddedKVProcessor()).","commonSituations":"Applying a standard diffusers inference-acceleration snippet (fuse_qkv + torch.compile) to a model variant that uses added-KV attention; fusing after loading a checkpoint with added-KV processors baked in.","solutions":["Skip fuse_qkv_projections for this model variant — it is not supported by design","If added-KV processors were set by mistake, reset to standard processors via set_attn_processor(AttnProcessor()) then fuse","Gate the call: if 'Added' not in str(processor) checks pass for all processors"],"exampleFix":"# before\nmodel.fuse_qkv_projections()\n# after\nhas_added_kv = any('Added' in str(p.__class__.__name__) for p in model.attn_processors.values())\nif not has_added_kv:\n    model.fuse_qkv_projections()","handlingStrategy":"type-guard","validationCode":"def can_fuse_qkv(model) -> bool:\n    return not any(\"Added\" in str(p.__class__.__name__) for p in model.attn_processors.values())","typeGuard":"def can_fuse_qkv(model) -> bool:\n    return not any(\"Added\" in str(p.__class__.__name__) for p in model.attn_processors.values())","tryCatchPattern":"try:\n    model.fuse_qkv_projections()\nexcept ValueError as e:\n    if \"added KV\" in str(e):\n        pass  # unsupported for this variant; skip fusion\n    else:\n        raise","preventionTips":["Check processor classes before calling fuse helpers","Wrap fusion in a capability check for shared pipelines","Document which model variants support QKV fusion"],"tags":["diffusers","attention","fusion","optimization"],"backgroundTag":"unsupported-operation-for-model-variant","analyzedSha":"7ad6a96a135feb81f755c84fb391818718f6beb2","analyzedAt":"2026-08-28T16:58:37.171Z","schemaVersion":2},"datasetVersion":"2026-08-28T21:17:43.275Z"}