sgl-project/sglang · error · ValueError

`fuse_qkv_projections()` is not supported for models having

Error message

`fuse_qkv_projections()` is not supported for models having added KV projections.

What it means

fuse_qkv_projections merges the Q, K, and V projection matrices into single kernels for speed. Models whose attention processors add extra KV projections (class name contains 'Added', e.g. AddedKVProcessor) cannot be fused because the added projections have no place in the fused layout, so the method raises before mutating state.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/vaes/autoencoder.py:589

        else:
            z = posterior.mode()
        dec = self.decode(z).sample

        return dec

    # Copied from diffusers.models.unets.unet_2d_condition.UNet2DConditionModel.fuse_qkv_projections
    def fuse_qkv_projections(self):
        """
        Enables fused QKV projections. For self-attention modules, all projection matrices (i.e., query, key, value)
        are fused. For cross-attention modules, key and value projection matrices are fused.

        > [!WARNING] > This API is 🧪 experimental.
        """
        self.original_attn_processors = None

        for _, attn_processor in self.attn_processors.items():
            if "Added" in str(attn_processor.__class__.__name__):
                raise ValueError(
                    "`fuse_qkv_projections()` is not supported for models having added KV projections."
                )

        self.original_attn_processors = self.attn_processors

        for module in self.modules():
            if isinstance(module, Attention):
                module.fuse_projections(fuse=True)

        self.set_attn_processor(FusedAttnProcessor2_0())


EntryClass = AutoencoderKL

View on GitHub (pinned to 0132848349)

Solutions

1. Skip fusion for this model — it's unsupported by design; remove the fuse_qkv_projections() call
2. If a custom processor with 'Added' in the name was set but not actually needed, reset to a standard processor first: set_attn_processor(AttnProcessor()), then fuse
3. Use torch.compile or other optimization that tolerates added KV projections

Example fix

# before
model.fuse_qkv_projections()  # raises on added-KV models
# after
# don't fuse; or reset processor first
model.set_attn_processor(AttnProcessor())
model.fuse_qkv_projections()

has_added_kv = any("Added" in type(p).__name__ for p in model.attn_processors.values())
if not has_added_kv:
    model.fuse_qkv_projections()

def can_fuse_qkv(model) -> bool:
    return not any("Added" in type(p).__name__ for p in model.attn_processors.values())

try:
    model.fuse_qkv_projections()
except ValueError:
    logger.warning("QKV fusion unsupported for this model; skipping")

When it happens

Trigger: Calling fuse_qkv_projections() on an autoencoder/attention model that uses an 'Added'-type attention processor (installed explicitly or by default in architectures with added KV layers).

Common situations: Applying a generic 'speed up inference by fusing QKV' recipe to a model architecture that uses added-KV attention; enabling fusion after a config change added KV projections; copying optimization code from a text-to-image model to a video/added-KV model.

Related errors

• Invalid threshold_type for topk: {threshold_type}. Choose 'q
• Invalid threshold_type: {threshold_type}. Choose 'query_head
• Invalid select_mode: {select_mode}. Choose 'topk' or 'thresh
• A dict of processors was passed, but the number of processor
• Cannot call `set_default_attn_processor` when attention proc

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/749ef8d66d944ce7. Report an issue: GitHub.