Comfy-Org/ComfyUI · critical · ValueError

Normalization mode {self.qkv_norm_mode} not found, only supp

Error message

Normalization mode {self.qkv_norm_mode} not found, only support 'per_head'

What it means

Cosmos Attention.__init__ normalizes q/k/v with a per-head RMSNorm applied after rearranging channels into heads; qkv_norm_mode currently supports only 'per_head' (norm_dim = dim_head). Any other mode string (e.g. 'per_channel' from NVIDIA's original codebase) raises at construction because the norm dimension would be ambiguous. This is a config-surface check on the attention block parameters.

Source

Thrown at comfy/ldm/cosmos/blocks.py:106

        weight_args={},
        operations=None,
    ) -> None:
        super().__init__()

        self.is_selfattn = context_dim is None  # self attention

        inner_dim = dim_head * heads
        context_dim = query_dim if context_dim is None else context_dim

        self.heads = heads
        self.dim_head = dim_head
        self.qkv_norm_mode = qkv_norm_mode
        self.qkv_format = qkv_format

        if self.qkv_norm_mode == "per_head":
            norm_dim = dim_head
        else:
            raise ValueError(f"Normalization mode {self.qkv_norm_mode} not found, only support 'per_head'")

        self.backend = backend

        self.to_q = nn.Sequential(
            operations.Linear(query_dim, inner_dim, bias=qkv_bias, **weight_args),
            get_normalization(qkv_norm[0], norm_dim, weight_args=weight_args, operations=operations),
        )
        self.to_k = nn.Sequential(
            operations.Linear(context_dim, inner_dim, bias=qkv_bias, **weight_args),
            get_normalization(qkv_norm[1], norm_dim, weight_args=weight_args, operations=operations),
        )
        self.to_v = nn.Sequential(
            operations.Linear(context_dim, inner_dim, bias=qkv_bias, **weight_args),
            get_normalization(qkv_norm[2], norm_dim, weight_args=weight_args, operations=operations),
        )

        self.to_out = nn.Sequential(
            operations.Linear(inner_dim, query_dim, bias=out_bias, **weight_args),

View on GitHub (pinned to 1c6d8d45b3)

Solutions

1. Set qkv_norm_mode='per_head' (or omit it, as it is the default) for Cosmos models in ComfyUI
2. If the checkpoint truly needs per_channel norms, implement that branch in the Attention block or update ComfyUI to a version supporting it
3. Verify you are loading a Cosmos variant supported by your ComfyUI version

Example fix

# before
Attention(1024, qkv_norm_mode="per_channel", ...)

# after
Attention(1024, qkv_norm_mode="per_head", ...)

if qkv_norm_mode != "per_head":
    qkv_norm_mode = "per_head"  # only supported mode in ComfyUI's Cosmos

def is_supported_qkv_norm_mode(mode: str) -> bool:
    return mode == "per_head"

When it happens

Trigger: Constructing comfy.ldm.cosmos.blocks.Attention (directly or via a Cosmos model config) with qkv_norm_mode='per_channel' or any non-'per_head' string. Standard Cosmos checkpoints ComfyUI ships use per_head, so this arises from custom configs or ported variants.

Common situations: Porting configs from the upstream cosmos-predict repository, where per_channel normalization exists; writing custom video model configs; merging old config dicts after an API change.

Related errors

• Normalization {name} not found
• `only_cross_attention` can only be set to True if `added_kv_
• Unknown normalization type: {norm_type}
• Hidden size {params.hidden_size} must be divisible by num_he
• Hidden size {params.hidden_size} must be divisible by num_he

AI-assisted analysis of Comfy-Org/ComfyUI@1c6d8d45b3 (2026-08-14). Data as JSON: /api/errors/646071382f637d04. Report an issue: GitHub.