{"record":{"id":"646071382f637d04","repo":"Comfy-Org/ComfyUI","slug":"normalization-mode-self-qkv-norm-mode-not-found","errorCode":null,"errorMessage":"Normalization mode {self.qkv_norm_mode} not found, only support 'per_head'","messagePattern":"Normalization mode (.+?) not found, only support 'per_head'","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"comfy/ldm/cosmos/blocks.py","lineNumber":106,"sourceCode":"        weight_args={},\n        operations=None,\n    ) -> None:\n        super().__init__()\n\n        self.is_selfattn = context_dim is None  # self attention\n\n        inner_dim = dim_head * heads\n        context_dim = query_dim if context_dim is None else context_dim\n\n        self.heads = heads\n        self.dim_head = dim_head\n        self.qkv_norm_mode = qkv_norm_mode\n        self.qkv_format = qkv_format\n\n        if self.qkv_norm_mode == \"per_head\":\n            norm_dim = dim_head\n        else:\n            raise ValueError(f\"Normalization mode {self.qkv_norm_mode} not found, only support 'per_head'\")\n\n        self.backend = backend\n\n        self.to_q = nn.Sequential(\n            operations.Linear(query_dim, inner_dim, bias=qkv_bias, **weight_args),\n            get_normalization(qkv_norm[0], norm_dim, weight_args=weight_args, operations=operations),\n        )\n        self.to_k = nn.Sequential(\n            operations.Linear(context_dim, inner_dim, bias=qkv_bias, **weight_args),\n            get_normalization(qkv_norm[1], norm_dim, weight_args=weight_args, operations=operations),\n        )\n        self.to_v = nn.Sequential(\n            operations.Linear(context_dim, inner_dim, bias=qkv_bias, **weight_args),\n            get_normalization(qkv_norm[2], norm_dim, weight_args=weight_args, operations=operations),\n        )\n\n        self.to_out = nn.Sequential(\n            operations.Linear(inner_dim, query_dim, bias=out_bias, **weight_args),","sourceCodeStart":88,"sourceCodeEnd":124,"githubUrl":"https://github.com/Comfy-Org/ComfyUI/blob/1c6d8d45b3693bfbb32385b410d813a7fd6be216/comfy/ldm/cosmos/blocks.py#L88-L124","documentation":"Cosmos Attention.__init__ normalizes q/k/v with a per-head RMSNorm applied after rearranging channels into heads; qkv_norm_mode currently supports only 'per_head' (norm_dim = dim_head). Any other mode string (e.g. 'per_channel' from NVIDIA's original codebase) raises at construction because the norm dimension would be ambiguous. This is a config-surface check on the attention block parameters.","triggerScenarios":"Constructing comfy.ldm.cosmos.blocks.Attention (directly or via a Cosmos model config) with qkv_norm_mode='per_channel' or any non-'per_head' string. Standard Cosmos checkpoints ComfyUI ships use per_head, so this arises from custom configs or ported variants.","commonSituations":"Porting configs from the upstream cosmos-predict repository, where per_channel normalization exists; writing custom video model configs; merging old config dicts after an API change.","solutions":["Set qkv_norm_mode='per_head' (or omit it, as it is the default) for Cosmos models in ComfyUI","If the checkpoint truly needs per_channel norms, implement that branch in the Attention block or update ComfyUI to a version supporting it","Verify you are loading a Cosmos variant supported by your ComfyUI version"],"exampleFix":"# before\nAttention(1024, qkv_norm_mode=\"per_channel\", ...)\n\n# after\nAttention(1024, qkv_norm_mode=\"per_head\", ...)","handlingStrategy":"validation","validationCode":"if qkv_norm_mode != \"per_head\":\n    qkv_norm_mode = \"per_head\"  # only supported mode in ComfyUI's Cosmos","typeGuard":"def is_supported_qkv_norm_mode(mode: str) -> bool:\n    return mode == \"per_head\"","tryCatchPattern":null,"preventionTips":["Omit qkv_norm_mode (defaults to per_head) unless you know otherwise","Check Cosmos variant support in your ComfyUI version before loading exotic checkpoints"],"tags":["cosmos","attention","normalization","config"],"backgroundTag":null,"analyzedSha":"1c6d8d45b3693bfbb32385b410d813a7fd6be216","analyzedAt":"2026-08-14T19:37:18.893Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}