{"record":{"id":"1bb751efcb02ffcc","repo":"invoke-ai/InvokeAI","slug":"must-provide-the-same-number-of-num-attention-hea","errorCode":null,"errorMessage":"Must provide the same number of `num_attention_heads` as `down_block_types`. `num_attention_heads`: {num_attention_heads}. `down_block_types`: {down_block_types}.","messagePattern":"Must provide the same number of `num_attention_heads` as `down_block_types`\\. `num_attention_heads`: (.+?)\\. `down_block_types`: (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/util/hotfixes.py","lineNumber":174,"sourceCode":"        # Changing `attention_head_dim` to `num_attention_heads` for 40,000+ configurations is too backwards breaking\n        # which is why we correct for the naming here.\n        num_attention_heads = num_attention_heads or attention_head_dim\n\n        # Check inputs\n        if len(block_out_channels) != len(down_block_types):\n            raise ValueError(\n                f\"Must provide the same number of `block_out_channels` as `down_block_types`. \\\n                    `block_out_channels`: {block_out_channels}. `down_block_types`: {down_block_types}.\"\n            )\n\n        if not isinstance(only_cross_attention, bool) and len(only_cross_attention) != len(down_block_types):\n            raise ValueError(\n                f\"Must provide the same number of `only_cross_attention` as `down_block_types`. \\\n                    `only_cross_attention`: {only_cross_attention}. `down_block_types`: {down_block_types}.\"\n            )\n\n        if not isinstance(num_attention_heads, int) and len(num_attention_heads) != len(down_block_types):\n            raise ValueError(\n                f\"Must provide the same number of `num_attention_heads` as `down_block_types`. \\\n                    `num_attention_heads`: {num_attention_heads}. `down_block_types`: {down_block_types}.\"\n            )\n\n        if isinstance(transformer_layers_per_block, int):\n            transformer_layers_per_block = [transformer_layers_per_block] * len(down_block_types)\n\n        # input\n        conv_in_kernel = 3\n        conv_in_padding = (conv_in_kernel - 1) // 2\n        self.conv_in = nn.Conv2d(\n            in_channels, block_out_channels[0], kernel_size=conv_in_kernel, padding=conv_in_padding\n        )\n\n        # time\n        time_embed_dim = block_out_channels[0] * 4\n        self.time_proj = Timesteps(block_out_channels[0], flip_sin_to_cos, freq_shift)\n        timestep_input_dim = block_out_channels[0]","sourceCodeStart":156,"sourceCodeEnd":192,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/util/hotfixes.py#L156-L192","documentation":"The hotfixed ControlNetModel.__init__ falls back num_attention_heads = num_attention_heads or attention_head_dim for backwards compatibility (diffusers historically misnamed this parameter), then requires that a non-int num_attention_heads be a sequence of length equal to down_block_types — one head count per down block. A mismatched-length sequence cannot be zipped against the blocks, so a ValueError is raised.","triggerScenarios":"Passing num_attention_heads=(4, 8) to a 4-block ControlNetModel; passing a tuple/list head count whose length differs from down_block_types; confusion from the legacy attention_head_dim aliasing (if num_attention_heads is provided it overrides attention_head_dim and is then length-checked).","commonSituations":"Hand-editing old SD 1.5/SDXL ControlNet configs where attention_head_dim was renamed to num_attention_heads; copying num_attention_heads from a model with a different block count; mixing fields from two different model configs.","solutions":["Provide one value per down block: len(num_attention_heads) == len(down_block_types).","Pass a single int if all blocks share the same head count (the length check is skipped for ints).","Omit num_attention_heads entirely and set attention_head_dim instead, letting the legacy fallback apply it to all blocks.","Copy the field verbatim from the model's original config.json instead of transcribing it."],"exampleFix":"// before\nControlNetModel(\n    down_block_types=(\"CrossAttnDownBlock2D\", \"CrossAttnDownBlock2D\", \"DownBlock2D\", \"DownBlock2D\"),\n    num_attention_heads=(8, 8, 8),\n)\n// after\nControlNetModel(\n    down_block_types=(\"CrossAttnDownBlock2D\", \"CrossAttnDownBlock2D\", \"DownBlock2D\", \"DownBlock2D\"),\n    num_attention_heads=8,  # or (8, 8, 8, 8)\n)","handlingStrategy":"validation","validationCode":"def validate_num_attention_heads(config: dict) -> None:\n    nah = config.get('num_attention_heads')\n    n = len(config['down_block_types'])\n    if isinstance(nah, (list, tuple)) and len(nah) != n:\n        raise ValueError(\n            f'num_attention_heads ({len(nah)}) must match down_block_types ({n})'\n        )\n\nvalidate_num_attention_heads(config_dict)","typeGuard":"def num_attention_heads_is_valid(config: dict) -> bool:\n    nah = config.get('num_attention_heads')\n    if nah is None or isinstance(nah, int):\n        return True\n    return isinstance(nah, (list, tuple)) and len(nah) == len(config.get('down_block_types', []))","tryCatchPattern":"try:\n    model = ControlNetModel.from_config(config_dict)\nexcept ValueError as e:\n    if 'num_attention_heads' in str(e):\n        n = len(config_dict['down_block_types'])\n        nah = config_dict.get('num_attention_heads')\n        config_dict['num_attention_heads'] = list(nah)[:1] * n  # broadcast first value\n        model = ControlNetModel.from_config(config_dict)\n    else:\n        raise","preventionTips":["Pass a single int for num_attention_heads when all blocks share the same head count.","Remember the legacy alias: setting attention_head_dim alone is enough — omit num_attention_heads to avoid the length check.","Keep num_attention_heads and down_block_types in sync when editing configs programmatically.","Diff your config against the model's original config.json before instantiating."],"tags":["config-validation","diffusers","controlnet","model-config"],"backgroundTag":"config-list-length-mismatch","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}