invoke-ai/InvokeAI · error · ValueError

Must provide the same number of `num_attention_heads` as `do

Error message

Must provide the same number of `num_attention_heads` as `down_block_types`. `num_attention_heads`: {num_attention_heads}. `down_block_types`: {down_block_types}.

What it means

The hotfixed ControlNetModel.__init__ falls back num_attention_heads = num_attention_heads or attention_head_dim for backwards compatibility (diffusers historically misnamed this parameter), then requires that a non-int num_attention_heads be a sequence of length equal to down_block_types — one head count per down block. A mismatched-length sequence cannot be zipped against the blocks, so a ValueError is raised.

Source

Thrown at invokeai/backend/util/hotfixes.py:174

        # Changing `attention_head_dim` to `num_attention_heads` for 40,000+ configurations is too backwards breaking
        # which is why we correct for the naming here.
        num_attention_heads = num_attention_heads or attention_head_dim

        # Check inputs
        if len(block_out_channels) != len(down_block_types):
            raise ValueError(
                f"Must provide the same number of `block_out_channels` as `down_block_types`. \
                    `block_out_channels`: {block_out_channels}. `down_block_types`: {down_block_types}."
            )

        if not isinstance(only_cross_attention, bool) and len(only_cross_attention) != len(down_block_types):
            raise ValueError(
                f"Must provide the same number of `only_cross_attention` as `down_block_types`. \
                    `only_cross_attention`: {only_cross_attention}. `down_block_types`: {down_block_types}."
            )

        if not isinstance(num_attention_heads, int) and len(num_attention_heads) != len(down_block_types):
            raise ValueError(
                f"Must provide the same number of `num_attention_heads` as `down_block_types`. \
                    `num_attention_heads`: {num_attention_heads}. `down_block_types`: {down_block_types}."
            )

        if isinstance(transformer_layers_per_block, int):
            transformer_layers_per_block = [transformer_layers_per_block] * len(down_block_types)

        # input
        conv_in_kernel = 3
        conv_in_padding = (conv_in_kernel - 1) // 2
        self.conv_in = nn.Conv2d(
            in_channels, block_out_channels[0], kernel_size=conv_in_kernel, padding=conv_in_padding
        )

        # time
        time_embed_dim = block_out_channels[0] * 4
        self.time_proj = Timesteps(block_out_channels[0], flip_sin_to_cos, freq_shift)
        timestep_input_dim = block_out_channels[0]

View on GitHub (pinned to 0b6a024f2f)

Solutions

1. Provide one value per down block: len(num_attention_heads) == len(down_block_types).
2. Pass a single int if all blocks share the same head count (the length check is skipped for ints).
3. Omit num_attention_heads entirely and set attention_head_dim instead, letting the legacy fallback apply it to all blocks.
4. Copy the field verbatim from the model's original config.json instead of transcribing it.

Example fix

// before
ControlNetModel(
    down_block_types=("CrossAttnDownBlock2D", "CrossAttnDownBlock2D", "DownBlock2D", "DownBlock2D"),
    num_attention_heads=(8, 8, 8),
)
// after
ControlNetModel(
    down_block_types=("CrossAttnDownBlock2D", "CrossAttnDownBlock2D", "DownBlock2D", "DownBlock2D"),
    num_attention_heads=8,  # or (8, 8, 8, 8)
)

def validate_num_attention_heads(config: dict) -> None:
    nah = config.get('num_attention_heads')
    n = len(config['down_block_types'])
    if isinstance(nah, (list, tuple)) and len(nah) != n:
        raise ValueError(
            f'num_attention_heads ({len(nah)}) must match down_block_types ({n})'
        )

validate_num_attention_heads(config_dict)

def num_attention_heads_is_valid(config: dict) -> bool:
    nah = config.get('num_attention_heads')
    if nah is None or isinstance(nah, int):
        return True
    return isinstance(nah, (list, tuple)) and len(nah) == len(config.get('down_block_types', []))

try:
    model = ControlNetModel.from_config(config_dict)
except ValueError as e:
    if 'num_attention_heads' in str(e):
        n = len(config_dict['down_block_types'])
        nah = config_dict.get('num_attention_heads')
        config_dict['num_attention_heads'] = list(nah)[:1] * n  # broadcast first value
        model = ControlNetModel.from_config(config_dict)
    else:
        raise

When it happens

Trigger: Passing num_attention_heads=(4, 8) to a 4-block ControlNetModel; passing a tuple/list head count whose length differs from down_block_types; confusion from the legacy attention_head_dim aliasing (if num_attention_heads is provided it overrides attention_head_dim and is then length-checked).

Common situations: Hand-editing old SD 1.5/SDXL ControlNet configs where attention_head_dim was renamed to num_attention_heads; copying num_attention_heads from a model with a different block count; mixing fields from two different model configs.

Related errors

• Must provide the same number of `block_out_channels` as `dow
• Must provide the same number of `only_cross_attention` as `d
• {self.__class__} has the config param `addition_embed_type`
• `encoder_hid_dim` has to be defined when `encoder_hid_dim_ty
• encoder_hid_dim_type: {encoder_hid_dim_type} must be None, '

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/1bb751efcb02ffcc. Report an issue: GitHub.