invoke-ai/InvokeAI · error · ValueError

Hidden size {params.hidden_size} must be divisible by num_he

Error message

Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}

What it means

XLabsControlNetFlux.__init__ requires hidden_size to be evenly divisible by num_heads so the attention heads partition the hidden dimension exactly. A non-divisible pair would make multi-head attention projections malformed, so construction fails with this ValueError.

Source

Thrown at invokeai/backend/flux/controlnet/xlabs_controlnet_flux.py:36


class XLabsControlNetFlux(torch.nn.Module):
    """A ControlNet model for FLUX.

    The architecture is very similar to the base FLUX model, with the following differences:
    - A `controlnet_depth` parameter is passed to control the number of double_blocks that the ControlNet is applied to.
      In order to keep the ControlNet small, this is typically much less than the depth of the base FLUX model.
    - There is a set of `controlnet_blocks` that are applied to the output of each double_block.
    """

    def __init__(self, params: FluxParams, controlnet_depth: int = 2):
        super().__init__()

        self.params = params
        self.in_channels = params.in_channels
        self.out_channels = self.in_channels
        if params.hidden_size % params.num_heads != 0:
            raise ValueError(f"Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}")
        pe_dim = params.hidden_size // params.num_heads
        if sum(params.axes_dim) != pe_dim:
            raise ValueError(f"Got {params.axes_dim} but expected positional dim {pe_dim}")
        self.hidden_size = params.hidden_size
        self.num_heads = params.num_heads
        self.pe_embedder = EmbedND(dim=pe_dim, theta=params.theta, axes_dim=params.axes_dim)
        self.img_in = torch.nn.Linear(self.in_channels, self.hidden_size, bias=True)
        self.time_in = MLPEmbedder(in_dim=256, hidden_dim=self.hidden_size)
        self.vector_in = MLPEmbedder(params.vec_in_dim, self.hidden_size)
        self.guidance_in = (
            MLPEmbedder(in_dim=256, hidden_dim=self.hidden_size) if params.guidance_embed else torch.nn.Identity()
        )
        self.txt_in = torch.nn.Linear(params.context_in_dim, self.hidden_size)

        self.double_blocks = torch.nn.ModuleList(
            [
                DoubleStreamBlock(
                    self.hidden_size,

View on GitHub (pinned to 0b6a024f2f)

Solutions

Choose num_heads that divides hidden_size evenly (canonical Flux: hidden_size=3072, num_heads=24).
Restore the checkpoint's original hidden_size/num_heads pair from its config file.
Validate before constructing: assert params.hidden_size % params.num_heads == 0.
Regenerate params from the checkpoint config loader rather than hardcoding values.

Example fix

// before
FluxParams(hidden_size=3072, num_heads=20, ...)  # 3072 % 20 != 0
// after
FluxParams(hidden_size=3072, num_heads=24, ...)  # 3072 % 24 == 0

Defensive patterns

Strategy: validation

Validate before calling

assert params.hidden_size % params.num_heads == 0, (
    f"hidden_size={params.hidden_size} not divisible by num_heads={params.num_heads}")

Type guard

def has_valid_head_config(p) -> bool:
    return p.num_heads > 0 and p.hidden_size % p.num_heads == 0

Try / catch

try:
    controlnet = XLabsControlNetFlux(params=params)
except ValueError as e:
    if "divisible by num_heads" in str(e):
        params = replace(params, num_heads=pick_divisor(params.hidden_size))
        controlnet = XLabsControlNetFlux(params=params)
    else:
        raise

Prevention

Validate hidden_size/num_heads divisibility in your config-loading layer.
Copy both values together from the same checkpoint config — never mix sources.
Canonical Flux pair: hidden_size=3072, num_heads=24.

When it happens

Trigger: Constructing XLabsControlNetFlux(params=FluxParams(...)) where params.hidden_size % params.num_heads != 0 — e.g. hidden_size=3072 with num_heads=20, or a hand-tuned hidden size with the default head count.

Common situations: Editing FluxParams for a smaller/larger model variant; mis-transcribing config values from a checkpoint's json; copying a config between Flux variants (dev/schnell/XLabs) with inconsistent head counts.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/f545114ce9150596. Report an issue: GitHub.