{"record":{"id":"f545114ce9150596","repo":"invoke-ai/InvokeAI","slug":"hidden-size-params-hidden-size-must-be-divisible-f54511","errorCode":null,"errorMessage":"Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}","messagePattern":"Hidden size (.+?) must be divisible by num_heads (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/flux/controlnet/xlabs_controlnet_flux.py","lineNumber":36,"sourceCode":"\n\nclass XLabsControlNetFlux(torch.nn.Module):\n    \"\"\"A ControlNet model for FLUX.\n\n    The architecture is very similar to the base FLUX model, with the following differences:\n    - A `controlnet_depth` parameter is passed to control the number of double_blocks that the ControlNet is applied to.\n      In order to keep the ControlNet small, this is typically much less than the depth of the base FLUX model.\n    - There is a set of `controlnet_blocks` that are applied to the output of each double_block.\n    \"\"\"\n\n    def __init__(self, params: FluxParams, controlnet_depth: int = 2):\n        super().__init__()\n\n        self.params = params\n        self.in_channels = params.in_channels\n        self.out_channels = self.in_channels\n        if params.hidden_size % params.num_heads != 0:\n            raise ValueError(f\"Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}\")\n        pe_dim = params.hidden_size // params.num_heads\n        if sum(params.axes_dim) != pe_dim:\n            raise ValueError(f\"Got {params.axes_dim} but expected positional dim {pe_dim}\")\n        self.hidden_size = params.hidden_size\n        self.num_heads = params.num_heads\n        self.pe_embedder = EmbedND(dim=pe_dim, theta=params.theta, axes_dim=params.axes_dim)\n        self.img_in = torch.nn.Linear(self.in_channels, self.hidden_size, bias=True)\n        self.time_in = MLPEmbedder(in_dim=256, hidden_dim=self.hidden_size)\n        self.vector_in = MLPEmbedder(params.vec_in_dim, self.hidden_size)\n        self.guidance_in = (\n            MLPEmbedder(in_dim=256, hidden_dim=self.hidden_size) if params.guidance_embed else torch.nn.Identity()\n        )\n        self.txt_in = torch.nn.Linear(params.context_in_dim, self.hidden_size)\n\n        self.double_blocks = torch.nn.ModuleList(\n            [\n                DoubleStreamBlock(\n                    self.hidden_size,","sourceCodeStart":18,"sourceCodeEnd":54,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/flux/controlnet/xlabs_controlnet_flux.py#L18-L54","documentation":"XLabsControlNetFlux.__init__ requires hidden_size to be evenly divisible by num_heads so the attention heads partition the hidden dimension exactly. A non-divisible pair would make multi-head attention projections malformed, so construction fails with this ValueError.","triggerScenarios":"Constructing XLabsControlNetFlux(params=FluxParams(...)) where params.hidden_size % params.num_heads != 0 — e.g. hidden_size=3072 with num_heads=20, or a hand-tuned hidden size with the default head count.","commonSituations":"Editing FluxParams for a smaller/larger model variant; mis-transcribing config values from a checkpoint's json; copying a config between Flux variants (dev/schnell/XLabs) with inconsistent head counts.","solutions":["Choose num_heads that divides hidden_size evenly (canonical Flux: hidden_size=3072, num_heads=24).","Restore the checkpoint's original hidden_size/num_heads pair from its config file.","Validate before constructing: assert params.hidden_size % params.num_heads == 0.","Regenerate params from the checkpoint config loader rather than hardcoding values."],"exampleFix":"// before\nFluxParams(hidden_size=3072, num_heads=20, ...)  # 3072 % 20 != 0\n// after\nFluxParams(hidden_size=3072, num_heads=24, ...)  # 3072 % 24 == 0","handlingStrategy":"validation","validationCode":"assert params.hidden_size % params.num_heads == 0, (\n    f\"hidden_size={params.hidden_size} not divisible by num_heads={params.num_heads}\")","typeGuard":"def has_valid_head_config(p) -> bool:\n    return p.num_heads > 0 and p.hidden_size % p.num_heads == 0","tryCatchPattern":"try:\n    controlnet = XLabsControlNetFlux(params=params)\nexcept ValueError as e:\n    if \"divisible by num_heads\" in str(e):\n        params = replace(params, num_heads=pick_divisor(params.hidden_size))\n        controlnet = XLabsControlNetFlux(params=params)\n    else:\n        raise","preventionTips":["Validate hidden_size/num_heads divisibility in your config-loading layer.","Copy both values together from the same checkpoint config — never mix sources.","Canonical Flux pair: hidden_size=3072, num_heads=24."],"tags":["config","validation","divisibility","flux"],"backgroundTag":"hidden-size-not-divisible-by-num-heads","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}