{"record":{"id":"a197d97a74d1c26b","repo":"invoke-ai/InvokeAI","slug":"hidden-size-params-hidden-size-must-be-divisible","errorCode":null,"errorMessage":"Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}","messagePattern":"Hidden size (.+?) must be divisible by num_heads (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"invokeai/backend/flux/controlnet/instantx_controlnet_flux.py","lineNumber":56,"sourceCode":"\n\nclass InstantXControlNetFlux(torch.nn.Module):\n    def __init__(self, params: FluxParams, num_control_modes: int | None = None):\n        \"\"\"\n        Args:\n            params (FluxParams): The parameters for the FLUX model.\n            num_control_modes (int | None, optional): The number of controlnet modes. If non-None, then the model is a\n                'union controlnet' model and expects a mode conditioning input at runtime.\n        \"\"\"\n        super().__init__()\n\n        # The following modules mirror the base FLUX transformer model.\n        # -------------------------------------------------------------\n        self.params = params\n        self.in_channels = params.in_channels\n        self.out_channels = self.in_channels\n        if params.hidden_size % params.num_heads != 0:\n            raise ValueError(f\"Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}\")\n        pe_dim = params.hidden_size // params.num_heads\n        if sum(params.axes_dim) != pe_dim:\n            raise ValueError(f\"Got {params.axes_dim} but expected positional dim {pe_dim}\")\n        self.hidden_size = params.hidden_size\n        self.num_heads = params.num_heads\n        self.pe_embedder = EmbedND(dim=pe_dim, theta=params.theta, axes_dim=params.axes_dim)\n        self.img_in = nn.Linear(self.in_channels, self.hidden_size, bias=True)\n        self.time_in = MLPEmbedder(in_dim=256, hidden_dim=self.hidden_size)\n        self.vector_in = MLPEmbedder(params.vec_in_dim, self.hidden_size)\n        self.guidance_in = (\n            MLPEmbedder(in_dim=256, hidden_dim=self.hidden_size) if params.guidance_embed else nn.Identity()\n        )\n        self.txt_in = nn.Linear(params.context_in_dim, self.hidden_size)\n\n        self.double_blocks = nn.ModuleList(\n            [\n                DoubleStreamBlock(\n                    self.hidden_size,","sourceCodeStart":38,"sourceCodeEnd":74,"githubUrl":"https://github.com/invoke-ai/InvokeAI/blob/0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06/invokeai/backend/flux/controlnet/instantx_controlnet_flux.py#L38-L74","documentation":"InstantX ControlNet FLUX mirrors the base FLUX transformer: attention head dimension is derived as hidden_size // num_heads. If hidden_size isn't divisible by num_heads, head splitting is impossible, so __init__ raises ValueError during module construction.","triggerScenarios":"Constructing FluxControlNetInstantXModel(params) with a FluxParams whose hidden_size % num_heads != 0, e.g. hidden_size=1280, num_heads=12.","commonSituations":"Hand-written model configs (YAML/JSON) mixing values from different FLUX variants; typos in hidden_size or num_heads; loading a config saved for another architecture into this class.","solutions":["Fix the config so hidden_size is divisible by num_heads (use the base FLUX values, e.g. hidden_size=3072, num_heads=24).","Copy params from the matching base FLUX transformer checkpoint rather than hand-authoring them.","Add a pre-construction config validation asserting hidden_size % num_heads == 0."],"exampleFix":"// before\nparams = FluxParams(hidden_size=1280, num_heads=12, ...)\nmodel = FluxControlNetInstantXModel(params)  # raises\n// after\nparams = FluxParams(hidden_size=3072, num_heads=24, ...)\nmodel = FluxControlNetInstantXModel(params)","handlingStrategy":"validation","validationCode":"assert params.hidden_size % params.num_heads == 0, f\"hidden_size={params.hidden_size} not divisible by num_heads={params.num_heads}\"","typeGuard":"def is_valid_flux_params(params) -> bool:\n    return params.hidden_size % params.num_heads == 0 and sum(params.axes_dim) == params.hidden_size // params.num_heads","tryCatchPattern":"try:\n    model = FluxControlNetInstantXModel(params)\nexcept ValueError as e:\n    if \"divisible by num_heads\" in str(e):\n        logger.error(\"Bad FLUX config: %s\", e)\n    raise","preventionTips":["Copy FluxParams from the loaded base transformer instead of hand-authoring configs.","Add a config lint that checks divisibility and axes_dim consistency.","Version-control known-good model configs per FLUX variant."],"tags":["pytorch","config","transformer","validation"],"backgroundTag":"invalid-parameter-range","analyzedSha":"0b6a024f2ff6a86bfb953dcdb9cc504ef7397a06","analyzedAt":"2026-08-29T04:46:49.967Z","schemaVersion":2},"datasetVersion":"2026-08-29T07:17:48.351Z"}