{"record":{"id":"f21d51606cf4bfaf","repo":"Comfy-Org/ComfyUI","slug":"hidden-size-params-hidden-size-must-be-divisible","errorCode":null,"errorMessage":"Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}","messagePattern":"Hidden size (.+?) must be divisible by num_heads (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"critical","filePath":"comfy/ldm/chroma/model.py","lineNumber":62,"sourceCode":"    vec_in_dim: int\n\n\n\nclass Chroma(nn.Module):\n    \"\"\"\n    Transformer model for flow matching on sequences.\n    \"\"\"\n\n    def __init__(self, image_model=None, final_layer=True, dtype=None, device=None, operations=None, **kwargs):\n        super().__init__()\n        self.dtype = dtype\n        params = ChromaParams(**kwargs)\n        self.params = params\n        self.patch_size = params.patch_size\n        self.in_channels = params.in_channels\n        self.out_channels = params.out_channels\n        if params.hidden_size % params.num_heads != 0:\n            raise ValueError(\n                f\"Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}\"\n            )\n        pe_dim = params.hidden_size // params.num_heads\n        if sum(params.axes_dim) != pe_dim:\n            raise ValueError(f\"Got {params.axes_dim} but expected positional dim {pe_dim}\")\n        self.hidden_size = params.hidden_size\n        self.num_heads = params.num_heads\n        self.in_dim = params.in_dim\n        self.out_dim = params.out_dim\n        self.hidden_dim = params.hidden_dim\n        self.n_layers = params.n_layers\n        self.pe_embedder = EmbedND(dim=pe_dim, theta=params.theta, axes_dim=params.axes_dim)\n        self.img_in = operations.Linear(self.in_channels, self.hidden_size, bias=True, dtype=dtype, device=device)\n        self.txt_in = operations.Linear(params.context_in_dim, self.hidden_size, dtype=dtype, device=device)\n        # set as nn identity for now, will overwrite it later.\n        self.distilled_guidance_layer = Approximator(\n                    in_dim=self.in_dim,\n                    hidden_dim=self.hidden_dim,","sourceCodeStart":44,"sourceCodeEnd":80,"githubUrl":"https://github.com/Comfy-Org/ComfyUI/blob/1c6d8d45b3693bfbb32385b410d813a7fd6be216/comfy/ldm/chroma/model.py#L44-L80","documentation":"Chroma (Qwen-Image based flow-matching transformer) validates at construction that hidden_size is evenly divisible by num_heads, because attention splits the hidden dim into num_heads slices of hidden_size/num_heads each. A non-divisible pair would produce ragged heads, so the constructor refuses immediately with a ValueError. The parameters come from the ChromaParams dataclass filled from **kwargs, which are derived from the checkpoint's model config during detection.","triggerScenarios":"Constructing comfy.ldm.chroma.model.Chroma with kwargs where hidden_size % num_heads != 0 (e.g. hidden_size=3072, num_heads=24 is fine; hidden_size=3000, num_heads=24 is not). Typically happens when a chroma-radiance or custom-tuned checkpoint writes unusual num_heads into its config, or when a script builds Chroma manually with mismatched values.","commonSituations":"Loading a quantized/community Chroma or Chroma-Radiance checkpoint whose embedded num_heads differs from the official 3000/24-style config; writing a custom loader that overrides num_heads or hidden_size; config-detection code picking the wrong values for a non-standard checkpoint.","solutions":["Inspect the checkpoint's saved config (num_heads, hidden_size) and confirm it matches an officially supported Chroma configuration","If building the model manually, set num_heads to a divisor of hidden_size (e.g. hidden_size // 64 head_dim)","Treat a mismatched pair read from a checkpoint as a sign of a corrupted or misidentified checkpoint and re-download it"],"exampleFix":"# before\nChroma(num_heads=24, hidden_size=3000, ...)  # ValueError: not divisible\n\n# after\nnum_heads = params.hidden_size // 64  # derive from head_dim so divisibility always holds\nChroma(num_heads=num_heads, hidden_size=params.hidden_size, ...)","handlingStrategy":"validation","validationCode":"def validate_chroma_params(hidden_size, num_heads, **_):\n    if hidden_size % num_heads != 0:\n        raise ValueError(f\"{hidden_size} not divisible by {num_heads}; pick num_heads = hidden_size // head_dim\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Derive num_heads from a fixed head_dim: num_heads = hidden_size // head_dim","Sanity-check checkpoint configs before loading: assert hidden_size % num_heads == 0"],"tags":["chroma","qwen-image","attention","config","model-init"],"backgroundTag":null,"analyzedSha":"1c6d8d45b3693bfbb32385b410d813a7fd6be216","analyzedAt":"2026-08-14T19:37:18.893Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}