Comfy-Org/ComfyUI · critical · ValueError

Hidden size {params.hidden_size} must be divisible by num_he

Error message

Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}

What it means

Chroma (Qwen-Image based flow-matching transformer) validates at construction that hidden_size is evenly divisible by num_heads, because attention splits the hidden dim into num_heads slices of hidden_size/num_heads each. A non-divisible pair would produce ragged heads, so the constructor refuses immediately with a ValueError. The parameters come from the ChromaParams dataclass filled from **kwargs, which are derived from the checkpoint's model config during detection.

Source

Thrown at comfy/ldm/chroma/model.py:62

    vec_in_dim: int



class Chroma(nn.Module):
    """
    Transformer model for flow matching on sequences.
    """

    def __init__(self, image_model=None, final_layer=True, dtype=None, device=None, operations=None, **kwargs):
        super().__init__()
        self.dtype = dtype
        params = ChromaParams(**kwargs)
        self.params = params
        self.patch_size = params.patch_size
        self.in_channels = params.in_channels
        self.out_channels = params.out_channels
        if params.hidden_size % params.num_heads != 0:
            raise ValueError(
                f"Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}"
            )
        pe_dim = params.hidden_size // params.num_heads
        if sum(params.axes_dim) != pe_dim:
            raise ValueError(f"Got {params.axes_dim} but expected positional dim {pe_dim}")
        self.hidden_size = params.hidden_size
        self.num_heads = params.num_heads
        self.in_dim = params.in_dim
        self.out_dim = params.out_dim
        self.hidden_dim = params.hidden_dim
        self.n_layers = params.n_layers
        self.pe_embedder = EmbedND(dim=pe_dim, theta=params.theta, axes_dim=params.axes_dim)
        self.img_in = operations.Linear(self.in_channels, self.hidden_size, bias=True, dtype=dtype, device=device)
        self.txt_in = operations.Linear(params.context_in_dim, self.hidden_size, dtype=dtype, device=device)
        # set as nn identity for now, will overwrite it later.
        self.distilled_guidance_layer = Approximator(
                    in_dim=self.in_dim,
                    hidden_dim=self.hidden_dim,

View on GitHub (pinned to 1c6d8d45b3)

Solutions

Inspect the checkpoint's saved config (num_heads, hidden_size) and confirm it matches an officially supported Chroma configuration
If building the model manually, set num_heads to a divisor of hidden_size (e.g. hidden_size // 64 head_dim)
Treat a mismatched pair read from a checkpoint as a sign of a corrupted or misidentified checkpoint and re-download it

Example fix

# before
Chroma(num_heads=24, hidden_size=3000, ...)  # ValueError: not divisible

# after
num_heads = params.hidden_size // 64  # derive from head_dim so divisibility always holds
Chroma(num_heads=num_heads, hidden_size=params.hidden_size, ...)

Defensive patterns

Strategy: validation

Validate before calling

def validate_chroma_params(hidden_size, num_heads, **_):
    if hidden_size % num_heads != 0:
        raise ValueError(f"{hidden_size} not divisible by {num_heads}; pick num_heads = hidden_size // head_dim")

Prevention

Derive num_heads from a fixed head_dim: num_heads = hidden_size // head_dim
Sanity-check checkpoint configs before loading: assert hidden_size % num_heads == 0

When it happens

Trigger: Constructing comfy.ldm.chroma.model.Chroma with kwargs where hidden_size % num_heads != 0 (e.g. hidden_size=3072, num_heads=24 is fine; hidden_size=3000, num_heads=24 is not). Typically happens when a chroma-radiance or custom-tuned checkpoint writes unusual num_heads into its config, or when a script builds Chroma manually with mismatched values.

Common situations: Loading a quantized/community Chroma or Chroma-Radiance checkpoint whose embedded num_heads differs from the official 3000/24-style config; writing a custom loader that overrides num_heads or hidden_size; config-detection code picking the wrong values for a non-standard checkpoint.

Related errors

AI-assisted analysis of Comfy-Org/ComfyUI@1c6d8d45b3 (2026-08-14). Data as JSON: /api/errors/f21d51606cf4bfaf. Report an issue: GitHub.