Comfy-Org/ComfyUI · error · ValueError

Hidden size {params.hidden_size} must be divisible by num_he

Error message

Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}

What it means

Flux's transformer __init__ derives the per-head dimension as hidden_size // num_heads and requires hidden_size to be exactly divisible by num_heads. A non-divisible pair leaves remainder channels that cannot be split across heads, so construction raises ValueError. Both values come from FluxParams (checkpoint config or Flux constructor kwargs).

Source

Thrown at comfy/ldm/flux/model.py:77

    return result


class Flux(nn.Module):
    """
    Transformer model for flow matching on sequences.
    """

    def __init__(self, image_model=None, final_layer=True, dtype=None, device=None, operations=None, **kwargs):
        super().__init__()
        self.dtype = dtype
        params = FluxParams(**kwargs)
        self.params = params
        self.patch_size = params.patch_size
        self.in_channels = params.in_channels * params.patch_size * params.patch_size
        self.out_channels = params.out_channels * params.patch_size * params.patch_size
        if params.hidden_size % params.num_heads != 0:
            raise ValueError(
                f"Hidden size {params.hidden_size} must be divisible by num_heads {params.num_heads}"
            )
        pe_dim = params.hidden_size // params.num_heads
        if sum(params.axes_dim) != pe_dim:
            raise ValueError(f"Got {params.axes_dim} but expected positional dim {pe_dim}")
        self.hidden_size = params.hidden_size
        self.num_heads = params.num_heads
        self.pe_embedder = EmbedND(dim=pe_dim, theta=params.theta, axes_dim=params.axes_dim)
        self.img_in = operations.Linear(self.in_channels, self.hidden_size, bias=params.ops_bias, dtype=dtype, device=device)
        self.time_in = MLPEmbedder(in_dim=256, hidden_dim=self.hidden_size, bias=params.ops_bias, dtype=dtype, device=device, operations=operations)
        if params.vec_in_dim is not None:
            self.vector_in = MLPEmbedder(params.vec_in_dim, self.hidden_size, dtype=dtype, device=device, operations=operations)
        else:
            self.vector_in = None

        self.guidance_in = (
            MLPEmbedder(in_dim=256, hidden_dim=self.hidden_size, bias=params.ops_bias, dtype=dtype, device=device, operations=operations) if params.guidance_embed else nn.Identity()
        )

View on GitHub (pinned to 1c6d8d45b3)

Solutions

1. Choose num_heads that divides hidden_size exactly (e.g. for 3072: 24, 16, 12, 8; for 1152: 18, 16, 12).
2. Restore the stock Flux config values (hidden_size 3072, num_heads 24 for dev/schnell base layers) instead of mixing custom values.
3. If a checkpoint truly has a non-divisible pair, its attention layout is nonstandard and unsupported here — do not try to pad hidden_size to force it.

Example fix

# before
Flux(hidden_size=3072, num_heads=28, ...)

# after
Flux(hidden_size=3072, num_heads=24, ...)

assert params.hidden_size % params.num_heads == 0, "hidden_size must be divisible by num_heads"

def valid_flux_head_config(hidden_size: int, num_heads: int) -> bool:
    return num_heads > 0 and hidden_size % num_heads == 0

When it happens

Trigger: Creating Flux(params...) with a custom config where hidden_size is not a multiple of num_heads (e.g. hidden_size=1152 with num_heads=16 is fine, but 1150/16 or 3072/28 is not); passing partially-overridden kwargs that break the pairing.

Common situations: Custom experiments that resize hidden_size without adjusting num_heads; merging config dicts where one field is updated and the other left stale; community checkpoints with nonstandard head counts.

Related errors

• Got {params.axes_dim} but expected positional dim {pe_dim}
• `only_cross_attention` can only be set to True if `added_kv_
• Hidden size {params.hidden_size} must be divisible by num_he
• Hidden size {params.hidden_size} must be divisible by num_he
• Normalization {name} not found

AI-assisted analysis of Comfy-Org/ComfyUI@1c6d8d45b3 (2026-08-14). Data as JSON: /api/errors/9319f9d88c085453. Report an issue: GitHub.