Comfy-Org/ComfyUI · error · ValueError

Hidden size {hidden_size} must be divisible by num_heads {nu

Error message

Hidden size {hidden_size} must be divisible by num_heads {num_heads}

What it means

Raised in the Hunyuan3D flow-matching transformer constructor when the requested hidden_size is not divisible by num_heads. Multi-head attention needs head_dim = hidden_size // num_heads to evenly split the projection, so an inconsistent config fails fast at init instead of crashing later with a cryptic reshape error.

Source

Thrown at comfy/ldm/hunyuan3d/model.py:34

        in_channels=64,
        context_in_dim=1536,
        hidden_size=1024,
        mlp_ratio=4.0,
        num_heads=16,
        depth=16,
        depth_single_blocks=32,
        qkv_bias=True,
        guidance_embed=False,
        image_model=None,
        dtype=None,
        device=None,
        operations=None
    ):
        super().__init__()
        self.dtype = dtype

        if hidden_size % num_heads != 0:
            raise ValueError(
                f"Hidden size {hidden_size} must be divisible by num_heads {num_heads}"
            )

        self.max_period = 1000  # While reimplementing the model I noticed that they messed up. This 1000 value was meant to be the time_factor but they set the max_period instead
        self.latent_in = operations.Linear(in_channels, hidden_size, bias=True, dtype=dtype, device=device)
        self.time_in = MLPEmbedder(in_dim=256, hidden_dim=hidden_size, dtype=dtype, device=device, operations=operations)
        self.guidance_in = (
            MLPEmbedder(in_dim=256, hidden_dim=hidden_size, dtype=dtype, device=device, operations=operations) if guidance_embed else None
        )
        self.cond_in = operations.Linear(context_in_dim, hidden_size, dtype=dtype, device=device)
        self.double_blocks = nn.ModuleList(
            [
                DoubleStreamBlock(
                    hidden_size,
                    num_heads,
                    mlp_ratio=mlp_ratio,
                    qkv_bias=qkv_bias,
                    dtype=dtype, device=device, operations=operations

View on GitHub (pinned to 1c6d8d45b3)

Solutions

1. Pick num_heads that divides hidden_size exactly (2048 -> 16 or 32 heads; 3072 -> 24 heads)
2. Use the model's canonical config values from the checkpoint instead of overriding kwargs
3. If a checkpoint demands an odd pairing, keep hidden_size and change num_heads so head_dim stays integral

Example fix

# before
hidden_size = 2048; num_heads = 12  # 2048 % 12 != 0
# after
hidden_size = 2048; num_heads = 16  # head_dim = 128

assert hidden_size % num_heads == 0, f"{hidden_size} not divisible by {num_heads}"

When it happens

Trigger: Instantiating the model with hidden_size/num_heads pairs like 2048/12 or any config where hidden_size % num_heads != 0 (e.g. loading a modified config or overriding kwargs at build time).

Common situations: Hand-editing model config dicts, porting a checkpoint with nonstandard head counts, or typos when copying hyperparameters from a paper.

Related errors

• Hidden size {params.hidden_size} must be divisible by num_he
• `only_cross_attention` can only be set to True if `added_kv_
• Hidden size {params.hidden_size} must be divisible by num_he
• Hidden size {params.hidden_size} must be divisible by num_he
• Normalization {name} not found

AI-assisted analysis of Comfy-Org/ComfyUI@1c6d8d45b3 (2026-08-14). Data as JSON: /api/errors/340713ae4c1090a0. Report an issue: GitHub.