{"record":{"id":"340713ae4c1090a0","repo":"Comfy-Org/ComfyUI","slug":"hidden-size-hidden-size-must-be-divisible-by-num","errorCode":null,"errorMessage":"Hidden size {hidden_size} must be divisible by num_heads {num_heads}","messagePattern":"Hidden size (.+?) must be divisible by num_heads (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"comfy/ldm/hunyuan3d/model.py","lineNumber":34,"sourceCode":"        in_channels=64,\n        context_in_dim=1536,\n        hidden_size=1024,\n        mlp_ratio=4.0,\n        num_heads=16,\n        depth=16,\n        depth_single_blocks=32,\n        qkv_bias=True,\n        guidance_embed=False,\n        image_model=None,\n        dtype=None,\n        device=None,\n        operations=None\n    ):\n        super().__init__()\n        self.dtype = dtype\n\n        if hidden_size % num_heads != 0:\n            raise ValueError(\n                f\"Hidden size {hidden_size} must be divisible by num_heads {num_heads}\"\n            )\n\n        self.max_period = 1000  # While reimplementing the model I noticed that they messed up. This 1000 value was meant to be the time_factor but they set the max_period instead\n        self.latent_in = operations.Linear(in_channels, hidden_size, bias=True, dtype=dtype, device=device)\n        self.time_in = MLPEmbedder(in_dim=256, hidden_dim=hidden_size, dtype=dtype, device=device, operations=operations)\n        self.guidance_in = (\n            MLPEmbedder(in_dim=256, hidden_dim=hidden_size, dtype=dtype, device=device, operations=operations) if guidance_embed else None\n        )\n        self.cond_in = operations.Linear(context_in_dim, hidden_size, dtype=dtype, device=device)\n        self.double_blocks = nn.ModuleList(\n            [\n                DoubleStreamBlock(\n                    hidden_size,\n                    num_heads,\n                    mlp_ratio=mlp_ratio,\n                    qkv_bias=qkv_bias,\n                    dtype=dtype, device=device, operations=operations","sourceCodeStart":16,"sourceCodeEnd":52,"githubUrl":"https://github.com/Comfy-Org/ComfyUI/blob/1c6d8d45b3693bfbb32385b410d813a7fd6be216/comfy/ldm/hunyuan3d/model.py#L16-L52","documentation":"Raised in the Hunyuan3D flow-matching transformer constructor when the requested hidden_size is not divisible by num_heads. Multi-head attention needs head_dim = hidden_size // num_heads to evenly split the projection, so an inconsistent config fails fast at init instead of crashing later with a cryptic reshape error.","triggerScenarios":"Instantiating the model with hidden_size/num_heads pairs like 2048/12 or any config where hidden_size % num_heads != 0 (e.g. loading a modified config or overriding kwargs at build time).","commonSituations":"Hand-editing model config dicts, porting a checkpoint with nonstandard head counts, or typos when copying hyperparameters from a paper.","solutions":["Pick num_heads that divides hidden_size exactly (2048 -> 16 or 32 heads; 3072 -> 24 heads)","Use the model's canonical config values from the checkpoint instead of overriding kwargs","If a checkpoint demands an odd pairing, keep hidden_size and change num_heads so head_dim stays integral"],"exampleFix":"# before\nhidden_size = 2048; num_heads = 12  # 2048 % 12 != 0\n# after\nhidden_size = 2048; num_heads = 16  # head_dim = 128","handlingStrategy":"validation","validationCode":"assert hidden_size % num_heads == 0, f\"{hidden_size} not divisible by {num_heads}\"","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Centralize model config in dataclasses validated at load time","Copy canonical hyperparameters from the checkpoint, never from prose"],"tags":["hunyuan3d","config","attention","init"],"backgroundTag":null,"analyzedSha":"1c6d8d45b3693bfbb32385b410d813a7fd6be216","analyzedAt":"2026-08-14T19:37:18.893Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}