sgl-project/sglang · error · ValueError

Got {axes_dim} but expected positional dim {pe_dim}

Error message

Got {axes_dim} but expected positional dim {pe_dim}

What it means

After computing pe_dim = hidden_size // num_heads, the Hunyuan3D transformer requires sum(axes_dim) to equal pe_dim because the rotary/axial position embedding is split across axes and must exactly fill the per-head dimension. This ValueError fires in __init__ when the axes_dim list doesn't sum to that value.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/hunyuan3d.py:527

        self.num_heads = num_heads
        self.num_attention_heads = num_heads
        self.depth = depth
        self.depth_single_blocks = depth_single_blocks
        self.axes_dim = axes_dim
        self.theta = theta
        self.qkv_bias = qkv_bias
        self.time_factor = time_factor
        self.out_channels = self.in_channels
        self.num_channels_latents = self.in_channels
        self.guidance_embed = guidance_embed

        if hidden_size % num_heads != 0:
            raise ValueError(
                f"Hidden size {hidden_size} must be divisible by num_heads {num_heads}"
            )
        pe_dim = hidden_size // num_heads
        if sum(axes_dim) != pe_dim:
            raise ValueError(f"Got {axes_dim} but expected positional dim {pe_dim}")
        self.latent_in = nn.Linear(self.in_channels, self.hidden_size, bias=True)
        self.time_in = _FluxMLPEmbedder(in_dim=256, hidden_dim=self.hidden_size)
        self.cond_in = nn.Linear(context_in_dim, self.hidden_size)
        self.guidance_in = (
            _FluxMLPEmbedder(in_dim=256, hidden_dim=self.hidden_size)
            if guidance_embed
            else nn.Identity()
        )

        self.double_blocks = nn.ModuleList(
            [
                _FluxDoubleStreamBlock(
                    self.hidden_size,
                    self.num_heads,
                    mlp_ratio=mlp_ratio,
                    qkv_bias=qkv_bias,
                    supported_attention_backends=supported_attention_backends,
                )

View on GitHub (pinned to 0132848349)

Solutions

1. Set axes_dim so its entries sum to hidden_size//num_heads (e.g. for pe_dim=72 use [24,24,24])
2. If you changed num_heads, recompute axes_dim accordingly
3. Use the axes_dim from the official Hunyuan3D config for the checkpoint you load

Example fix

# before
axes_dim=[16, 56, 32]  # sums to 104, pe_dim=72

# after
axes_dim=[24, 24, 24]  # sums to 72 = 1152//16

pe_dim = hidden_size // num_heads
assert sum(axes_dim) == pe_dim, f'axes_dim {axes_dim} sums to {sum(axes_dim)}, expected {pe_dim}'

def axes_dim_valid(axes_dim, hidden_size, num_heads) -> bool:
    return sum(axes_dim) == hidden_size // num_heads

When it happens

Trigger: Configuring the model with axes_dim whose total doesn't equal hidden_size//num_heads, e.g. axes_dim=[16,56,32]=104 but pe_dim=72 for hidden_size=1152, num_heads=16.

Common situations: Changing num_heads (which changes pe_dim) without updating axes_dim; porting a 2D axes_dim config (two entries) to a 3D model that expects three entries summing differently.

Related errors

• unknown qk_norm: {qk_norm}. Should be one of None, 'layer_no
• unknown norm_type {norm_type}
• Unknown history_scale_mode: {history_scale_mode}
• Hidden size {hidden_size} must be divisible by num_heads {nu
• Got {config.rope_axes_dim} but expected positional dim {pe_d

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/005cd88395d95a14. Report an issue: GitHub.