sgl-project/sglang · error · ValueError

adaln out_features mismatch: {out_features} != {expand_ratio

Error message

adaln out_features mismatch: {out_features} != {expand_ratio}*{arch.hidden_size}*{modality_num}

What it means

The AdaLN projection layer's out_features must equal expand_ratio * hidden_size * modality_num so per-modality modulation signals can be split evenly. A mismatch means the architecture config, expand ratio, or modality count are inconsistent with the instantiated linear layer.

Source

Thrown at python/sglang/multimodal_gen/runtime/models/dits/minimax_h3.py:1070

    Per block, three modalities each produce six H-wide vectors:
    [M, t_dim] -> [M, 3*6H] -> view(M*3, 6H) -> chunk(6).
    The final layer uses one modality and produces two H-wide vectors:
    [M, t_dim] -> [M, 2H] -> chunk(2).
    """

    def __init__(
        self,
        arch: MiniMaxH3DiTArchConfig,
        out_features: int,
        quant_config: QuantizationConfig | None,
        *,
        prefix: str,
        expand_ratio: int,
        modality_num: int,
    ) -> None:
        super().__init__()
        if out_features != expand_ratio * arch.hidden_size * modality_num:
            raise ValueError(
                "adaln out_features mismatch: "
                f"{out_features} != {expand_ratio}*{arch.hidden_size}*{modality_num}"
            )
        self.expand_ratio = expand_ratio
        self.modality_num = modality_num
        self.hidden_size = arch.hidden_size
        # Curve checkpoints store both the sampled curve and their reduced
        # AdaLN projections in FP32. Preserve that precision island to match
        # the published pruned implementation; these outputs intentionally do
        # not enter the BF16-only fused modulation kernels.
        params_dtype = _FP32_DTYPE if arch.adaln_curve_grid is not None else _BF16_DTYPE
        self.linear = ColumnParallelLinear(
            arch.time_embed_dim,
            out_features,
            bias=True,
            gather_output=False,
            params_dtype=params_dtype,
            quant_config=quant_config,

View on GitHub (pinned to 0132848349)

Solutions

1. Print/inspect arch.hidden_size, expand_ratio, modality_num and out_features to find the inconsistent one
2. Regenerate or fix the architecture config from the checkpoint's real dimensions
3. Align expand_ratio and modality_num with the values used to build the projection weights

Example fix

// before
AdaLN(arch, out_features=6144, expand_ratio=2, modality_num=3)  # hidden=1024 -> expects 6144 ok; with hidden=1280 expects 7680
// after
expected = expand_ratio * arch.hidden_size * modality_num
AdaLN(arch, out_features=expected, expand_ratio=expand_ratio, modality_num=modality_num)

expected = expand_ratio * arch.hidden_size * modality_num
assert out_features == expected, (out_features, expected)

def adaln_dims_consistent(out_features: int, arch, expand_ratio: int, modality_num: int) -> bool:
    return out_features == expand_ratio * arch.hidden_size * modality_num

When it happens

Trigger: Constructing the AdaLN module with an out_features value not equal to expand_ratio * arch.hidden_size * modality_num, typically from a mis-synced arch config or wrong expand_ratio/modality_num arguments.

Common situations: Porting a checkpoint with a different hidden_size than the code assumes, changing modality_num (adding/removing a modality branch) without updating out_features, or a config refactor that left stale values.

Related errors

• MiniMax-H3 checkpoint shards disagree on adaln_t_table shape
• --minimax-h3-adaln-online rebuilds AdaLN outputs from the sa
• img_position_ids must be [1, S, 3], got {list(img_position_i
• MiniMax H3 attention heads must be divisible by TP size: {ar
• MiniMax H3 pruned curve checkpoints cannot use a separate Ad

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f65aa9850d8e796d. Report an issue: GitHub.