sgl-project/sglang · error · ValueError

The input_size of down's weight = {intermediate_size_per_par

Error message

The input_size of down's weight = {intermediate_size_per_partition} is not divisible by weight quantization block_k = {block_k}.

What it means

Companion check to the block_n error: with TP > 1, the down projection's input size per partition (intermediate_size_per_partition) must be divisible by block_k for row-parallel FP8 block-quantized MoE weights to keep scale alignment.

Source

Thrown at python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8_moe.py:119

            layer.weight_block_size = self.weight_block_size
            tp_size = get_parallel().tp_size
            block_n, block_k = (
                self.weight_block_size[0],
                self.weight_block_size[1],
            )
            # NOTE: To ensure proper alignment of the block-wise quantization
            # scales, the output_size of the weights for both the gate and up
            # layers must be divisible by block_n.
            # Required by column parallel or enabling merged weights
            if intermediate_size_per_partition % block_n != 0:
                raise ValueError(
                    f"The output_size of gate's and up's weight = "
                    f"{intermediate_size_per_partition} is not divisible by "
                    f"weight quantization block_n = {block_n}."
                )
            if tp_size > 1 and intermediate_size_per_partition % block_k != 0:
                # Required by row parallel
                raise ValueError(
                    f"The input_size of down's weight = "
                    f"{intermediate_size_per_partition} is not divisible by "
                    f"weight quantization block_k = {block_k}."
                )

        w13_up_dim, w2_down_dim, weight_padded = get_moe_weight_sizes(
            intermediate_size_per_partition,
            is_aiter_moe=_use_aiter,
            is_concat=True,
            is_packed=False,
        )

        extra_weight_attrs.update(
            {"weight_padded": weight_padded},
        )

        # WEIGHTS
        w13_weight = torch.nn.Parameter(

View on GitHub (pinned to 0132848349)

Solutions

1. Adjust --tp so intermediate_size / tp is divisible by block_k
2. Re-quantize with a block size compatible with the sharding plan
3. Run with TP=1 if feasible

block_k = cfg["quantization_config"]["weights"]["block_size"][0]
assert tp_size == 1 or (2*inter//tp) % block_k == 0

When it happens

Trigger: Block-quantized FP8 MoE with tp_size > 1 and intermediate_size_per_partition % block_k != 0 in create_weights.

Common situations: Same as 4185: unusual TP degrees or intermediate sizes vs the checkpoint's weight_block_size.

Related errors

• The output_size of gate's and up's weight = {intermediate_si
• Weight output_partition_size = {output_partition_size} is no
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par
• Only block_quant=True is supported in Quark MXFP4 requantiza

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/707b590c9a3a6f3d. Report an issue: GitHub.