sgl-project/sglang · error · ValueError

Weight output_partition_size = {output_partition_size} is no

Error message

Weight output_partition_size = {output_partition_size} is not divisible by weight quantization block_n = {block_n}.

What it means

For column-parallel or merged-weight FP8 block-quant layers, each output partition size must be divisible by block_n so per-block weight scales align with partition boundaries. validate_block_quant_shapes raises when any output_partition_size % block_n != 0.

Source

Thrown at python/sglang/srt/layers/quantization/fp8.py:519

                "Skipping block quantization checks for weight partition."
            )
        else:
            tp_size = get_parallel().tp_size
            # Required by row parallel
            if tp_size > 1 and input_size // input_size_per_partition == tp_size:
                if input_size_per_partition % block_k != 0:
                    raise ValueError(
                        f"Weight input_size_per_partition = "
                        f"{input_size_per_partition} is not divisible by "
                        f"weight quantization block_k = {block_k}."
                    )
            # Required by column parallel or enabling merged weights
            if (
                tp_size > 1 and output_size // output_size_per_partition == tp_size
            ) or len(output_partition_sizes) > 1:
                for output_partition_size in output_partition_sizes:
                    if output_partition_size % block_n != 0:
                        raise ValueError(
                            f"Weight output_partition_size = "
                            f"{output_partition_size} is not divisible by "
                            f"weight quantization block_n = {block_n}."
                        )

    @staticmethod
    def create_fp8_weight_(
        layer: torch.nn.Module,
        block_quant: bool,
        quant_config,
        use_mxfp8: bool,
        output_size_per_partition: int,
        input_size_per_partition: int,
        output_partition_sizes: List[int],
        input_size: int,
        output_size: int,
        params_dtype: torch.dtype,
        weight_loader,

View on GitHub (pinned to 0132848349)

Solutions

1. Choose a TP size such that every output partition is a multiple of 128 (check head counts and intermediate sizes)
2. Use TP=1 or a power-of-two TP that evenly divides the model's dimensions
3. Verify the checkpoint's weight_block_size; some models use [128,128] and need dimensions divisible accordingly

Example fix

# before
--tp 3  # head partition = 4096/3 not multiple of 128
# after
--tp 2  # each partition divisible by block_n

block_n = 128
for part in output_partition_sizes:  # e.g. [num_heads/tp*head_dim, ...]
    assert part % block_n == 0, f"partition {part} not divisible by {block_n}; change --tp-size"

def partitions_ok(sizes: list[int], block_n: int = 128) -> bool:
    return all(s % block_n == 0 for s in sizes)

When it happens

Trigger: Creating merged column-parallel weights (e.g. gate+up QKV fused layers) where an individual partition size (intermediate_size/tp, num_heads*head_dim/tp) is not a multiple of block_n=128 under tp_size>1 or when len(output_partition_sizes)>1.

Common situations: MoE intermediate sizes or attention head configurations not divisible into 128-multiples after TP sharding; unusual TP sizes; models with head_dim like 80 or intermediate sizes like 1536 with TP>1.

Related errors

• Weight input_size_per_partition = {input_size_per_partition}
• The output_size of gate's and up's weight = {intermediate_si
• The input_size of down's weight = {intermediate_size_per_par
• Weight input_size_per_partition = {input_size_per_partition}
• Weight output_partition_size = {output_partition_size} is no

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/20a66a422e31e9bd. Report an issue: GitHub.