sgl-project/sglang · error · ValueError

Weight output_partition_size = {output_partition_size} is no

Error message

Weight output_partition_size = {output_partition_size} is not divisible by weight quantization block_n = {block_n}.

What it means

Thrown when building FP8 block-quantized weights for a column-parallel or merged linear layer whose per-partition output size is not divisible by the quantization block_n size. The weight layout for block FP8 requires each shard's output dimension to be a multiple of the block size. It fires in create_weights before any tensor is allocated.

Source

Thrown at python/sglang/multimodal_gen/runtime/layers/quantization/fp8.py:169

            block_n, block_k = (
                self.quant_config.weight_block_size[0],
                self.quant_config.weight_block_size[1],
            )
            # Required by row parallel
            if tp_size > 1 and input_size // input_size_per_partition == tp_size:
                if input_size_per_partition % block_k != 0:
                    raise ValueError(
                        f"Weight input_size_per_partition = "
                        f"{input_size_per_partition} is not divisible by "
                        f"weight quantization block_k = {block_k}."
                    )
            # Required by column parallel or enabling merged weights
            if (
                tp_size > 1 and output_size // output_size_per_partition == tp_size
            ) or len(output_partition_sizes) > 1:
                for output_partition_size in output_partition_sizes:
                    if output_partition_size % block_n != 0:
                        raise ValueError(
                            f"Weight output_partition_size = "
                            f"{output_partition_size} is not divisible by "
                            f"weight quantization block_n = {block_n}."
                        )

        layer.logical_widths = output_partition_sizes
        layer.input_size_per_partition = input_size_per_partition
        layer.output_size_per_partition = output_size_per_partition
        layer.orig_dtype = params_dtype

        # WEIGHT
        weight_dtype = (
            torch.float8_e4m3fn
            if self.quant_config.is_checkpoint_fp8_serialized
            else params_dtype
        )

        weight = ModelWeightParameter(

View on GitHub (pinned to 0132848349)

Solutions

Use a tensor-parallel size that divides the output dimensions so each shard is a multiple of block_n
Check the model's per-head/fused output sizes (e.g. QKV partition sizes) and pick block_n from the checkpoint quant config that divides them
Avoid merging or use a config where each partition size is divisible by block_n
Requantize the checkpoint with block sizes compatible with the model dims

Example fix

# before
python -m sglang.launch_server --model m --tp 3 --quantization fp8  # shard 3381 % 128 != 0
# after
python -m sglang.launch_server --model m --tp 2 --quantization fp8  # shards divisible by 128

Defensive patterns

Strategy: validation

Validate before calling

block_n = quant_cfg.weight_block_size[1]
assert all(ps % block_n == 0 for ps in output_partition_sizes), f'partitions {output_partition_sizes} not aligned to block_n={block_n}'

Try / catch

try:
    layer.create_weights(...)
except ValueError as e:
    if 'not divisible by' in str(e):
        raise SystemExit(f'TP/layout incompatible with FP8 block_n: {e}') from e
    raise

Prevention

Assert output dims % block_n == 0 per shard before enabling TP
Prefer power-of-two TP degrees
Validate quant block size against model fused-projection sizes at startup

When it happens

Trigger: Using FP8 block quantization (block_n, e.g. 128) with tensor parallelism > 1 on a column-parallel layer, or a merged layer (e.g. QKV projection) where any individual output_partition_size % block_n != 0.

Common situations: Model hidden sizes or intermediate dims not multiples of 128/16 (e.g. odd fused QKV splits), running TP>1 on small models, or overriding quantization config block sizes that mismatch the model architecture.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4bb9be2ceb5ed710. Report an issue: GitHub.