sgl-project/sglang · error · ValueError

Weight output_size_per_partition = {output_size_per_partitio

Error message

Weight output_size_per_partition = {output_size_per_partition} is not divisible by pack_factor = {self.quant_config.pack_factor}.

What it means

After the min_n_threads check, create_weights also requires output_size_per_partition divisible by the config's pack_factor (int32 words pack multiple sub-byte weights, so local N must align to the packing width). A TP shard that passed the thread check but breaks packing alignment raises this ValueError.

Source

Thrown at python/sglang/srt/layers/quantization/marlin_utils.py:768

    ):
        del output_size  # Unused.
        weight_loader = extra_weight_attrs["weight_loader"]

        if params_dtype != torch.float16:
            raise ValueError(
                f"The params dtype must be float16, but got {params_dtype}"
            )

        # Validate output_size_per_partition
        output_size_per_partition = sum(output_partition_sizes)
        if output_size_per_partition % self.quant_config.min_n_threads != 0:
            raise ValueError(
                f"Weight output_size_per_partition = "
                f"{output_size_per_partition} is not divisible by "
                f"min_n_threads = {self.quant_config.min_n_threads}."
            )
        if output_size_per_partition % self.quant_config.pack_factor != 0:
            raise ValueError(
                f"Weight output_size_per_partition = "
                f"{output_size_per_partition} is not divisible by "
                f"pack_factor = {self.quant_config.pack_factor}."
            )

        # Validate input_size_per_partition
        if input_size_per_partition % self.quant_config.min_k_threads != 0:
            raise ValueError(
                f"Weight input_size_per_partition = "
                f"{input_size_per_partition} is not divisible by "
                f"min_k_threads = {self.quant_config.min_k_threads}."
            )
        if (
            self.quant_config.group_size != -1
            and input_size_per_partition % self.quant_config.group_size != 0
        ):
            raise ValueError(
                f"Weight input_size_per_partition = "

View on GitHub (pinned to 0132848349)

Solutions

Use a power-of-two tensor_parallel_size dividing the output dimension cleanly
Check each output_partition_size individually for pack_factor alignment in custom layers
Fall back to --quantization gptq if the shard layout can't change

Example fix

# before
--tensor-parallel-size 3  # local N 4096/... misaligned to pack_factor
# after
--tensor-parallel-size 4

Defensive patterns

Strategy: validation

Validate before calling

assert output_size_per_partition % pack_factor == 0, "local N breaks Marlin packing"

Prevention

Ensure each output_partition_size is individually pack-aligned in fused layers
Prefer power-of-two TP

When it happens

Trigger: TP sharding where local N is divisible by min_n_threads but not by pack_factor (e.g. pack_factor 8 with local N ≡ 4 mod 8); mis-computed output_partition_sizes for fused/moe layers.

Common situations: Odd TP degrees again; custom layers passing concatenated partition sizes that don't individually align.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a9931dd97ff9cd5d. Report an issue: GitHub.