sgl-project/sglang · error · ValueError

W4AFP8 group_size must be a positive integer, got {group_siz

Error message

W4AFP8 group_size must be a positive integer, got {group_size!r}.

What it means

The W4AFP8 Humming group-size constructor validates that group_size is a positive integer (rejecting bools, zero, negatives, floats, strings); invalid values come from the quantization config and fail fast in __init__.

Source

Thrown at python/sglang/srt/layers/quantization/humming.py:235

            param_dtype=param_dtype,
            num_experts=num_experts,
            has_bias=has_bias,
            pad_n_to_multiple=pad_n_to_multiple,
            pad_k_to_multiple=pad_k_to_multiple,
            stack_size=stack_size,
        )


class _W4AFp8CheckpointWeightSchema(_CheckpointWeightSchema):
    quant_method = "w4afp8"

    def __init__(self, group_size: int = 128):
        if (
            not isinstance(group_size, int)
            or isinstance(group_size, bool)
            or group_size <= 0
        ):
            raise ValueError(
                f"W4AFP8 group_size must be a positive integer, got {group_size!r}."
            )
        self.group_size = group_size

    def get_tensors_attrs(
        self,
        shape_n: int,
        shape_k: int,
        param_dtype: torch.dtype,
        num_experts: int | None = None,
        has_bias: bool = False,
        stack_size: int = 1,
    ) -> dict[str, dict[str, Any]]:
        if shape_k % self.group_size != 0:
            raise ValueError(
                f"W4AFP8 shape_k = {shape_k} must be divisible by group_size = "
                f"{self.group_size}. Choose a tensor-parallel configuration whose "
                "local K dimension preserves quantization groups."

View on GitHub (pinned to 0132848349)

Solutions

1. Set group_size to a positive integer such as 32, 64, or 128 in the quantization config
2. If per-channel scaling was intended, use the value the humming config format expects rather than -1
3. Validate/sanitize config values before passing them to from_config

Example fix

# before
{"quant_method": "humming", "w4afp8": {"group_size": -1}}
# after
{"quant_method": "humming", "w4afp8": {"group_size": 128}}

gs = config.get("group_size", 128)
assert isinstance(gs, int) and not isinstance(gs, bool) and gs > 0, (
    f"group_size must be a positive int, got {gs!r}")

def is_valid_group_size(gs: object) -> TypeGuard[int]:
    return isinstance(gs, int) and not isinstance(gs, bool) and gs > 0

try:
    cfg = HummingW4AFP8Config(group_size=raw_gs)
except ValueError as e:
    if "group_size" in str(e):
        cfg = HummingW4AFP8Config(group_size=128)  # sane default
    else:
        raise

When it happens

Trigger: Constructing the Humming W4AFP8 schema with group_size=0, -1, 128.0, "128", or True — usually parsed from a model's quantization_config.json or an explicit config dict passed to from_config.

Common situations: Hand-edited quantization configs with float group sizes; YAML/JSON parsing turning 128 into a string; copy-paste of group_size: -1 (the per-channel convention from GPTQ configs) into a humming config.

Related errors

• kitchen_int8 group_size must be one of {_SUPPORTED_GROUP_SIZ
• W4AFP8 shape_k = {shape_k} must be divisible by group_size =
• FP8 weight_block_size must contain two positive integers, go
• Unsupported Comfy INT8 format for {prefix!r}: {marker.get('f
• Serialized kitchen_int8 layer {prefix!r} must declare convro

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/388bf9d83d3f9713. Report an issue: GitHub.