{"record":{"id":"388bf9d83d3f9713","repo":"sgl-project/sglang","slug":"w4afp8-group-size-must-be-a-positive-integer-got","errorCode":null,"errorMessage":"W4AFP8 group_size must be a positive integer, got {group_size!r}.","messagePattern":"W4AFP8 group_size must be a positive integer, got (.+?)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/humming.py","lineNumber":235,"sourceCode":"            param_dtype=param_dtype,\n            num_experts=num_experts,\n            has_bias=has_bias,\n            pad_n_to_multiple=pad_n_to_multiple,\n            pad_k_to_multiple=pad_k_to_multiple,\n            stack_size=stack_size,\n        )\n\n\nclass _W4AFp8CheckpointWeightSchema(_CheckpointWeightSchema):\n    quant_method = \"w4afp8\"\n\n    def __init__(self, group_size: int = 128):\n        if (\n            not isinstance(group_size, int)\n            or isinstance(group_size, bool)\n            or group_size <= 0\n        ):\n            raise ValueError(\n                f\"W4AFP8 group_size must be a positive integer, got {group_size!r}.\"\n            )\n        self.group_size = group_size\n\n    def get_tensors_attrs(\n        self,\n        shape_n: int,\n        shape_k: int,\n        param_dtype: torch.dtype,\n        num_experts: int | None = None,\n        has_bias: bool = False,\n        stack_size: int = 1,\n    ) -> dict[str, dict[str, Any]]:\n        if shape_k % self.group_size != 0:\n            raise ValueError(\n                f\"W4AFP8 shape_k = {shape_k} must be divisible by group_size = \"\n                f\"{self.group_size}. Choose a tensor-parallel configuration whose \"\n                \"local K dimension preserves quantization groups.\"","sourceCodeStart":217,"sourceCodeEnd":253,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/humming.py#L217-L253","documentation":"The W4AFP8 Humming group-size constructor validates that group_size is a positive integer (rejecting bools, zero, negatives, floats, strings); invalid values come from the quantization config and fail fast in __init__.","triggerScenarios":"Constructing the Humming W4AFP8 schema with group_size=0, -1, 128.0, \"128\", or True — usually parsed from a model's quantization_config.json or an explicit config dict passed to from_config.","commonSituations":"Hand-edited quantization configs with float group sizes; YAML/JSON parsing turning 128 into a string; copy-paste of group_size: -1 (the per-channel convention from GPTQ configs) into a humming config.","solutions":["Set group_size to a positive integer such as 32, 64, or 128 in the quantization config","If per-channel scaling was intended, use the value the humming config format expects rather than -1","Validate/sanitize config values before passing them to from_config"],"exampleFix":"# before\n{\"quant_method\": \"humming\", \"w4afp8\": {\"group_size\": -1}}\n# after\n{\"quant_method\": \"humming\", \"w4afp8\": {\"group_size\": 128}}","handlingStrategy":"type-guard","validationCode":"gs = config.get(\"group_size\", 128)\nassert isinstance(gs, int) and not isinstance(gs, bool) and gs > 0, (\n    f\"group_size must be a positive int, got {gs!r}\")","typeGuard":"def is_valid_group_size(gs: object) -> TypeGuard[int]:\n    return isinstance(gs, int) and not isinstance(gs, bool) and gs > 0","tryCatchPattern":"try:\n    cfg = HummingW4AFP8Config(group_size=raw_gs)\nexcept ValueError as e:\n    if \"group_size\" in str(e):\n        cfg = HummingW4AFP8Config(group_size=128)  # sane default\n    else:\n        raise","preventionTips":["Never copy group_size: -1 from GPTQ configs into humming configs","Validate quantization config values against the schema before model load"],"tags":["humming","group-size","config-validation","quantization"],"backgroundTag":"invalid-configuration-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}