{"record":{"id":"b8e777fb416db577","repo":"sgl-project/sglang","slug":"quantconfig-has-static-quantization-but-found-act","errorCode":null,"errorMessage":"QuantConfig has static quantization, but found activation scales are None.","messagePattern":"QuantConfig has static quantization, but found activation scales are None\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8_moe.py","lineNumber":246,"sourceCode":"            )\n            layer.register_parameter(\"w13_input_scale\", w13_input_scale)\n            set_weight_attrs(w13_input_scale, extra_weight_attrs)\n\n            w2_input_scale = torch.nn.Parameter(\n                torch.ones(num_experts, dtype=torch.float32), requires_grad=False\n            )\n            layer.register_parameter(\"w2_input_scale\", w2_input_scale)\n            set_weight_attrs(w2_input_scale, extra_weight_attrs)\n        else:\n            layer.w13_input_scale = None\n            layer.w2_input_scale = None\n\n    def process_weights_after_loading(self, layer: torch.nn.Module | FusedMoE) -> None:\n        # Fp8 moe kernels require a single activation scale.\n        # We take the max of all the scales in case they differ.\n        if self.static_input_scales:\n            if layer.w13_input_scale is None or layer.w2_input_scale is None:\n                raise ValueError(\n                    \"QuantConfig has static quantization, but found \"\n                    \"activation scales are None.\"\n                )\n            if not all_close_1d(layer.w13_input_scale) or not all_close_1d(\n                layer.w2_input_scale\n            ):\n                logger.warning(\n                    \"Found input_scales that are not equal for \"\n                    \"fp8 MoE layer. Using the maximum across experts \"\n                    \"for each layer.\"\n                )\n            layer.w13_input_scale = torch.nn.Parameter(\n                layer.w13_input_scale.max(), requires_grad=False\n            )\n            layer.w2_input_scale = torch.nn.Parameter(\n                layer.w2_input_scale.max(), requires_grad=False\n            )\n","sourceCodeStart":228,"sourceCodeEnd":264,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8_moe.py#L228-L264","documentation":"The FP8 MoE config declares static input quantization (dynamic == False), but after loading, w13_input_scale or w2_input_scale is None. The fused kernels require a single activation scale, so the missing calibrated scales abort post-load processing.","triggerScenarios":"Checkpoint advertises static input scales but the input_scale tensors are absent from the safetensors weights, so layer.w13_input_scale/w2_input_scale are None in process_weights_after_loading.","commonSituations":"Quantization pipeline that wrote the config but failed to save activation scale tensors; truncated or mixed checkpoints; a conversion script dropping small scale tensors.","solutions":["Re-run quantization ensuring calibration scales are saved (input_scale keys present in safetensors)","Re-quantize with dynamic input quantization instead","Inspect the checkpoint files for missing *.input_scale tensors and re-export them"],"exampleFix":"# check: python -c \"from safetensors import safe_open; ...\" verify input_scale keys exist\n# fix: re-quantize with llmcompressor static input scales properly exported","handlingStrategy":"validation","validationCode":"from safetensors import safe_open\n# verify activation scales exist\nkeys = set()\nfor f in shard_files:\n    with safe_open(f, framework=\"pt\") as fh: keys.update(fh.keys())\nassert any(\"input_scale\" in k for k in keys) or cfg[\"input_quant\"].get(\"dynamic\", True)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Run a checkpoint integrity script listing expected scale tensors","Prefer dynamic input quant to avoid dependence on saved scales"],"tags":["quantization","fp8","moe","missing-scales","checkpoint-corruption"],"backgroundTag":"missing-quantization-scales","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}