{"record":{"id":"afe0e4e5b61a2331","repo":"sgl-project/sglang","slug":"unknown-quantization-strategy-self-strategy","errorCode":null,"errorMessage":"Unknown quantization strategy {self.strategy}","messagePattern":"Unknown quantization strategy (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8.py","lineNumber":220,"sourceCode":"                layer.weight_scale.requires_grad_(False)\n\n            # On Blackwell, block-FP8 dispatches to DeepGEMM, which needs the\n            # weight scales UE8M0-packed to match its UE8M0 activation scales.\n            use_deepgemm_runner = (\n                self.w8a8_block_fp8_linear\n                is deepgemm_w8a8_block_fp8_linear_with_fallback\n            )\n            requant_block_scale_ue8m0_for_deepgemm(\n                layer.weight,\n                layer.weight_scale,\n                self.weight_block_size,\n                use_deepgemm_runner=use_deepgemm_runner,\n                output_dtype=getattr(layer, \"orig_dtype\", None),\n                weight_shape=layer.weight.shape,\n            )\n\n        else:\n            raise ValueError(f\"Unknown quantization strategy {self.strategy}\")\n\n        # INPUT SCALE\n        if self.is_static_input_scheme and hasattr(layer, \"input_scale\"):\n            layer.input_scale = Parameter(layer.input_scale.max(), requires_grad=False)\n        else:\n            layer.input_scale = None\n\n    def apply_weights(\n        self,\n        layer: torch.nn.Module,\n        x: torch.Tensor,\n        bias: Optional[torch.Tensor] = None,\n    ) -> torch.Tensor:\n        if isinstance(x, tuple):\n            # Pre-quantized activation from a fused RMSNorm+FP8 quant kernel:\n            # x = (fp8_input, per_tensor_input_scale[, orig_dtype]).\n            # apply_fp8_linear detects the fp8 dtype and skips re-quantizing;\n            # orig_dtype (when present) sets the GEMM output dtype.","sourceCodeStart":202,"sourceCodeEnd":238,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/layers/quantization/compressed_tensors/schemes/compressed_tensors_w8a8_fp8.py#L202-L238","documentation":"In process_weights_after_loading of the W8A8 FP8 scheme, only CHANNEL and TENSOR weight strategies have repacking branches; any other strategy reaches the fallthrough raise. It means the checkpoint's weight layout does not match a scheme the post-load path can process.","triggerScenarios":"Loading a compressed-tensors W8A8 model whose weight strategy is BLOCK or GROUP while the scheme was constructed for fp8 dynamic/static linear layers, then process_weights_after_loading dispatches on self.strategy.","commonSituations":"Mixed recipes (block weights + token activations) slipping through init checks; newer quant formats not understood by the installed SGLang version.","solutions":["Check quantization_config.weights.strategy in the checkpoint; use channel or tensor","If block quantization is desired, upgrade SGLang so the block-quantized FP8 path handles it","Re-quantize the model with a recipe matching W8A8 (channel weights + token dynamic activations)"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"assert cfg[\"quantization_config\"][\"weights\"][\"strategy\"] in (\"channel\", \"tensor\", \"block\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Pin SGLang and llmcompressor versions used to produce checkpoints","Smoke-test load quantized models in CI before deployment"],"tags":["quantization","fp8","w8a8","strategy","post-load"],"backgroundTag":"unsupported-quantization-strategy","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}