{"record":{"id":"30ee52b8435e052a","repo":"sgl-project/sglang","slug":"quantize-and-serve-functionality-is-currently-disa","errorCode":null,"errorMessage":"quantize_and_serve functionality is currently disabled due to compatibility issues. Please use the separate quantize-then-deploy workflow instead. Step 1: Quantize and export model. Step 2: Deploy the exported model.","messagePattern":"quantize_and_serve functionality is currently disabled due to compatibility issues\\. Please use the separate quantize-then-deploy workflow instead\\. Step 1: Quantize and export model\\. Step 2: Deploy the exported model\\.","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/configs/model_config.py","lineNumber":1495,"sourceCode":"        _MODELOPT_QUANTIZATION_METHODS = [\n            \"modelopt\",\n            \"modelopt_fp8\",\n            \"modelopt_fp4\",\n            \"nvfp4_online\",\n            \"modelopt_mixed\",\n        ]\n        modelopt_quantization_specified = (\n            self.quantization in _MODELOPT_QUANTIZATION_METHODS\n        )\n\n        if not modelopt_quantization_specified:\n            raise ValueError(\n                \"quantize_and_serve requires ModelOpt quantization (set with --quantization \"\n                f\"{{{', '.join(sorted(_MODELOPT_QUANTIZATION_METHODS))}}})\"\n            )\n\n        # quantize_and_serve is disabled due to compatibility issues\n        raise NotImplementedError(\n            \"quantize_and_serve functionality is currently disabled due to compatibility issues. \"\n            \"Please use the separate quantize-then-deploy workflow instead. \"\n            \"Step 1: Quantize and export model. \"\n            \"Step 2: Deploy the exported model.\"\n        )\n\n    # adapted from https://github.com/vllm-project/vllm/blob/v0.6.4.post1/vllm/config.py\n    def _verify_quantization(self) -> None:\n        supported_quantization = [*QUANTIZATION_METHODS]\n        rocm_supported_quantization = [\n            \"awq\",\n            \"gptq\",\n            \"fp8\",\n            \"compressed_tensors\",\n            \"compressed-tensors\",\n            \"w8a8_fp8\",\n            \"petit_nvfp4\",\n            \"quark\",","sourceCodeStart":1477,"sourceCodeEnd":1513,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/configs/model_config.py#L1477-L1513","documentation":"The quantize_and_serve code path is explicitly disabled in this sglang version because of compatibility issues; after validating the quantization method, _validate_quantize_and_serve_config unconditionally raises NotImplementedError directing users to the two-step workflow.","triggerScenarios":"Any launch with quantize_and_serve enabled AND a valid ModelOpt --quantization value: the ValueError check passes and execution falls through to the unconditional raise NotImplementedError at model_config.py:1495.","commonSituations":"Following older tutorials or docs that demonstrated quantize_and_serve, or upgrading to a sglang version where the feature was disabled after it broke.","solutions":["Use the separate workflow: Step 1 quantize and export the model (e.g. via ModelOpt / sglang quantize export tooling), Step 2 serve the exported checkpoint with the matching --quantization","Pin to an older sglang release where quantize_and_serve was still enabled, if you must use it","Track the sglang repo for re-enablement of the feature"],"exampleFix":"# before\npython -m sglang.launch_server --model MODEL --quantize-and-serve --quantization modelopt\n# after\n# step 1: quantize+export, step 2:\npython -m sglang.launch_server --model ./exported-model --quantization modelopt-fp8","handlingStrategy":"fallback","validationCode":"import sglang.srt.configs.model_config as mc, inspect\nsrc = inspect.getsource(mc)\nif 'currently disabled due to compatibility issues' in src:\n    plan = 'quantize-export-then-deploy'  # skip quantize_and_serve","typeGuard":null,"tryCatchPattern":"try:\n    cfg = ModelConfig(model_path=..., quantization='modelopt', enable_quantize_and_serve=True)\nexcept NotImplementedError:\n    # fall back to serving an already-quantized export\n    cfg = ModelConfig(model_path=EXPORTED_PATH, quantization='modelopt')","preventionTips":["Don't build production pipelines on quantize_and_serve; use explicit export + deploy","Watch release notes for feature-disable announcements"],"tags":["quantization","modelopt","not-implemented","feature-disabled","sglang"],"backgroundTag":"unsupported-feature-disabled","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}