{"record":{"id":"3f83de2c2922d1b1","repo":"sgl-project/sglang","slug":"quantize-and-serve-requires-modelopt-quantization","errorCode":null,"errorMessage":"quantize_and_serve requires ModelOpt quantization (set with --quantization {})","messagePattern":"quantize_and_serve requires ModelOpt quantization \\(set with --quantization (.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/configs/model_config.py","lineNumber":1489,"sourceCode":"    def _validate_quantize_and_serve_config(self):\n        \"\"\"Validate quantize_and_serve configuration.\"\"\"\n        if not self.quantize_and_serve:\n            return\n\n        # Check if ModelOpt quantization is specified\n        _MODELOPT_QUANTIZATION_METHODS = [\n            \"modelopt\",\n            \"modelopt_fp8\",\n            \"modelopt_fp4\",\n            \"nvfp4_online\",\n            \"modelopt_mixed\",\n        ]\n        modelopt_quantization_specified = (\n            self.quantization in _MODELOPT_QUANTIZATION_METHODS\n        )\n\n        if not modelopt_quantization_specified:\n            raise ValueError(\n                \"quantize_and_serve requires ModelOpt quantization (set with --quantization \"\n                f\"{{{', '.join(sorted(_MODELOPT_QUANTIZATION_METHODS))}}})\"\n            )\n\n        # quantize_and_serve is disabled due to compatibility issues\n        raise NotImplementedError(\n            \"quantize_and_serve functionality is currently disabled due to compatibility issues. \"\n            \"Please use the separate quantize-then-deploy workflow instead. \"\n            \"Step 1: Quantize and export model. \"\n            \"Step 2: Deploy the exported model.\"\n        )\n\n    # adapted from https://github.com/vllm-project/vllm/blob/v0.6.4.post1/vllm/config.py\n    def _verify_quantization(self) -> None:\n        supported_quantization = [*QUANTIZATION_METHODS]\n        rocm_supported_quantization = [\n            \"awq\",\n            \"gptq\",","sourceCodeStart":1471,"sourceCodeEnd":1507,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/configs/model_config.py#L1471-L1507","documentation":"ModelOpt's quantize_and_serve mode (on-the-fly PTQ during serving) is only wired for ModelOpt quantization methods. During ModelConfig.__init__, _validate_quantize_and_serve_config checks that self.quantization is one of the _MODELOPT_QUANTIZATION_METHODS and raises this ValueError if not.","triggerScenarios":"Launching the server with --quantize-and-serve (or enable_quantize_and_serve=True) while --quantization is unset or set to a non-ModelOpt method (e.g. fp8, awq, gptq). The check runs in ModelConfig.__init__ before any model loading.","commonSituations":"Enabling quantize_and_serve without reading the ModelOpt docs, assuming it works with any quantizer, or passing a generic quantization name like 'fp8' instead of a ModelOpt-specific one such as 'modelopt'.","solutions":["Set --quantization to one of the ModelOpt quantization methods printed in the error message, e.g. --quantization modelopt","If you don't need on-the-fly quantization, drop the --quantize-and-serve flag and use the standard quantize-then-deploy workflow","Check _MODELOPT_QUANTIZATION_METHODS in python/sglang/srt/configs/model_config.py for the exact accepted values in your sglang version"],"exampleFix":"# before\npython -m sglang.launch_server --model MODEL --quantize-and-serve\n# after\npython -m sglang.launch_server --model MODEL --quantize-and-serve --quantization modelopt","handlingStrategy":"validation","validationCode":"from sglang.srt.configs.model_config import _MODELOPT_QUANTIZATION_METHODS\nif server_args.quantize_and_serve and server_args.quantization not in _MODELOPT_QUANTIZATION_METHODS:\n    raise SystemExit(f\"--quantize-and-serve needs one of {sorted(_MODELOPT_QUANTIZATION_METHODS)})\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate quantize_and_serve + quantization pairs in your launch wrapper before starting the server","Keep the accepted ModelOpt method list pinned next to your deploy scripts"],"tags":["quantization","modelopt","config-validation","sglang"],"backgroundTag":"invalid-argument-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}