{"record":{"id":"c80d5fe11f9317b6","repo":"sgl-project/sglang","slug":"invalid-quantization-choice-quant-choice-str","errorCode":null,"errorMessage":"Invalid quantization choice: '{quant_choice_str}'. Available choices: {list(QUANT_CFG_CHOICES.keys())}","messagePattern":"Invalid quantization choice: '(.+?)'\\. Available choices: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/model_loader/loader.py","lineNumber":3944,"sourceCode":"            import modelopt.torch.quantization as mtq\n        except ImportError:\n            logger.error(\n                \"NVIDIA Model Optimizer (modelopt) library not found. \"\n                \"Please install it to use ModelOpt quantization.\"\n            )\n            raise\n\n        # Handle both old modelopt_quant and new unified quantization flags\n        if hasattr(model_config, \"modelopt_quant\") and model_config.modelopt_quant:\n            # Legacy modelopt_quant flag\n            quant_choice_str = model_config.modelopt_quant\n        else:\n            # Unified quantization flag - extract the type (fp8/fp4)\n            quant_choice_str = model_config._get_modelopt_quant_type()\n\n        quant_cfg_name = QUANT_CFG_CHOICES.get(quant_choice_str)\n        if not quant_cfg_name:\n            raise ValueError(\n                f\"Invalid quantization choice: '{quant_choice_str}'. \"\n                f\"Available choices: {list(QUANT_CFG_CHOICES.keys())}\"\n            )\n\n        try:\n            # getattr will fetch the config object, e.g., mtq.FP8_DEFAULT_CFG\n            quant_cfg = getattr(mtq, quant_cfg_name)\n        except AttributeError:\n            raise AttributeError(\n                f\"ModelOpt quantization config '{quant_cfg_name}' not found. \"\n                \"Please verify the ModelOpt library installation.\"\n            )\n\n        logger.info(\n            f\"Quantizing model with ModelOpt using config: mtq.{quant_cfg_name}\"\n        )\n\n        # Get ModelOpt configuration from LoadConfig","sourceCodeStart":3926,"sourceCodeEnd":3962,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/model_loader/loader.py#L3926-L3962","documentation":"The ModelOpt loader maps a quantization choice string (fp8/fp4 etc.) to a named config in QUANT_CFG_CHOICES; when the user-supplied string doesn't match any key it raises ValueError listing the valid choices. It means the value passed via the unified quantization flag (or model config override) is misspelled or unsupported.","triggerScenarios":"Passing something like --quantization modelopt-fp9 or overriding modelopt quant type to an unknown value; the string returned by model_config._get_modelopt_quant_type() not being in QUANT_CFG_CHOICES (e.g. 'fp8', 'fp4' vs 'int8').","commonSituations":"Typos in quantization arg; using a newer/older sglang where the accepted choice set differs; copying flags from docs of a different version; passing a quant method name instead of the choice keyword.","solutions":["Check the error message — it lists valid choices; correct the flag to one of them (e.g. fp8 or fp4)","Inspect QUANT_CFG_CHOICES in python/sglang/srt/model_loader/loader.py for your sglang version","Update sglang — newer versions may support more choices","Ensure you're not double-specifying quantization via both model config and the flag"],"exampleFix":"# before\n--quantization modelopt-fp8  # invalid choice\n\n# after\n--quantization fp8 --load-format modelopt  # pass bare choice per QUANT_CFG_CHOICES","handlingStrategy":"validation","validationCode":"from sglang.srt.model_loader.loader import QUANT_CFG_CHOICES\nassert quant_choice in QUANT_CFG_CHOICES, f\"pick one of {list(QUANT_CFG_CHOICES)}\"","typeGuard":"def is_valid_quant_choice(s: str) -> bool:\n    from sglang.srt.model_loader.loader import QUANT_CFG_CHOICES\n    return s in QUANT_CFG_CHOICES","tryCatchPattern":"try:\n    launch(args)\nexcept ValueError as e:\n    if \"Invalid quantization choice\" in str(e):\n        # parse listed choices from message and surface to user\n        raise","preventionTips":["Derive the allowed set from QUANT_CFG_CHOICES instead of hardcoding","Validate CLI/config values in a pre-launch lint step","Version-pin sglang so documented choices stay valid"],"tags":["modelopt","quantization","invalid-argument","config-validation","sglang"],"backgroundTag":"invalid-enum-value","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}