sgl-project/sglang · error · ValueError

Invalid quantization choice: '{quant_choice_str}'. Available

Error message

Invalid quantization choice: '{quant_choice_str}'. Available choices: {list(QUANT_CFG_CHOICES.keys())}

What it means

The ModelOpt loader maps a quantization choice string (fp8/fp4 etc.) to a named config in QUANT_CFG_CHOICES; when the user-supplied string doesn't match any key it raises ValueError listing the valid choices. It means the value passed via the unified quantization flag (or model config override) is misspelled or unsupported.

Source

Thrown at python/sglang/srt/model_loader/loader.py:3944

            import modelopt.torch.quantization as mtq
        except ImportError:
            logger.error(
                "NVIDIA Model Optimizer (modelopt) library not found. "
                "Please install it to use ModelOpt quantization."
            )
            raise

        # Handle both old modelopt_quant and new unified quantization flags
        if hasattr(model_config, "modelopt_quant") and model_config.modelopt_quant:
            # Legacy modelopt_quant flag
            quant_choice_str = model_config.modelopt_quant
        else:
            # Unified quantization flag - extract the type (fp8/fp4)
            quant_choice_str = model_config._get_modelopt_quant_type()

        quant_cfg_name = QUANT_CFG_CHOICES.get(quant_choice_str)
        if not quant_cfg_name:
            raise ValueError(
                f"Invalid quantization choice: '{quant_choice_str}'. "
                f"Available choices: {list(QUANT_CFG_CHOICES.keys())}"
            )

        try:
            # getattr will fetch the config object, e.g., mtq.FP8_DEFAULT_CFG
            quant_cfg = getattr(mtq, quant_cfg_name)
        except AttributeError:
            raise AttributeError(
                f"ModelOpt quantization config '{quant_cfg_name}' not found. "
                "Please verify the ModelOpt library installation."
            )

        logger.info(
            f"Quantizing model with ModelOpt using config: mtq.{quant_cfg_name}"
        )

        # Get ModelOpt configuration from LoadConfig

View on GitHub (pinned to 0132848349)

Solutions

1. Check the error message — it lists valid choices; correct the flag to one of them (e.g. fp8 or fp4)
2. Inspect QUANT_CFG_CHOICES in python/sglang/srt/model_loader/loader.py for your sglang version
3. Update sglang — newer versions may support more choices
4. Ensure you're not double-specifying quantization via both model config and the flag

Example fix

# before
--quantization modelopt-fp8  # invalid choice

# after
--quantization fp8 --load-format modelopt  # pass bare choice per QUANT_CFG_CHOICES

from sglang.srt.model_loader.loader import QUANT_CFG_CHOICES
assert quant_choice in QUANT_CFG_CHOICES, f"pick one of {list(QUANT_CFG_CHOICES)}"

def is_valid_quant_choice(s: str) -> bool:
    from sglang.srt.model_loader.loader import QUANT_CFG_CHOICES
    return s in QUANT_CFG_CHOICES

try:
    launch(args)
except ValueError as e:
    if "Invalid quantization choice" in str(e):
        # parse listed choices from message and surface to user
        raise

When it happens

Trigger: Passing something like --quantization modelopt-fp9 or overriding modelopt quant type to an unknown value; the string returned by model_config._get_modelopt_quant_type() not being in QUANT_CFG_CHOICES (e.g. 'fp8', 'fp4' vs 'int8').

Common situations: Typos in quantization arg; using a newer/older sglang where the accepted choice set differs; copying flags from docs of a different version; passing a quant method name instead of the choice keyword.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

• quantize_and_serve requires ModelOpt quantization (set with
• quantize_and_serve functionality is currently disabled due t
• The hpc_ops MoE runner backend only supports FP8-quantized M
• ModelOpt is not available. Please install modelopt.
• Failed to set up ModelOpt quantization: {e}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/c80d5fe11f9317b6. Report an issue: GitHub.