sgl-project/sglang · error · ValueError

quantize_and_serve requires ModelOpt quantization (set with

Error message

quantize_and_serve requires ModelOpt quantization (set with --quantization {})

What it means

ModelOpt's quantize_and_serve mode (on-the-fly PTQ during serving) is only wired for ModelOpt quantization methods. During ModelConfig.__init__, _validate_quantize_and_serve_config checks that self.quantization is one of the _MODELOPT_QUANTIZATION_METHODS and raises this ValueError if not.

Source

Thrown at python/sglang/srt/configs/model_config.py:1489

    def _validate_quantize_and_serve_config(self):
        """Validate quantize_and_serve configuration."""
        if not self.quantize_and_serve:
            return

        # Check if ModelOpt quantization is specified
        _MODELOPT_QUANTIZATION_METHODS = [
            "modelopt",
            "modelopt_fp8",
            "modelopt_fp4",
            "nvfp4_online",
            "modelopt_mixed",
        ]
        modelopt_quantization_specified = (
            self.quantization in _MODELOPT_QUANTIZATION_METHODS
        )

        if not modelopt_quantization_specified:
            raise ValueError(
                "quantize_and_serve requires ModelOpt quantization (set with --quantization "
                f"{{{', '.join(sorted(_MODELOPT_QUANTIZATION_METHODS))}}})"
            )

        # quantize_and_serve is disabled due to compatibility issues
        raise NotImplementedError(
            "quantize_and_serve functionality is currently disabled due to compatibility issues. "
            "Please use the separate quantize-then-deploy workflow instead. "
            "Step 1: Quantize and export model. "
            "Step 2: Deploy the exported model."
        )

    # adapted from https://github.com/vllm-project/vllm/blob/v0.6.4.post1/vllm/config.py
    def _verify_quantization(self) -> None:
        supported_quantization = [*QUANTIZATION_METHODS]
        rocm_supported_quantization = [
            "awq",
            "gptq",

View on GitHub (pinned to 0132848349)

Solutions

1. Set --quantization to one of the ModelOpt quantization methods printed in the error message, e.g. --quantization modelopt
2. If you don't need on-the-fly quantization, drop the --quantize-and-serve flag and use the standard quantize-then-deploy workflow
3. Check _MODELOPT_QUANTIZATION_METHODS in python/sglang/srt/configs/model_config.py for the exact accepted values in your sglang version

Example fix

# before
python -m sglang.launch_server --model MODEL --quantize-and-serve
# after
python -m sglang.launch_server --model MODEL --quantize-and-serve --quantization modelopt

from sglang.srt.configs.model_config import _MODELOPT_QUANTIZATION_METHODS
if server_args.quantize_and_serve and server_args.quantization not in _MODELOPT_QUANTIZATION_METHODS:
    raise SystemExit(f"--quantize-and-serve needs one of {sorted(_MODELOPT_QUANTIZATION_METHODS)})")

When it happens

Trigger: Launching the server with --quantize-and-serve (or enable_quantize_and_serve=True) while --quantization is unset or set to a non-ModelOpt method (e.g. fp8, awq, gptq). The check runs in ModelConfig.__init__ before any model loading.

Common situations: Enabling quantize_and_serve without reading the ModelOpt docs, assuming it works with any quantizer, or passing a generic quantization name like 'fp8' instead of a ModelOpt-specific one such as 'modelopt'.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

• Invalid quantization choice: '{quant_choice_str}'. Available
• quantize_and_serve functionality is currently disabled due t
• The hpc_ops MoE runner backend only supports FP8-quantized M
• ModelOpt is not available. Please install modelopt.
• Failed to set up ModelOpt quantization: {e}

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3f83de2c2922d1b1. Report an issue: GitHub.