sgl-project/sglang · error · NotImplementedError

quantize_and_serve functionality is currently disabled due t

Error message

quantize_and_serve functionality is currently disabled due to compatibility issues. Please use the separate quantize-then-deploy workflow instead. Step 1: Quantize and export model. Step 2: Deploy the exported model.

What it means

The quantize_and_serve code path is explicitly disabled in this sglang version because of compatibility issues; after validating the quantization method, _validate_quantize_and_serve_config unconditionally raises NotImplementedError directing users to the two-step workflow.

Source

Thrown at python/sglang/srt/configs/model_config.py:1495

        _MODELOPT_QUANTIZATION_METHODS = [
            "modelopt",
            "modelopt_fp8",
            "modelopt_fp4",
            "nvfp4_online",
            "modelopt_mixed",
        ]
        modelopt_quantization_specified = (
            self.quantization in _MODELOPT_QUANTIZATION_METHODS
        )

        if not modelopt_quantization_specified:
            raise ValueError(
                "quantize_and_serve requires ModelOpt quantization (set with --quantization "
                f"{{{', '.join(sorted(_MODELOPT_QUANTIZATION_METHODS))}}})"
            )

        # quantize_and_serve is disabled due to compatibility issues
        raise NotImplementedError(
            "quantize_and_serve functionality is currently disabled due to compatibility issues. "
            "Please use the separate quantize-then-deploy workflow instead. "
            "Step 1: Quantize and export model. "
            "Step 2: Deploy the exported model."
        )

    # adapted from https://github.com/vllm-project/vllm/blob/v0.6.4.post1/vllm/config.py
    def _verify_quantization(self) -> None:
        supported_quantization = [*QUANTIZATION_METHODS]
        rocm_supported_quantization = [
            "awq",
            "gptq",
            "fp8",
            "compressed_tensors",
            "compressed-tensors",
            "w8a8_fp8",
            "petit_nvfp4",
            "quark",

View on GitHub (pinned to 0132848349)

Solutions

1. Use the separate workflow: Step 1 quantize and export the model (e.g. via ModelOpt / sglang quantize export tooling), Step 2 serve the exported checkpoint with the matching --quantization
2. Pin to an older sglang release where quantize_and_serve was still enabled, if you must use it
3. Track the sglang repo for re-enablement of the feature

Example fix

# before
python -m sglang.launch_server --model MODEL --quantize-and-serve --quantization modelopt
# after
# step 1: quantize+export, step 2:
python -m sglang.launch_server --model ./exported-model --quantization modelopt-fp8

import sglang.srt.configs.model_config as mc, inspect
src = inspect.getsource(mc)
if 'currently disabled due to compatibility issues' in src:
    plan = 'quantize-export-then-deploy'  # skip quantize_and_serve

try:
    cfg = ModelConfig(model_path=..., quantization='modelopt', enable_quantize_and_serve=True)
except NotImplementedError:
    # fall back to serving an already-quantized export
    cfg = ModelConfig(model_path=EXPORTED_PATH, quantization='modelopt')

When it happens

Trigger: Any launch with quantize_and_serve enabled AND a valid ModelOpt --quantization value: the ValueError check passes and execution falls through to the unconditional raise NotImplementedError at model_config.py:1495.

Common situations: Following older tutorials or docs that demonstrated quantize_and_serve, or upgrading to a sglang version where the feature was disabled after it broke.

Related errors

• quantize_and_serve requires ModelOpt quantization (set with
• ModelOpt is not available. Please install modelopt.
• Failed to set up ModelOpt quantization: {e}
• Invalid quantization choice: '{quant_choice_str}'. Available
• ModelOpt quantization config '{quant_cfg_name}' not found. P

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/30ee52b8435e052a. Report an issue: GitHub.