sgl-project/sglang · error · AttributeError

ModelOpt quantization config '{quant_cfg_name}' not found. P

Error message

ModelOpt quantization config '{quant_cfg_name}' not found. Please verify the ModelOpt library installation.

What it means

After mapping the choice to a name like FP8_DEFAULT_CFG, the loader does getattr(mtq, quant_cfg_name) on the installed modelopt package; AttributeError means the installed modelopt version does not define that config symbol. This is a version skew between the sglang loader and the modelopt library.

Source

Thrown at python/sglang/srt/model_loader/loader.py:3953

        if hasattr(model_config, "modelopt_quant") and model_config.modelopt_quant:
            # Legacy modelopt_quant flag
            quant_choice_str = model_config.modelopt_quant
        else:
            # Unified quantization flag - extract the type (fp8/fp4)
            quant_choice_str = model_config._get_modelopt_quant_type()

        quant_cfg_name = QUANT_CFG_CHOICES.get(quant_choice_str)
        if not quant_cfg_name:
            raise ValueError(
                f"Invalid quantization choice: '{quant_choice_str}'. "
                f"Available choices: {list(QUANT_CFG_CHOICES.keys())}"
            )

        try:
            # getattr will fetch the config object, e.g., mtq.FP8_DEFAULT_CFG
            quant_cfg = getattr(mtq, quant_cfg_name)
        except AttributeError:
            raise AttributeError(
                f"ModelOpt quantization config '{quant_cfg_name}' not found. "
                "Please verify the ModelOpt library installation."
            )

        logger.info(
            f"Quantizing model with ModelOpt using config: mtq.{quant_cfg_name}"
        )

        # Get ModelOpt configuration from LoadConfig
        modelopt_config = self.load_config.modelopt_config
        quantized_ckpt_restore_path = (
            modelopt_config.checkpoint_restore_path if modelopt_config else None
        )
        quantized_ckpt_save_path = (
            modelopt_config.checkpoint_save_path if modelopt_config else None
        )
        export_path = modelopt_config.export_path if modelopt_config else None
        tokenizer = AutoTokenizer.from_pretrained(

View on GitHub (pinned to 0132848349)

Solutions

1. Upgrade nvidia-modelopt to the version matching your sglang release (check sglang requirements/modelopt Dockerfile)
2. Verify the symbol: python -c "import modelopt.torch.quantization as mtq; print(hasattr(mtq, 'FP8_DEFAULT_CFG'))"
3. If upgrading modelopt is not possible, downgrade/pin sglang to a version compatible with your modelopt
4. For FP4 quantization, use a modelopt build with NVFP4 support (recent versions / CUDA 12.8+)

Example fix

# before
pip install nvidia-modelopt==0.08  # no FP4 config
# AttributeError: ModelOpt quantization config 'NVFP4_DEFAULT_CFG' not found

# after
pip install -U nvidia-modelopt
python -c "import modelopt.torch.quantization as mtq; assert hasattr(mtq,'NVFP4_DEFAULT_CFG')"

import modelopt.torch.quantization as mtq
assert hasattr(mtq, quant_cfg_name), (
    f"installed modelopt lacks {quant_cfg_name}; upgrade nvidia-modelopt")

def modelopt_has_cfg(name: str) -> bool:
    import modelopt.torch.quantization as mtq
    return hasattr(mtq, name)

try:
    launch(args)
except AttributeError as e:
    if "quantization config" in str(e): print("pip install -U nvidia-modelopt"); raise

When it happens

Trigger: sglang expects e.g. mtq.FP8_DEFAULT_CFG or an NVFP4 config that the installed (typically older) nvidia-modelopt does not export; NVFP4 configs notably require recent modelopt builds.

Common situations: Old modelopt pinned in a Docker image; new sglang expecting new FP4/FP8 config names; mixing nightly sglang with stable modelopt or vice versa.

Related errors

• quantize_and_serve requires ModelOpt quantization (set with
• quantize_and_serve functionality is currently disabled due t
• ModelOpt is not available. Please install modelopt.
• Failed to set up ModelOpt quantization: {e}
• ModelOpt export functionality is not available. Please ensur

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b469e13d04bfdde6. Report an issue: GitHub.