sgl-project/sglang · error · ValueError

ModelOptMixedPrecisionConfig only supports MIXED_PRECISION c

Error message

ModelOptMixedPrecisionConfig only supports MIXED_PRECISION checkpoints.

What it means

Error "ModelOptMixedPrecisionConfig only supports MIXED_PRECISION checkpoints." thrown in sgl-project/sglang.

Source

Thrown at python/sglang/srt/layers/quantization/modelopt_quant.py:846

                    kv_cache_scheme.get("type") == "float"
                    and kv_cache_scheme.get("num_bits") == 4
                ):
                    kv_cache_quant_algo = "NVFP4"
                else:
                    kv_cache_quant_algo = "auto"
            else:
                kv_cache_quant_algo = config.get("kv_cache_quant_algo")
            exclude_modules = config.get("ignore", config.get("exclude_modules"))
            quantized_layers = config.get("quantized_layers", {})
        else:
            quantization_section = cls.get_from_keys(config, ["quantization"])
            quant_algo = quantization_section.get("quant_algo")
            kv_cache_quant_algo = quantization_section.get("kv_cache_quant_algo")
            exclude_modules = quantization_section.get("exclude_modules")
            quantized_layers = quantization_section.get("quantized_layers", {})

        if quant_algo != "MIXED_PRECISION":
            raise ValueError(
                "ModelOptMixedPrecisionConfig only supports MIXED_PRECISION checkpoints."
            )
        if not quantized_layers:
            raise ValueError(
                "MIXED_PRECISION quantization requires a non-empty quantized_layers map."
            )

        group_size = None
        for layer_info in quantized_layers.values():
            if layer_info.get("quant_algo", "").upper() in (
                "NVFP4",
                "W4A16_NVFP4",
            ):
                group_size = layer_info.get("group_size", 16)
                break
        if group_size is None:
            group_size = 16

View on GitHub (pinned to 0132848349)

When it happens

Trigger: Thrown at python/sglang/srt/layers/quantization/modelopt_quant.py:846 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/44a16e8e81055d13. Report an issue: GitHub.