sgl-project/sglang · error · ValueError

Unsupported online_scheme: {online_scheme}

Error message

Unsupported online_scheme: {online_scheme}

What it means

QuarkConfig.__init__ accepts an online_scheme that names the online requantization scheme to build when no explicit quant_config is given. Currently only "quark_mxfp4" is implemented; any other string (typos, future scheme names) hits the else branch and raises ValueError.

Source

Thrown at python/sglang/srt/layers/quantization/quark/quark.py:314

        kv_cache_config: Optional[dict[str, Any]] = None,
        pack_method: str = "reorder",
        is_prequantized: bool = False,
        online_scheme: Optional[str] = None,
        dequantization_config: Optional[QuantizationConfig] = None,
        excluded_fp8_config: Optional[Fp8Config] = None,
    ):
        super().__init__()
        if kv_cache_group is None:
            kv_cache_group = []

        if online_scheme is not None:
            assert not is_prequantized
            if online_scheme == "quark_mxfp4":
                quant_config = self._create_online_mxfp4_config(
                    model_type=hf_config.model_type
                )
            else:
                raise ValueError(f"Unsupported online_scheme: {online_scheme}")

        if quant_config is None:
            raise ValueError("Either quant_config or online_scheme must be provided")

        self.online_scheme = online_scheme
        self.quant_config = quant_config
        self.kv_cache_group = kv_cache_group
        self.kv_cache_config = kv_cache_config
        self.pack_method = pack_method
        self.exclude_layers = cast(list[str], self.quant_config.get("exclude", []))
        # Both are consumed by _is_draft_layer(), which has to tell an appended
        # MTP/NextN draft layer from a target-model one. "No draft stack" is
        # spelled None as often as it is spelled absent -- ModelConfig defaults
        # the same field to None -- so coerce rather than let range() raise.
        self.num_hidden_layers = getattr(hf_config, "num_hidden_layers", None)
        self.num_nextn_predict_layers = int(
            getattr(hf_config, "num_nextn_predict_layers", 0) or 0
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Use online_scheme="quark_mxfp4" (i.e. --quantization quark_mxfp4), the only supported scheme.
2. If the checkpoint asks for a different requantization method, serve it with its native quant config instead (pass quant_config, not online_scheme).
3. Upgrade sglang if a newer release implements the scheme named in the error.

Example fix

# before
QuarkConfig(hf_config=cfg, is_prequantized=False, dequantization_config=d, online_scheme="quark_fp8")

# after
QuarkConfig(hf_config=cfg, is_prequantized=False, dequantization_config=d, online_scheme="quark_mxfp4")

SUPPORTED_SCHEMES = {"quark_mxfp4"}
if online_scheme not in SUPPORTED_SCHEMES:
    raise ValueError(f"online_scheme must be one of {SUPPORTED_SCHEMES}, got {online_scheme!r}")
cfg = QuarkConfig(hf_config=hf_config, is_prequantized=False, dequantization_config=d, online_scheme=online_scheme)

def is_supported_online_scheme(scheme: str) -> bool:
    return scheme == "quark_mxfp4"

try:
    QuarkConfig(hf_config=hf_config, is_prequantized=False, dequantization_config=d, online_scheme=scheme)
except ValueError as e:
    if "Unsupported online_scheme" in str(e):
        scheme = "quark_mxfp4"  # or abort with clear message
    raise

When it happens

Trigger: Constructing QuarkConfig(online_scheme=...) (typically via from_config mapping config["requantization_method"] to online_scheme) with a value other than "quark_mxfp4", e.g. "quark_fp8" or "mxfp4".

Common situations: A checkpoint's requantization_method field contains a scheme sglang doesn't implement; user passes a custom --quantization value that is forwarded as online_scheme; version skew between checkpoint producer and sglang release.

Related errors

• MIXED_PRECISION layer group {tail!r} has inconsistent quant
• {error_msg}
• Online MXFP4 requantization from compressed-tensors NVFP4 ch
• MIXED_PRECISION layer group {tail!r} uses unsupported quant
• Either quant_config or online_scheme must be provided

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1110e72c6bada75d. Report an issue: GitHub.