sgl-project/sglang · error · ValueError

Either quant_config or online_scheme must be provided

Error message

Either quant_config or online_scheme must be provided

What it means

QuarkConfig.__init__ requires either an explicit quant_config or a recognized online_scheme; when both are absent (quant_config is None and the online_scheme branch was never taken successfully), construction is rejected with this ValueError. It is a constructor precondition guarding against a useless, unconfigured instance.

Source

Thrown at python/sglang/srt/layers/quantization/quark/quark.py:317

        online_scheme: Optional[str] = None,
        dequantization_config: Optional[QuantizationConfig] = None,
        excluded_fp8_config: Optional[Fp8Config] = None,
    ):
        super().__init__()
        if kv_cache_group is None:
            kv_cache_group = []

        if online_scheme is not None:
            assert not is_prequantized
            if online_scheme == "quark_mxfp4":
                quant_config = self._create_online_mxfp4_config(
                    model_type=hf_config.model_type
                )
            else:
                raise ValueError(f"Unsupported online_scheme: {online_scheme}")

        if quant_config is None:
            raise ValueError("Either quant_config or online_scheme must be provided")

        self.online_scheme = online_scheme
        self.quant_config = quant_config
        self.kv_cache_group = kv_cache_group
        self.kv_cache_config = kv_cache_config
        self.pack_method = pack_method
        self.exclude_layers = cast(list[str], self.quant_config.get("exclude", []))
        # Both are consumed by _is_draft_layer(), which has to tell an appended
        # MTP/NextN draft layer from a target-model one. "No draft stack" is
        # spelled None as often as it is spelled absent -- ModelConfig defaults
        # the same field to None -- so coerce rather than let range() raise.
        self.num_hidden_layers = getattr(hf_config, "num_hidden_layers", None)
        self.num_nextn_predict_layers = int(
            getattr(hf_config, "num_nextn_predict_layers", 0) or 0
        )
        self.is_prequantized = is_prequantized
        self.dequantization_config = dequantization_config
        # Load-as-is FP8 config for excluded layers of a mixed-precision source

View on GitHub (pinned to 0132848349)

Solutions

1. Pass a quant_config dict from the checkpoint (e.g. the contents of quant_config.json / hf_config.quantization_config), or pass online_scheme="quark_mxfp4".
2. If loading from a checkpoint, verify the quantization_config section exists and is forwarded: QuarkConfig.from_config(quant_config=config, hf_config=hf_config, ...).
3. Pre-validate the checkpoint has a quant config before constructing (see validation code).

Example fix

# before
qc = QuarkConfig(hf_config=hf_config)  # raises ValueError

# after
qc = QuarkConfig(quant_config=raw_quant_config, hf_config=hf_config, kv_cache_group=None, kv_cache_config=None)

if quant_config is None and online_scheme not in ("quark_mxfp4",):
    raise ValueError("Must pass quant_config or online_scheme='quark_mxfp4'")
cfg = QuarkConfig(quant_config=quant_config, hf_config=hf_config, ...)

def has_quark_config_source(quant_config, online_scheme) -> bool:
    return quant_config is not None or online_scheme == "quark_mxfp4"

try:
    QuarkConfig(hf_config=hf_config, **kwargs)
except ValueError as e:
    if "quant_config or online_scheme" in str(e):
        kwargs["quant_config"] = load_checkpoint_quant_config(model_path)
    raise

When it happens

Trigger: Constructing QuarkConfig(hf_config=..., ...) passing neither quant_config nor online_scheme; or passing online_scheme=None/empty with no quant_config (from_config only reaches this when the checkpoint provided no usable quant config and no requantization method).

Common situations: Programmatic use of QuarkConfig by subclass/tooling that forgets to forward the quant config; checkpoints with missing or empty quant_config sections combined with a requantization_method of None; refactors that drop the config argument.

Understand the failure class

Background: "X is required", "must be set", "cannot be empty": the missing-required-config error family, from Vertex AI project/location to WeChat keys — this error's family across 18 libraries.

Related errors

• Online MXFP4 requantization from compressed-tensors NVFP4 ch
• MIXED_PRECISION layer group {tail!r} has inconsistent quant
• MIXED_PRECISION layer group {tail!r} uses unsupported quant
• Unsupported online_scheme: {online_scheme}
• MIXED_PRECISION checkpoint has no NVFP4 layers to requantize

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b380142eb635773c. Report an issue: GitHub.