sgl-project/sglang · error · NotImplementedError

Requantization into {config['requantization_method']} is not

Error message

Requantization into {config['requantization_method']} is not supported, from the original quant_method={config['quant_method']} and activation_scheme={config.get('activation_scheme')}.

What it means

from_config supports a limited set of requantization transitions, decided by the checkpoint's quant_method + activation_scheme versus the requested requantization_method. When the pair doesn't match any implemented branch (the code falls through all supported cases for config['requantization_method']), it raises NotImplementedError naming the original quant_method and activation_scheme so the user knows which transition is missing.

Source

Thrown at python/sglang/srt/layers/quantization/quark/quark.py:494

            # Pure FP8 source: every layer is requantized FP8 -> MXFP4.
            if (
                config.get("quant_method") == "fp8"
                and config.get("activation_scheme") == "dynamic"
            ):
                quant_config = QuarkConfig._create_online_mxfp4_config(
                    model_type=hf_config.model_type
                )
                dequantization_config = Fp8Config.from_config(config)
                return cls(
                    quant_config=quant_config,
                    hf_config=hf_config,
                    is_prequantized=False,
                    dequantization_config=dequantization_config,
                    online_scheme=config["requantization_method"],
                )

            raise NotImplementedError(
                f"Requantization into {config['requantization_method']} is not supported, "
                f"from the original quant_method={config['quant_method']} "
                f"and activation_scheme={config.get('activation_scheme')}."
            )

        if config["quant_method"] != "quark":
            raise ValueError(
                f"QuarkConfig.from_config invoked with non-quark quant_method "
                f"{config['quant_method']!r} but no requantization_method set."
            )

        export_config = config.get("export")
        if export_config is None:
            raise ValueError(
                "The export key should be included in "
                "the configurations of Quark quantized model"
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Serve the checkpoint with its native quantization (remove/blank requantization_method or don't force --quantization), letting the base quant_method path handle it.
2. Re-export the checkpoint using a supported transition (NVFP4 checkpoint + requantization to quark MXFP4).
3. Upgrade sglang to a release that implements the requested transition; check release notes for requantization support.

Example fix

# before
# quant_config.json: quant_method="fp8", activation_scheme="static", requantization_method="quark_mxfp4" -> raises

# after
# quant_config.json: quant_method="NVFP4", requantization_method="quark_mxfp4" (supported transition)

SUPPORTED_TRANSITIONS = {("NVFP4", None, "quark_mxfp4")}  # (quant_method, activation_scheme, requantization_method)
key = (config.get("quant_method"), config.get("activation_scheme"), config.get("requantization_method"))
if config.get("requantization_method") and key not in SUPPORTED_TRANSITIONS:
    raise RuntimeError(f"Unsupported requantization transition: {key}; serve natively")

def requantization_transition_supported(config: dict) -> bool:
    rm = config.get("requantization_method")
    return rm is None or (config.get("quant_method"), config.get("activation_scheme"), rm) in {
        ("NVFP4", None, "quark_mxfp4"),
    }

try:
    QuarkConfig.from_config(quant_config=config, hf_config=hf_config)
except NotImplementedError as e:
    if "Requantization into" in str(e):
        config = {k: v for k, v in config.items() if k != "requantization_method"}
        QuarkConfig.from_config(quant_config=config, hf_config=hf_config)
    else:
        raise

When it happens

Trigger: Loading a checkpoint whose quant_config sets requantization_method to something unsupported for its base quant_method/activation_scheme (e.g. requantization into MXFP4 from an FP8 static-scale checkpoint, or any combination outside the implemented NVFP4->quark_mxfp4 style transitions).

Common situations: Checkpoints produced by newer llm-compressor/quark exporters requesting transitions sglang hasn't implemented; hand-edited quant_config.json with a different requantization_method; version mismatch between exporter and sglang runtime.

Related errors

• Online MXFP4 requantization from compressed-tensors NVFP4 ch
• MIXED_PRECISION layer group {tail!r} has inconsistent quant
• MIXED_PRECISION layer group {tail!r} uses unsupported quant
• Unsupported online_scheme: {online_scheme}
• Either quant_config or online_scheme must be provided

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f1f9804caecb2412. Report an issue: GitHub.