sgl-project/sglang · error · ValueError

This tool only supports ModelOpt diffusers FP8 exports (quan

Error message

This tool only supports ModelOpt diffusers FP8 exports (quant_method=modelopt).

What it means

quantization_config exists but its quant_method is not 'modelopt', so the export came from a different quantizer (awq, gptq, fp8 native, etc.) which this tool cannot process.

Source

Thrown at python/sglang/multimodal_gen/tools/build_modelopt_fp8_transformer.py:570

    model_type: str = "auto",
    keep_bf16_patterns: Sequence[str] | None = None,
    maxbound: float = FP8_E4M3_MAXBOUND,
    overwrite: bool = False,
) -> dict[str, int]:
    source_dir = _resolve_transformer_dir(modelopt_hf_dir)
    backbone_ckpt_path = _resolve_backbone_ckpt(modelopt_backbone_ckpt)
    base_dir = (
        _resolve_transformer_dir(base_transformer_dir) if base_transformer_dir else None
    )

    config = _load_config(source_dir)
    quant_config = config.get("quantization_config")
    if not isinstance(quant_config, dict):
        raise ValueError(
            "Expected a flat quantization_config dict in the ModelOpt export."
        )
    if quant_config.get("quant_method") != "modelopt":
        raise ValueError(
            "This tool only supports ModelOpt diffusers FP8 exports "
            "(quant_method=modelopt)."
        )

    source_weight_map_all, index_filename = _load_weight_map(source_dir)
    source_metadata = _load_first_shard_metadata(source_dir, source_weight_map_all)
    is_ltx2_export = _is_ltx2_x0_export(
        config=config,
        source_metadata=source_metadata,
        source_weight_map=source_weight_map_all,
    )
    class_name = config.get("_class_name")
    runtime_name_mapper = _get_runtime_module_name_mapper(
        model_type=model_type, class_name=class_name
    )
    ignore_patterns = list(quant_config.get("ignore", []) or [])
    patterns = list(
        get_default_keep_bf16_patterns(model_type=model_type, class_name=class_name)

View on GitHub (pinned to 0132848349)

Solutions

1. Use a checkpoint quantized with ModelOpt (quant_method == 'modelopt')
2. Or use the appropriate tool for the actual quantizer
3. Check quantization_config.quant_method in config.json before running

qc = json.load(open(cfg_path)).get("quantization_config", {})
assert qc.get("quant_method") == "modelopt", f"unsupported quantizer: {qc.get('quant_method')}"

def is_modelopt_export(cfg: dict) -> bool:
    qc = cfg.get("quantization_config")
    return isinstance(qc, dict) and qc.get("quant_method") == "modelopt"

When it happens

Trigger: Passing an AWQ/GPTQ/other-quantized checkpoint to the ModelOpt FP8 transformer builder.

Common situations: Mixing up checkpoint variants; a pipeline quantized with vLLM-native fp8 instead of ModelOpt.

Related errors

• Only per-tensor FP8 scales are supported for diffusion check
• Expected a flat quantization_config dict in the ModelOpt exp
• intermediate_size must be specified for scaled activation fu
• kv_scales supplied but unified_kv is {unified_kv.dtype}, exp
• int32-packed scale buffers require scale_ue8m0=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/d9991014a8b254e4. Report an issue: GitHub.