sgl-project/sglang · error · ValueError

Expected a flat quantization_config dict in the ModelOpt exp

Error message

Expected a flat quantization_config dict in the ModelOpt export.

What it means

The tool reads config.json from the source dir and requires quantization_config to be a flat dict; it is missing or not a dict (e.g. null, a list, or nested structure).

Source

Thrown at python/sglang/multimodal_gen/tools/build_modelopt_fp8_transformer.py:566

    modelopt_hf_dir: str,
    modelopt_backbone_ckpt: str,
    output_dir: str,
    base_transformer_dir: str | None = None,
    model_type: str = "auto",
    keep_bf16_patterns: Sequence[str] | None = None,
    maxbound: float = FP8_E4M3_MAXBOUND,
    overwrite: bool = False,
) -> dict[str, int]:
    source_dir = _resolve_transformer_dir(modelopt_hf_dir)
    backbone_ckpt_path = _resolve_backbone_ckpt(modelopt_backbone_ckpt)
    base_dir = (
        _resolve_transformer_dir(base_transformer_dir) if base_transformer_dir else None
    )

    config = _load_config(source_dir)
    quant_config = config.get("quantization_config")
    if not isinstance(quant_config, dict):
        raise ValueError(
            "Expected a flat quantization_config dict in the ModelOpt export."
        )
    if quant_config.get("quant_method") != "modelopt":
        raise ValueError(
            "This tool only supports ModelOpt diffusers FP8 exports "
            "(quant_method=modelopt)."
        )

    source_weight_map_all, index_filename = _load_weight_map(source_dir)
    source_metadata = _load_first_shard_metadata(source_dir, source_weight_map_all)
    is_ltx2_export = _is_ltx2_x0_export(
        config=config,
        source_metadata=source_metadata,
        source_weight_map=source_weight_map_all,
    )
    class_name = config.get("_class_name")
    runtime_name_mapper = _get_runtime_module_name_mapper(
        model_type=model_type, class_name=class_name

View on GitHub (pinned to 0132848349)

Solutions

1. Confirm you are pointing at a ModelOpt-quantized export
2. Re-run ModelOpt quantization so config.json contains quantization_config
3. If hand-editing config.json, restore the quantization_config dict

import json
qc = json.load(open(cfg_path)).get("quantization_config")
assert isinstance(qc, dict), "not a ModelOpt FP8 export"

def is_modelopt_export(cfg: dict) -> bool:
    qc = cfg.get("quantization_config")
    return isinstance(qc, dict) and qc.get("quant_method") == "modelopt"

When it happens

Trigger: Running the builder on an export whose config.json lacks quantization_config, or where it was written as a non-dict value.

Common situations: Pointing at a plain BF16 export instead of a ModelOpt FP8 export; config.json edited or generated by a tool that dropped the section.

Related errors

• This tool only supports ModelOpt diffusers FP8 exports (quan
• Could not resolve a transformer directory from: {path}
• Could not resolve backbone.pt from: {path}
• Expected an index file or a single safetensors shard in {mod
• Only per-tensor FP8 scales are supported for diffusion check

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/377813c377dbe19d. Report an issue: GitHub.