sgl-project/sglang · error · ValueError

model_config_parser={model_config_parser!r} is incompatible

Error message

model_config_parser={model_config_parser!r} is incompatible with GGUF inputs; only 'hf' (or 'auto') is supported.

What it means

get_config rejects GGUF model inputs when model_config_parser is set to anything other than 'auto' or 'hf', because GGUF configs can only be parsed via the HF/transformers path (or a sidecar config.json).

Source

Thrown at python/sglang/srt/utils/hf_transformers/config.py:232

        return load_mistral_config(
            model, trust_remote_code=trust_remote_code, revision=revision
        )


@lru_cache_frozenset(maxsize=32)
def get_config(
    model: str,
    trust_remote_code: bool,
    revision: Optional[str] = None,
    model_override_args: Optional[dict] = None,
    model_config_parser: str = "auto",
    **kwargs,
):
    is_gguf = check_gguf_file(model)
    gguf_has_sidecar_config = False
    if is_gguf:
        if model_config_parser not in ("auto", "hf"):
            raise ValueError(
                f"model_config_parser={model_config_parser!r} is incompatible "
                "with GGUF inputs; only 'hf' (or 'auto') is supported."
            )
        _ensure_gguf_version()
        gguf_has_sidecar_config = gguf_sidecar_dir(model, "config.json") is not None
        if not gguf_has_sidecar_config and has_native_gguf_support(model):
            config = build_gguf_config(model)
            if model_override_args:
                config.update(model_override_args)
            return config
        if not gguf_has_sidecar_config:
            kwargs["gguf_file"] = model
        model = Path(model).parent
        # Skip auto-resolution for GGUF: the name-based Mistral heuristic
        # would misfire on the rewritten parent dir.
        model_config_parser = "hf"

    model = resolve_runai_obj_uri(model)

View on GitHub (pinned to 0132848349)

Solutions

1. Drop --model-config-parser or set it to hf/auto for GGUF models
2. If you meant to serve a Mistral-format model, point --model at the mistral model, not a GGUF repo

Example fix

# before
--model repo/model-Q4_K_M.gguf --model-config-parser mistral
# after
--model repo/model-Q4_K_M.gguf --model-config-parser hf

if model.endswith('.gguf') or '::' in model:
    assert model_config_parser in ('auto', 'hf')

When it happens

Trigger: Launching with a .gguf model path plus --model-config-parser mistral (or any non-hf value).

Common situations: Users carry over mistral-config flags from Mistral-format models to GGUF-quantized HF models, or set the parser globally in a wrapper script.

Related errors

• GGUFConfig must be constructed from a GGUF checkpoint
• {component_name!r} does not support an explicit quantization
• --swa-full-tokens-ratio should be in range (0, 1.0].
• Can't get gguf config for {config.model_type}. Place a confi
• This browser cannot encode H.264 MP4

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/a1bfbf03de40cbaf. Report an issue: GitHub.