sgl-project/sglang · error · RuntimeError

Can't get gguf config for {config.model_type}. Place a confi

Error message

Can't get gguf config for {config.model_type}. Place a config.json next to the .gguf file to load the config from there instead.

What it means

For a GGUF file without a sidecar config.json, sglang derives the config from transformers' model_type registry; the model's model_type is not in MODEL_FOR_CAUSAL_LM_MAPPING_NAMES so no architecture can be inferred. Place a config.json beside the .gguf to supply it.

Source

Thrown at python/sglang/srt/utils/hf_transformers/config.py:279

    parser = get_model_config_parser(model_config_parser)
    config = parser.parse(
        model, trust_remote_code=trust_remote_code, revision=revision, **kwargs
    )

    if model_override_args:
        # A plain update() setattrs a dict-valued override straight onto the
        # config, so '{"text_config": {...}}' on a VLM would replace the whole
        # sub-config with a dict and break attribute access downstream.
        for key, value in model_override_args.items():
            current = getattr(config, key, None)
            if isinstance(value, dict) and isinstance(current, PretrainedConfig):
                current.update(value)
            else:
                setattr(config, key, value)

    if is_gguf and not gguf_has_sidecar_config:
        if config.model_type not in MODEL_FOR_CAUSAL_LM_MAPPING_NAMES:
            raise RuntimeError(
                f"Can't get gguf config for {config.model_type}. Place a "
                "config.json next to the .gguf file to load the config from "
                "there instead."
            )
        _set_architectures(config, MODEL_FOR_CAUSAL_LM_MAPPING_NAMES[config.model_type])

    return config

View on GitHub (pinned to 0132848349)

Solutions

1. Put a valid config.json (with correct model_type/architectures) next to the .gguf file or in the HF repo
2. Upgrade transformers (and sglang) so the architecture is registered
3. Use an official GGUF upload that ships config.json

Example fix

# before
repo/
  model-Q4_K_M.gguf
# after
repo/
  model-Q4_K_M.gguf
  config.json   # {"model_type": "...", "architectures": ["...ForCausalLM"]}

from pathlib import Path
has_sidecar = Path(model_dir, 'config.json').is_file() if is_gguf else True

try:
    get_config(model)
except RuntimeError as e:
    if 'gguf config' in str(e): ship_or_point_to_sidecar_config()

When it happens

Trigger: Serving a GGUF of a model architecture unknown to the installed transformers version, without a config.json in the same directory/repo.

Common situations: Newly released architectures on older transformers/sglang versions, or hand-converted GGUF files that omit config metadata.

Related errors

• GGUFConfig must be constructed from a GGUF checkpoint
• model_config_parser={model_config_parser!r} is incompatible
• This browser cannot encode H.264 MP4
• H.264 encoder did not return MP4 decoder config
• This browser does not support gzip stream decoding

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/86c00ae21e8079b8. Report an issue: GitHub.