{"record":{"id":"86c00ae21e8079b8","repo":"sgl-project/sglang","slug":"can-t-get-gguf-config-for-config-model-type-pla","errorCode":null,"errorMessage":"Can't get gguf config for {config.model_type}. Place a config.json next to the .gguf file to load the config from there instead.","messagePattern":"Can't get gguf config for (.+?)\\. Place a config\\.json next to the \\.gguf file to load the config from there instead\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/hf_transformers/config.py","lineNumber":279,"sourceCode":"    parser = get_model_config_parser(model_config_parser)\n    config = parser.parse(\n        model, trust_remote_code=trust_remote_code, revision=revision, **kwargs\n    )\n\n    if model_override_args:\n        # A plain update() setattrs a dict-valued override straight onto the\n        # config, so '{\"text_config\": {...}}' on a VLM would replace the whole\n        # sub-config with a dict and break attribute access downstream.\n        for key, value in model_override_args.items():\n            current = getattr(config, key, None)\n            if isinstance(value, dict) and isinstance(current, PretrainedConfig):\n                current.update(value)\n            else:\n                setattr(config, key, value)\n\n    if is_gguf and not gguf_has_sidecar_config:\n        if config.model_type not in MODEL_FOR_CAUSAL_LM_MAPPING_NAMES:\n            raise RuntimeError(\n                f\"Can't get gguf config for {config.model_type}. Place a \"\n                \"config.json next to the .gguf file to load the config from \"\n                \"there instead.\"\n            )\n        _set_architectures(config, MODEL_FOR_CAUSAL_LM_MAPPING_NAMES[config.model_type])\n\n    return config\n","sourceCodeStart":261,"sourceCodeEnd":287,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/hf_transformers/config.py#L261-L287","documentation":"For a GGUF file without a sidecar config.json, sglang derives the config from transformers' model_type registry; the model's model_type is not in MODEL_FOR_CAUSAL_LM_MAPPING_NAMES so no architecture can be inferred. Place a config.json beside the .gguf to supply it.","triggerScenarios":"Serving a GGUF of a model architecture unknown to the installed transformers version, without a config.json in the same directory/repo.","commonSituations":"Newly released architectures on older transformers/sglang versions, or hand-converted GGUF files that omit config metadata.","solutions":["Put a valid config.json (with correct model_type/architectures) next to the .gguf file or in the HF repo","Upgrade transformers (and sglang) so the architecture is registered","Use an official GGUF upload that ships config.json"],"exampleFix":"# before\nrepo/\n  model-Q4_K_M.gguf\n# after\nrepo/\n  model-Q4_K_M.gguf\n  config.json   # {\"model_type\": \"...\", \"architectures\": [\"...ForCausalLM\"]}","handlingStrategy":"try-catch","validationCode":"from pathlib import Path\nhas_sidecar = Path(model_dir, 'config.json').is_file() if is_gguf else True","typeGuard":null,"tryCatchPattern":"try:\n    get_config(model)\nexcept RuntimeError as e:\n    if 'gguf config' in str(e): ship_or_point_to_sidecar_config()","preventionTips":["Keep a config.json next to every custom-converted GGUF","Keep transformers/sglang updated for new architectures"],"tags":["gguf","config","unsupported-architecture"],"backgroundTag":"unsupported-model-architecture","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}