{"record":{"id":"7c4b209520152e47","repo":"sgl-project/sglang","slug":"model-contains-len-candidates-gguf-files-na","errorCode":null,"errorMessage":"{model} contains {len(candidates)} .gguf files; name the one to serve:\\n  {listing}","messagePattern":"(.+?) contains (.+?) \\.gguf files; name the one to serve:\\\\n  (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/utils/hf_transformers/common.py","lineNumber":371,"sourceCode":"        return None\n\n    from huggingface_hub import HfApi\n\n    try:\n        files = [\n            s.rfilename for s in HfApi().repo_info(model, revision=revision).siblings\n        ]\n    except Exception:\n        return None\n    if any(f == \"config.json\" for f in files):\n        return None\n\n    candidates = [f for f in files if f.endswith(\".gguf\")]\n    if not candidates:\n        return None\n    if len(candidates) > 1:\n        listing = \"\\n  \".join(f\"{model}/{f}\" for f in sorted(candidates))\n        raise ValueError(\n            f\"{model} contains {len(candidates)} .gguf files; name the one to \"\n            f\"serve:\\n  {listing}\"\n        )\n    return hf_hub_download(model, candidates[0], revision=revision)\n\n\ndef gguf_sidecar_dir(\n    gguf_path: Union[str, os.PathLike], sentinel: str\n) -> Optional[Path]:\n    \"\"\"Directory containing *sentinel* next to a .gguf file, if there is one.\"\"\"\n    directory = Path(gguf_path).parent\n    return directory if (directory / sentinel).is_file() else None\n\n\n# ---------------------------------------------------------------------------\n# Rope / text config helpers\n# ---------------------------------------------------------------------------\n","sourceCodeStart":353,"sourceCodeEnd":389,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/utils/hf_transformers/common.py#L353-L389","documentation":"The repo contains multiple .gguf files and you did not specify which one to serve via a ::quant suffix or explicit file path, so the resolver refuses to guess and prints the full file listing.","triggerScenarios":"Pointing --model at an HF repo with several GGUF quants (e.g. a repo with Q4, Q5, Q8 variants) without a ::quant_type qualifier.","commonSituations":"Users copy the repo name from HF without noticing it bundles many quant files; common with Bartowski/TheBloke-style GGUF repos.","solutions":["Append ::QUANT to the model, e.g. repo::Q4_K_M","Or give the full owner/repo/file.gguf path from the listing in the message"],"exampleFix":"# before\n--model Qwen/Qwen2.5-7B-Instruct-GGUF\n# after\n--model Qwen/Qwen2.5-7B-Instruct-GGUF::q4_k_m","handlingStrategy":"validation","validationCode":"ggufs = [f for f in list_repo_files(repo) if f.endswith('.gguf')]\nif len(ggufs) > 1: require explicit '::quant' or file path from the user","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always append ::QUANT_TYPE or the exact filename when serving GGUF repos"],"tags":["gguf","huggingface","model-selection"],"backgroundTag":"ambiguous-model-reference","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}