sgl-project/sglang · error · ValueError
contains .gguf files; name the one to serve:\n
Error message
{model} contains {len(candidates)} .gguf files; name the one to serve:\n {listing} What it means
GGUF repository resolution: the given model id/name matched multiple .gguf files in the repository and none was singled out. The message lists the candidates; re-request with the exact file name to serve.
Solutions
- Append ::QUANT to the model, e.g. repo::Q4_K_M
- Or give the full owner/repo/file.gguf path from the listing in the message
Example fix
# before --model Qwen/Qwen2.5-7B-Instruct-GGUF # after --model Qwen/Qwen2.5-7B-Instruct-GGUF::q4_k_m
Defensive patterns
Strategy: validation
Validate before calling
ggufs = [f for f in list_repo_files(repo) if f.endswith('.gguf')]
if len(ggufs) > 1: require explicit '::quant' or file path from the user Prevention
- Always append ::QUANT_TYPE or the exact filename when serving GGUF repos
When it happens
Trigger: Pointing --model at an HF repo with several GGUF quants (e.g. a repo with Q4, Q5, Q8 variants) without a ::quant_type qualifier.
Common situations: Users copy the repo name from HF without noticing it bundles many quant files; common with Bartowski/TheBloke-style GGUF repos.
Related errors
- No file matching quant type
- Quant type is ambiguous in : . Pass the full…
- A GGUF encoder checkpoint cannot be combined with a second…
- Can't get gguf config for
- Cannot parse checkpoint quantization for
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/7c4b209520152e47.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/srt/utils/hf_transformers/common.py:371
return None
from huggingface_hub import HfApi
try:
files = [
s.rfilename for s in HfApi().repo_info(model, revision=revision).siblings
]
except Exception:
return None
if any(f == "config.json" for f in files):
return None
candidates = [f for f in files if f.endswith(".gguf")]
if not candidates:
return None
if len(candidates) > 1:
listing = "\n ".join(f"{model}/{f}" for f in sorted(candidates))
raise ValueError(
f"{model} contains {len(candidates)} .gguf files; name the one to "
f"serve:\n {listing}"
)
return hf_hub_download(model, candidates[0], revision=revision)
def gguf_sidecar_dir(
gguf_path: Union[str, os.PathLike], sentinel: str
) -> Optional[Path]:
"""Directory containing *sentinel* next to a .gguf file, if there is one."""
directory = Path(gguf_path).parent
return directory if (directory / sentinel).is_file() else None
# ---------------------------------------------------------------------------
# Rope / text config helpers
# ---------------------------------------------------------------------------
View on GitHub (pinned to 0132848349)