sgl-project/sglang · error · ValueError

{model} contains {len(candidates)} .gguf files; name the one

Error message

{model} contains {len(candidates)} .gguf files; name the one to serve:\n  {listing}

What it means

The repo contains multiple .gguf files and you did not specify which one to serve via a ::quant suffix or explicit file path, so the resolver refuses to guess and prints the full file listing.

Source

Thrown at python/sglang/srt/utils/hf_transformers/common.py:371

        return None

    from huggingface_hub import HfApi

    try:
        files = [
            s.rfilename for s in HfApi().repo_info(model, revision=revision).siblings
        ]
    except Exception:
        return None
    if any(f == "config.json" for f in files):
        return None

    candidates = [f for f in files if f.endswith(".gguf")]
    if not candidates:
        return None
    if len(candidates) > 1:
        listing = "\n  ".join(f"{model}/{f}" for f in sorted(candidates))
        raise ValueError(
            f"{model} contains {len(candidates)} .gguf files; name the one to "
            f"serve:\n  {listing}"
        )
    return hf_hub_download(model, candidates[0], revision=revision)


def gguf_sidecar_dir(
    gguf_path: Union[str, os.PathLike], sentinel: str
) -> Optional[Path]:
    """Directory containing *sentinel* next to a .gguf file, if there is one."""
    directory = Path(gguf_path).parent
    return directory if (directory / sentinel).is_file() else None


# ---------------------------------------------------------------------------
# Rope / text config helpers
# ---------------------------------------------------------------------------

View on GitHub (pinned to 0132848349)

Solutions

1. Append ::QUANT to the model, e.g. repo::Q4_K_M
2. Or give the full owner/repo/file.gguf path from the listing in the message

Example fix

# before
--model Qwen/Qwen2.5-7B-Instruct-GGUF
# after
--model Qwen/Qwen2.5-7B-Instruct-GGUF::q4_k_m

ggufs = [f for f in list_repo_files(repo) if f.endswith('.gguf')]
if len(ggufs) > 1: require explicit '::quant' or file path from the user

When it happens

Trigger: Pointing --model at an HF repo with several GGUF quants (e.g. a repo with Q4, Q5, Q8 variants) without a ::quant_type qualifier.

Common situations: Users copy the repo name from HF without noticing it bundles many quant files; common with Bartowski/TheBloke-style GGUF repos.

Related errors

• No file matching quant type {quant_type!r} in {repo_id}. Ava
• Quant type {quant_type!r} is ambiguous in {repo_id}: {sorted
• GGUFConfig must be constructed from a GGUF checkpoint
• A GGUF encoder checkpoint cannot be combined with a second q
• Cannot parse checkpoint quantization for {component_name!r}:

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7c4b209520152e47. Report an issue: GitHub.