sgl-project/sglang · error · ValueError

Quant type {quant_type!r} is ambiguous in {repo_id}: {sorted

Error message

Quant type {quant_type!r} is ambiguous in {repo_id}: {sorted(candidates)}. Pass the full owner/repo/path/file.gguf reference instead.

What it means

More than one file in the HF repo ends with -<quant_type>.gguf, so quant-type resolution is ambiguous. The error lists the matching files and asks you to name the exact one.

Source

Thrown at python/sglang/srt/utils/hf_transformers/common.py:336

        from huggingface_hub import HfApi

        files = [
            sibling.rfilename
            for sibling in HfApi().repo_info(repo_id, revision=revision).siblings
        ]
        suffix = f"-{quant_type}.gguf"
        candidates = [filename for filename in files if filename.endswith(suffix)]
        if not candidates:
            available = sorted(
                filename for filename in files if filename.endswith(".gguf")
            )
            raise ValueError(
                f"No file matching quant type {quant_type!r} in {repo_id}. "
                f"Available GGUF files: {available}"
            )
        if len(candidates) > 1:
            raise ValueError(
                f"Quant type {quant_type!r} is ambiguous in {repo_id}: "
                f"{sorted(candidates)}. Pass the full owner/repo/path/file.gguf "
                "reference instead."
            )
        return hf_hub_download(repo_id, candidates[0], revision=revision)

    parts = model.strip("/").split("/")
    if len(parts) < 2:
        return None

    if len(parts) > 2 and model.endswith(".gguf"):
        repo_id = "/".join(parts[:2])
        filename = "/".join(parts[2:])
        return hf_hub_download(repo_id, filename, revision=revision)

    if len(parts) != 2:
        return None

View on GitHub (pinned to 0132848349)

Solutions

1. Pass the full owner/repo/path/file.gguf reference shown in the message
2. Or copy the exact filename from the sorted candidates list into the model string

Example fix

# before
--model org/repo::Q4_K_M
# after
--model org/repo/model-B-Q4_K_M.gguf

matches = [f for f in files if f.endswith(f'-{quant}.gguf')]
if len(matches) > 1: raise SystemExit(f'ambiguous: {matches}; pick a full path')

except ValueError as e:
    if 'ambiguous' in str(e): use_full_gguf_path()

When it happens

Trigger: Serving owner/repo::Q4_K_M when the repo contains e.g. both model-A-Q4_K_M.gguf and model-B-Q4_K_M.gguf.

Common situations: Multi-file GGUF repos that host several architectures or sizes with the same quantization.

Related errors

• No file matching quant type {quant_type!r} in {repo_id}. Ava
• {model} contains {len(candidates)} .gguf files; name the one
• GGUFConfig must be constructed from a GGUF checkpoint
• A GGUF encoder checkpoint cannot be combined with a second q
• Cannot parse checkpoint quantization for {component_name!r}:

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/927f41bab1fd9f36. Report an issue: GitHub.