sgl-project/sglang · error · ValueError

No file matching quant type {quant_type!r} in {repo_id}. Ava

Error message

No file matching quant type {quant_type!r} in {repo_id}. Available GGUF files: {available}

What it means

When resolving an HF GGUF reference like owner/repo::Q4_K_M, no file in the repo ends with -<quant_type>.gguf. The error lists all .gguf files actually available so you can pick a valid quant type or full file reference.

Source

Thrown at python/sglang/srt/utils/hf_transformers/common.py:331

    if ":" in model:
        repo_id, _, quant_type = model.rpartition(":")
        if repo_id.count("/") != 1 or not quant_type:
            return None

        from huggingface_hub import HfApi

        files = [
            sibling.rfilename
            for sibling in HfApi().repo_info(repo_id, revision=revision).siblings
        ]
        suffix = f"-{quant_type}.gguf"
        candidates = [filename for filename in files if filename.endswith(suffix)]
        if not candidates:
            available = sorted(
                filename for filename in files if filename.endswith(".gguf")
            )
            raise ValueError(
                f"No file matching quant type {quant_type!r} in {repo_id}. "
                f"Available GGUF files: {available}"
            )
        if len(candidates) > 1:
            raise ValueError(
                f"Quant type {quant_type!r} is ambiguous in {repo_id}: "
                f"{sorted(candidates)}. Pass the full owner/repo/path/file.gguf "
                "reference instead."
            )
        return hf_hub_download(repo_id, candidates[0], revision=revision)

    parts = model.strip("/").split("/")
    if len(parts) < 2:
        return None

    if len(parts) > 2 and model.endswith(".gguf"):
        repo_id = "/".join(parts[:2])
        filename = "/".join(parts[2:])

View on GitHub (pinned to 0132848349)

Solutions

1. Read the 'Available GGUF files' list in the message and use one of those quant types
2. Or pass the full reference owner/repo/path/file.gguf to skip quant resolution
3. Check the HF repo page to confirm the exact filename and its quant suffix

Example fix

# before
python -m sglang.launch_server --model Qwen/Qwen2.5-7B-Instruct-GGUF::Q4_K_M
# after (pick an available file)
python -m sglang.launch_server --model Qwen/Qwen2.5-7B-Instruct-GGUF::q4_0

from huggingface_hub import list_repo_files
files = [f for f in list_repo_files(repo_id, revision=revision) if f.endswith('.gguf')]
quants = {f.rsplit('-', 1)[1][:-5] for f in files if '-' in f}
assert quant_type in quants, f'{quant_type} not in {sorted(quants)}'

try:
    path = resolve_hf_gguf_reference(model)
except ValueError as e:
    # parse 'Available GGUF files' from str(e) and prompt user to choose
    raise

When it happens

Trigger: Calling resolve_hf_gguf_reference (or serving model::quant) where the repo has GGUF files but none whose name ends with the requested quant suffix, e.g. repo contains model-Q4_0.gguf but you asked for Q4_K_M.

Common situations: Typos in quant type names (q4km vs Q4_K_M), repos with non-standard filenames, or requesting a quant the uploader never created.

Related errors

• No model weights found in {path} (expected model.safetensors
• Quant type {quant_type!r} is ambiguous in {repo_id}: {sorted
• {model} contains {len(candidates)} .gguf files; name the one
• Unsupported model type: {model_type}
• Z-Image transformer has no `rotary_emb`. It likely loaded vi

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/f4535e700bbb26be. Report an issue: GitHub.