unslothai/unsloth · error · ValueError

GGUF variant '{variant}' not found in {identifier}. Availabl

Error message

GGUF variant '{variant}' not found in {identifier}. Available variants: {available}

What it means

When the caller explicitly requests a GGUF variant (quant) for a repo, the code lists repo files and asks _gguf_files_for_variant() whether that variant exists; if neither the live listing nor a verified local cache (cached_gguf_for_load with verify_sizes=True) contains it, it raises with the variants actually available (their quant names) so the user can immediately re-pick. The preflight exists to reject before the load path unloads the resident model.

Source

Thrown at studio/backend/utils/models/model_config.py:3859

                    # Reject before the load path unloads the resident model.
                    # Only a live, complete repo listing can prove the variant
                    # absent; without one, let the load path resolve it. The
                    # cache escape mirrors the load path's own reuse predicate.
                    try:
                        from huggingface_hub import list_repo_files
                        repo_files = list_repo_files(identifier, token = hf_token)
                    except Exception:
                        repo_files = None
                    if (
                        repo_files
                        and not _gguf_files_for_variant(repo_files, variant)
                        and not cached_gguf_for_load(
                            identifier, variant, verify_sizes = True, hf_token = hf_token
                        )
                    ):
                        available = ", ".join(v.quant for v in variants)
                        raise ValueError(
                            f"GGUF variant '{variant}' not found in {identifier}. "
                            f"Available variants: {available}"
                        )
                if not variant:  # auto-select best quant
                    # ROOT rows when there are any. _pick_best_gguf keeps whichever filename it
                    # met first among equals, so an LTX-style listing that puts distilled/...-Q6_K
                    # before the root ...-Q6_K made a bare repo id load the distilled checkpoint --
                    # while local_model_resolver, the auto-download map and /gguf-variants all
                    # define a bare id as the root. This is the LOAD path, so it has to agree.
                    root_rows = [
                        v.filename
                        for v in variants
                        if "/" not in _qualified_variant_name(v.filename, v.quant)
                    ]
                    variant_filenames = root_rows or [v.filename for v in variants]
                    best = _pick_best_gguf(variant_filenames)
                    if best:
                        # The SAME identity the lister advertised for that file. Converting the

View on GitHub (pinned to 203007d190)

Solutions

1. Read the message's 'Available variants' list and re-request one of those (or drop gguf_variant to auto-select the best quant)
2. If you expected the variant, check the repo page for renamed/split files (e.g. '-00001-of-00002.gguf') and pass the exact quant token the listing uses
3. Delete the partial cached GGUF and retry, so a size-verified redownload can satisfy the cache fallback
4. Call the /gguf-variants endpoint for the identifier to see the canonical variant list before loading

Example fix

# before
load_model('org/model-GGUF', gguf_variant='Q4_K_M')
# ValueError: GGUF variant 'Q4_K_M' not found ... Available variants: Q6_K, Q8_0

# after
load_model('org/model-GGUF', gguf_variant='Q6_K')
# or auto-select
load_model('org/model-GGUF')

from utils.models.model_config import list_gguf_variants

def variant_valid(identifier: str, variant: str, hf_token=None) -> bool:
    variants, _ = list_gguf_variants(identifier, hf_token=hf_token)
    return variant in {v.quant for v in variants}

try:
    load_model(repo_id, gguf_variant=v)
except ValueError as e:
    if 'Available variants:' in str(e):
        offered = parse_variants(str(e))
        v = pick_from(offered)  # re-request with a valid quant or None for auto
    else:
        raise

When it happens

Trigger: Passing gguf_variant='Q4_K_M' when the repo only ships Q8_0/Q6_K (or uses a naming scheme _gguf_files_for_variant does not map to that quant token); the requested variant's local copy is size-mismatched so verify_sizes rejects the cache fallback. A network failure in list_repo_files is tolerated (repo_files=None) and does NOT raise here.

Common situations: Hardcoding a quant from a different model card; repos with distilled/split GGUFs whose names parse to unexpected quants; requesting a variant deleted upstream while a stale UI list still shows it; interrupted downloads leaving incomplete cached files.

Related errors

• This is a GGUF model, but the llama.cpp runtime (llama-serve
• '{repo_id}' is a single-file GGUF repo; load it with model_k
• LoRA is not supported for this model/quantisation on the dif
• Metadata unavailable while resolving GGUF variant '{variant}
• gguf_variant is required when export_type is 'gguf'

AI-assisted analysis of unslothai/unsloth@203007d190 (2026-08-15). Data as JSON: /api/errors/3b4670f9df4f2cf2. Report an issue: GitHub.