mudler/LocalAI · error · FileNotFoundError

Model not found: {model_ref}

Error message

Model not found: {model_ref}

What it means

_resolve_model_assets accepts either a local path or a HuggingFace repo id. It first checks Path(model_ref).exists(); if not, and the ref contains '/' (looks like an HF id), it snapshot-downloads it; anything else (nonexistent path, or a ref without '/' that is not on disk) falls through to FileNotFoundError.

Source

Thrown at backend/python/tinygrad/backend.py:103

    if p.exists():
        return p
    if "/" in model_ref and not model_ref.startswith(("/", ".")):
        from huggingface_hub import snapshot_download
        local = snapshot_download(
            repo_id=model_ref,
            allow_patterns=[
                "config.json",
                "tokenizer.json",
                "tokenizer_config.json",
                "special_tokens_map.json",
                "generation_config.json",
                "*.safetensors",
                "*.safetensors.index.json",
                "*.gguf",
            ],
        )
        return Path(local)
    raise FileNotFoundError(f"Model not found: {model_ref}")


def _gguf_path(model_ref: Path) -> Optional[Path]:
    """Return the GGUF file to load from a path that may be a file or dir."""
    if model_ref.is_file() and str(model_ref).endswith(".gguf"):
        return model_ref
    if model_ref.is_dir():
        ggufs = sorted(model_ref.glob("*.gguf"))
        if ggufs:
            return ggufs[0]
    return None


def _load_hf_safetensors(model_dir: Path) -> dict[str, Any]:
    """Load sharded or single-file HF safetensors from a directory."""
    from tinygrad.nn.state import safe_load

    index = model_dir / "model.safetensors.index.json"

View on GitHub (pinned to 44413a9d06)

Solutions

1. If you meant a HuggingFace repo, use the fully-qualified id with namespace, e.g. 'Qwen/Qwen3-0.6B'.
2. If you meant a local file, verify the path exists from the backend's working directory (use an absolute path).
3. For private repos, make sure HF_TOKEN is set so snapshot_download does not fail (a failed download of an existing repo id surfaces as this or a related error).

Example fix

# before
model_ref = "Qwen3-0.6B"          # no namespace, not a local file
# after
model_ref = "Qwen/Qwen3-0.6B"      # HF repo id, snapshot_download path

from pathlib import Path

def resolve_model_ref(ref: str) -> Path:
    p = Path(ref)
    if p.exists():
        return p
    if "/" in ref and not ref.startswith(("/", ".")):
        return None  # will be snapshot-downloaded
    raise FileNotFoundError(f"local path missing and not an HF id: {ref}")

try:
    model_dir = _resolve_model_assets(ref)
except FileNotFoundError:
    if "/" not in ref:
        ref = f"org/{ref}"  # only if you know the org
    model_dir = _resolve_model_assets(ref)

When it happens

Trigger: Loading a tinygrad LLM with a model ref that is a typo'd local path, a path that does not exist yet, a bare HF repo name without namespace (no '/'), or a relative path starting with './' or '/' which is treated as filesystem-only and fails when absent.

Common situations: Model name typo ('Qwen3-0.6B' instead of 'Qwen/Qwen3-0.6B'); pointing at a directory that was never downloaded or was removed; relative path resolved against a different working directory.

Related errors

• Expected HF model directory, got file: {model_path}
• config.json not found under {model_path}
• tokenizer.json not found under {model_dir}
• No safetensors weights found under {model_dir}
• ONNX model not found: {onnx_path}

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/03b9ee9e7220afcf. Report an issue: GitHub.