mudler/LocalAI · error · FileNotFoundError

Expected HF model directory, got file: {model_path}

Error message

Expected HF model directory, got file: {model_path}

What it means

In the LLM constructor's HF-safetensors branch (taken when the model ref is a directory without a GGUF file), model_path must be a directory containing an HF layout. If the path exists but is a regular file that is not .gguf (e.g. a single .safetensors or .bin file), the branch refuses it with FileNotFoundError.

Source

Thrown at backend/python/tinygrad/backend.py:315

            model, kv = Transformer.from_gguf(gguf_tensor, max_context=max_context_cap)
            self.llm_model = model
            self.max_context = model.max_context
            # Preserve a config-shaped dict for tool-parser heuristics and
            # the "loaded" message.
            arch = kv.get("general.architecture", "")
            self.llm_config = {
                "architectures": [kv.get("general.name", arch) or arch],
                "gguf_kv": kv,
            }

            # Tokenizer: prefer sidecar tokenizer.json (richer HF Jinja2
            # templates), fall back to apps.llm's SimpleTokenizer built
            # from GGUF metadata.
            self._load_tokenizer_for_dir(model_path if model_path.is_dir() else gguf_file.parent, gguf_kv=kv)
        else:
            # HF safetensors path.
            if not model_path.is_dir():
                raise FileNotFoundError(f"Expected HF model directory, got file: {model_path}")
            config_path = model_path / "config.json"
            if not config_path.exists():
                raise FileNotFoundError(f"config.json not found under {model_path}")
            with open(config_path) as fp:
                hf_config = json.load(fp)
            self.llm_config = hf_config

            raw_weights = _load_hf_safetensors(model_path)
            n_layers = hf_config["num_hidden_layers"]
            state_dict = _hf_to_appsllm_state_dict(raw_weights, n_layers)

            kwargs = _hf_to_transformer_kwargs(hf_config, state_dict, max_context_cap)
            self.max_context = kwargs["max_context"]

            model = Transformer(**kwargs)
            load_state_dict(model, state_dict, strict=False, consume=True)
            self.llm_model = model

View on GitHub (pinned to 44413a9d06)

Solutions

1. Pass the directory that contains config.json and the safetensors files, not an individual weight file.
2. If you only have a single GGUF file, keep using it — that path is supported; for HF safetensors the directory layout is required.
3. Re-download/restore the full HF repo layout (config.json + tokenizer + weights) into a directory and point the model at it.

Example fix

# before
model = TinyGradLLM(model_path=Path("/models/qwen/model.safetensors"))
# after
model = TinyGradLLM(model_path=Path("/models/qwen"))  # dir with config.json

from pathlib import Path

def is_hf_model_dir(p) -> bool:
    p = Path(p)
    return p.is_dir() and (p / "config.json").exists()

def is_hf_model_dir(p) -> bool:
    p = Path(p)
    return p.is_dir() and (p / "config.json").exists()

try:
    llm = LLM(model_path)
except FileNotFoundError as e:
    if "Expected HF model directory" in str(e):
        model_path = Path(model_path).parent  # only if parent is the real repo root
    raise

When it happens

Trigger: Passing a single weight file such as /models/qwen/model.safetensors as the model path instead of its parent directory; pointing at a symlinked file; passing a .pt or .bin checkpoint where the GGUF branch is not taken either.

Common situations: User extracts one downloaded file and passes it directly; misconfigured gallery entry with a filename instead of directory; conflating GGUF single-file usage (supported) with safetensors single-file usage (not supported).

Related errors

• Model not found: {model_ref}
• config.json not found under {model_path}
• tokenizer.json not found under {model_dir}
• No safetensors weights found under {model_dir}
• ONNX model not found: {onnx_path}

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/1b82cbf89af9091d. Report an issue: GitHub.