{"record":{"id":"1b82cbf89af9091d","repo":"mudler/LocalAI","slug":"expected-hf-model-directory-got-file-model-path","errorCode":null,"errorMessage":"Expected HF model directory, got file: {model_path}","messagePattern":"Expected HF model directory, got file: (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"error","filePath":"backend/python/tinygrad/backend.py","lineNumber":315,"sourceCode":"            model, kv = Transformer.from_gguf(gguf_tensor, max_context=max_context_cap)\n            self.llm_model = model\n            self.max_context = model.max_context\n            # Preserve a config-shaped dict for tool-parser heuristics and\n            # the \"loaded\" message.\n            arch = kv.get(\"general.architecture\", \"\")\n            self.llm_config = {\n                \"architectures\": [kv.get(\"general.name\", arch) or arch],\n                \"gguf_kv\": kv,\n            }\n\n            # Tokenizer: prefer sidecar tokenizer.json (richer HF Jinja2\n            # templates), fall back to apps.llm's SimpleTokenizer built\n            # from GGUF metadata.\n            self._load_tokenizer_for_dir(model_path if model_path.is_dir() else gguf_file.parent, gguf_kv=kv)\n        else:\n            # HF safetensors path.\n            if not model_path.is_dir():\n                raise FileNotFoundError(f\"Expected HF model directory, got file: {model_path}\")\n            config_path = model_path / \"config.json\"\n            if not config_path.exists():\n                raise FileNotFoundError(f\"config.json not found under {model_path}\")\n            with open(config_path) as fp:\n                hf_config = json.load(fp)\n            self.llm_config = hf_config\n\n            raw_weights = _load_hf_safetensors(model_path)\n            n_layers = hf_config[\"num_hidden_layers\"]\n            state_dict = _hf_to_appsllm_state_dict(raw_weights, n_layers)\n\n            kwargs = _hf_to_transformer_kwargs(hf_config, state_dict, max_context_cap)\n            self.max_context = kwargs[\"max_context\"]\n\n            model = Transformer(**kwargs)\n            load_state_dict(model, state_dict, strict=False, consume=True)\n            self.llm_model = model\n","sourceCodeStart":297,"sourceCodeEnd":333,"githubUrl":"https://github.com/mudler/LocalAI/blob/44413a9d06bf5bc52ce088ba8ca74e5a2e8bee26/backend/python/tinygrad/backend.py#L297-L333","documentation":"In the LLM constructor's HF-safetensors branch (taken when the model ref is a directory without a GGUF file), model_path must be a directory containing an HF layout. If the path exists but is a regular file that is not .gguf (e.g. a single .safetensors or .bin file), the branch refuses it with FileNotFoundError.","triggerScenarios":"Passing a single weight file such as /models/qwen/model.safetensors as the model path instead of its parent directory; pointing at a symlinked file; passing a .pt or .bin checkpoint where the GGUF branch is not taken either.","commonSituations":"User extracts one downloaded file and passes it directly; misconfigured gallery entry with a filename instead of directory; conflating GGUF single-file usage (supported) with safetensors single-file usage (not supported).","solutions":["Pass the directory that contains config.json and the safetensors files, not an individual weight file.","If you only have a single GGUF file, keep using it — that path is supported; for HF safetensors the directory layout is required.","Re-download/restore the full HF repo layout (config.json + tokenizer + weights) into a directory and point the model at it."],"exampleFix":"# before\nmodel = TinyGradLLM(model_path=Path(\"/models/qwen/model.safetensors\"))\n# after\nmodel = TinyGradLLM(model_path=Path(\"/models/qwen\"))  # dir with config.json","handlingStrategy":"type-guard","validationCode":"from pathlib import Path\n\ndef is_hf_model_dir(p) -> bool:\n    p = Path(p)\n    return p.is_dir() and (p / \"config.json\").exists()","typeGuard":"def is_hf_model_dir(p) -> bool:\n    p = Path(p)\n    return p.is_dir() and (p / \"config.json\").exists()","tryCatchPattern":"try:\n    llm = LLM(model_path)\nexcept FileNotFoundError as e:\n    if \"Expected HF model directory\" in str(e):\n        model_path = Path(model_path).parent  # only if parent is the real repo root\n    raise","preventionTips":["Always pass the directory containing config.json for HF models.","Remember: single-file works only for .gguf, not for .safetensors.","Add an assert in test fixtures that model paths are directories."],"tags":["tinygrad","huggingface","model-layout","file-not-found","localai"],"backgroundTag":null,"analyzedSha":"44413a9d06bf5bc52ce088ba8ca74e5a2e8bee26","analyzedAt":"2026-08-15T10:13:50.291Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}