{"record":{"id":"9beb41459f979432","repo":"mudler/LocalAI","slug":"config-json-not-found-under-model-path","errorCode":null,"errorMessage":"config.json not found under {model_path}","messagePattern":"config\\.json not found under (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"error","filePath":"backend/python/tinygrad/backend.py","lineNumber":318,"sourceCode":"            # Preserve a config-shaped dict for tool-parser heuristics and\n            # the \"loaded\" message.\n            arch = kv.get(\"general.architecture\", \"\")\n            self.llm_config = {\n                \"architectures\": [kv.get(\"general.name\", arch) or arch],\n                \"gguf_kv\": kv,\n            }\n\n            # Tokenizer: prefer sidecar tokenizer.json (richer HF Jinja2\n            # templates), fall back to apps.llm's SimpleTokenizer built\n            # from GGUF metadata.\n            self._load_tokenizer_for_dir(model_path if model_path.is_dir() else gguf_file.parent, gguf_kv=kv)\n        else:\n            # HF safetensors path.\n            if not model_path.is_dir():\n                raise FileNotFoundError(f\"Expected HF model directory, got file: {model_path}\")\n            config_path = model_path / \"config.json\"\n            if not config_path.exists():\n                raise FileNotFoundError(f\"config.json not found under {model_path}\")\n            with open(config_path) as fp:\n                hf_config = json.load(fp)\n            self.llm_config = hf_config\n\n            raw_weights = _load_hf_safetensors(model_path)\n            n_layers = hf_config[\"num_hidden_layers\"]\n            state_dict = _hf_to_appsllm_state_dict(raw_weights, n_layers)\n\n            kwargs = _hf_to_transformer_kwargs(hf_config, state_dict, max_context_cap)\n            self.max_context = kwargs[\"max_context\"]\n\n            model = Transformer(**kwargs)\n            load_state_dict(model, state_dict, strict=False, consume=True)\n            self.llm_model = model\n\n            self._load_tokenizer_for_dir(model_path, gguf_kv=None)\n\n        # Auto-pick tool parser from options or model family.","sourceCodeStart":300,"sourceCodeEnd":336,"githubUrl":"https://github.com/mudler/LocalAI/blob/44413a9d06bf5bc52ce088ba8ca74e5a2e8bee26/backend/python/tinygrad/backend.py#L300-L336","documentation":"The HF branch of the tinygrad LLM loader requires config.json in the model directory because it supplies architectures, num_hidden_layers, and the transformer kwargs. Its absence means the directory is not a complete HF snapshot (weights-only dir, or a GGUF dir misrouted into this branch).","triggerScenarios":"Model directory contains safetensors but config.json was filtered out of the snapshot (allow_patterns omissions), manually pruned, or the path points at a weights cache subfolder rather than the repo root.","commonSituations":"snapshot_download allow_patterns list that forgot config.json; users copying only weight files; nested HF cache layout where the wrong level of the directory tree is passed.","solutions":["ls the directory and confirm config.json is present; if missing, copy it from the HF repo page or re-download with config.json included.","Point at the repo snapshot root (the directory that HF shows containing config.json), not an inner folder.","Ensure allow_patterns in any custom download code includes 'config.json'."],"exampleFix":"# before: allow_patterns missing config.json\nallow_patterns=[\"*.safetensors\", \"tokenizer.json\"]\n# after\nallow_patterns=[\"config.json\", \"tokenizer.json\", \"*.safetensors\"]","handlingStrategy":"validation","validationCode":"from pathlib import Path\n\ndef hf_dir_complete(d: str) -> bool:\n    d = Path(d)\n    return (d / \"config.json\").is_file() and (d / \"model.safetensors\").is_file() or (d / \"model.safetensors.index.json\").is_file()","typeGuard":null,"tryCatchPattern":"try:\n    ...load...\nexcept FileNotFoundError as e:\n    if \"config.json\" in str(e):\n        # fetch just the config from the hub\n        from huggingface_hub import hf_hub_download\n        hf_hub_download(repo_id, \"config.json\", local_dir=d)","preventionTips":["Include config.json in every snapshot allow_patterns.","Add an integration test that loads each configured model after download.","Use huggingface_hub.snapshot_download unfiltered for small models to avoid pattern mistakes."],"tags":["tinygrad","huggingface","config","file-not-found","localai"],"backgroundTag":null,"analyzedSha":"44413a9d06bf5bc52ce088ba8ca74e5a2e8bee26","analyzedAt":"2026-08-15T10:13:50.291Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}