mudler/LocalAI · error · FileNotFoundError

config.json not found under {model_path}

Error message

config.json not found under {model_path}

What it means

The HF branch of the tinygrad LLM loader requires config.json in the model directory because it supplies architectures, num_hidden_layers, and the transformer kwargs. Its absence means the directory is not a complete HF snapshot (weights-only dir, or a GGUF dir misrouted into this branch).

Source

Thrown at backend/python/tinygrad/backend.py:318

            # Preserve a config-shaped dict for tool-parser heuristics and
            # the "loaded" message.
            arch = kv.get("general.architecture", "")
            self.llm_config = {
                "architectures": [kv.get("general.name", arch) or arch],
                "gguf_kv": kv,
            }

            # Tokenizer: prefer sidecar tokenizer.json (richer HF Jinja2
            # templates), fall back to apps.llm's SimpleTokenizer built
            # from GGUF metadata.
            self._load_tokenizer_for_dir(model_path if model_path.is_dir() else gguf_file.parent, gguf_kv=kv)
        else:
            # HF safetensors path.
            if not model_path.is_dir():
                raise FileNotFoundError(f"Expected HF model directory, got file: {model_path}")
            config_path = model_path / "config.json"
            if not config_path.exists():
                raise FileNotFoundError(f"config.json not found under {model_path}")
            with open(config_path) as fp:
                hf_config = json.load(fp)
            self.llm_config = hf_config

            raw_weights = _load_hf_safetensors(model_path)
            n_layers = hf_config["num_hidden_layers"]
            state_dict = _hf_to_appsllm_state_dict(raw_weights, n_layers)

            kwargs = _hf_to_transformer_kwargs(hf_config, state_dict, max_context_cap)
            self.max_context = kwargs["max_context"]

            model = Transformer(**kwargs)
            load_state_dict(model, state_dict, strict=False, consume=True)
            self.llm_model = model

            self._load_tokenizer_for_dir(model_path, gguf_kv=None)

        # Auto-pick tool parser from options or model family.

View on GitHub (pinned to 44413a9d06)

Solutions

1. ls the directory and confirm config.json is present; if missing, copy it from the HF repo page or re-download with config.json included.
2. Point at the repo snapshot root (the directory that HF shows containing config.json), not an inner folder.
3. Ensure allow_patterns in any custom download code includes 'config.json'.

Example fix

# before: allow_patterns missing config.json
allow_patterns=["*.safetensors", "tokenizer.json"]
# after
allow_patterns=["config.json", "tokenizer.json", "*.safetensors"]

from pathlib import Path

def hf_dir_complete(d: str) -> bool:
    d = Path(d)
    return (d / "config.json").is_file() and (d / "model.safetensors").is_file() or (d / "model.safetensors.index.json").is_file()

try:
    ...load...
except FileNotFoundError as e:
    if "config.json" in str(e):
        # fetch just the config from the hub
        from huggingface_hub import hf_hub_download
        hf_hub_download(repo_id, "config.json", local_dir=d)

When it happens

Trigger: Model directory contains safetensors but config.json was filtered out of the snapshot (allow_patterns omissions), manually pruned, or the path points at a weights cache subfolder rather than the repo root.

Common situations: snapshot_download allow_patterns list that forgot config.json; users copying only weight files; nested HF cache layout where the wrong level of the directory tree is passed.

Related errors

• Model not found: {model_ref}
• Expected HF model directory, got file: {model_path}
• tokenizer.json not found under {model_dir}
• No safetensors weights found under {model_dir}
• ONNX model not found: {onnx_path}

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/9beb41459f979432. Report an issue: GitHub.