mudler/LocalAI · error · FileNotFoundError

No safetensors weights found under {model_dir}

Error message

No safetensors weights found under {model_dir}

What it means

_load_hf_safetensors loads sharded weights via model.safetensors.index.json, or the single model.safetensors file, from a model directory. If neither the index nor the single-file weights exist in the directory, it raises FileNotFoundError. Only safetensors is supported — .bin (pickle) weights, or an index whose shards failed the allow-pattern download, produce this.

Source

Thrown at backend/python/tinygrad/backend.py:134

def _load_hf_safetensors(model_dir: Path) -> dict[str, Any]:
    """Load sharded or single-file HF safetensors from a directory."""
    from tinygrad.nn.state import safe_load

    index = model_dir / "model.safetensors.index.json"
    if index.exists():
        with open(index) as fp:
            weight_map = json.load(fp)["weight_map"]
        shards: dict[str, Any] = {}
        for shard_name in set(weight_map.values()):
            shards[shard_name] = safe_load(str(model_dir / shard_name))
        return {k: shards[n][k] for k, n in weight_map.items()}

    single = model_dir / "model.safetensors"
    if single.exists():
        return safe_load(str(single))

    raise FileNotFoundError(f"No safetensors weights found under {model_dir}")


def _auto_tool_parser(model_ref: Optional[str], config: dict) -> Optional[str]:
    """Pick a tool parser automatically from model family heuristics.

    Order of precedence: architecture name from config.json, then model ref
    string. Returns None to fall through to the passthrough parser.
    """
    arches = " ".join(a.lower() for a in config.get("architectures", []))
    ref = (model_ref or "").lower()
    blob = f"{arches} {ref}"

    if "qwen3" in blob:
        return "qwen3_xml"
    if "hermes" in blob or "qwen2" in blob or "qwen" in blob:
        return "hermes"
    if "llama-3" in blob or "llama_3" in blob or "llama3" in blob:
        return "llama3_json"

View on GitHub (pinned to 44413a9d06)

Solutions

1. Verify model.safetensors or model.safetensors.index.json + shard files are present in the directory: ls <model_dir>.
2. If the repo only has .bin weights, convert them to safetensors or pick a repo revision that ships safetensors.
3. Re-download with a clean cache (rm the HF cache entry) so interrupted snapshot downloads complete; ensure allow_patterns include '*.safetensors*'.

Example fix

# before: repo with only pytorch_model.bin
model_ref = "some/old-llm"
# after: safetensors revision / converted repo
model_ref = "some/old-llm-safetensors"

from pathlib import Path

def has_safetensors(model_dir: str) -> bool:
    d = Path(model_dir)
    return (d / "model.safetensors").is_file() or (d / "model.safetensors.index.json").is_file()

try:
    weights = _load_hf_safetensors(model_dir)
except FileNotFoundError:
    raise ModelLayoutError(
        f"{model_dir} has no safetensors; convert .bin weights or use a safetensors repo"
    )

When it happens

Trigger: Pointing the tinygrad backend at an HF directory that only ships pytorch_model*.bin weights; a snapshot_download with allow_patterns that skipped safetensors; a directory containing only config/tokenizer files because the weights are in a Git-LFS pointer state.

Common situations: Older HF repos that predate safetensors; partial/interrupted downloads; manually copying a repo dir and missing the large weight files; GGUF-only repos reaching this code path by mistake.

Related errors

• Model not found: {model_ref}
• Expected HF model directory, got file: {model_path}
• config.json not found under {model_path}
• tokenizer.json not found under {model_dir}
• ONNX model not found: {onnx_path}

AI-assisted analysis of mudler/LocalAI@44413a9d06 (2026-08-15). Data as JSON: /api/errors/c2d1f8e3cd9dc101. Report an issue: GitHub.