{"record":{"id":"c2d1f8e3cd9dc101","repo":"mudler/LocalAI","slug":"no-safetensors-weights-found-under-model-dir","errorCode":null,"errorMessage":"No safetensors weights found under {model_dir}","messagePattern":"No safetensors weights found under (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"error","filePath":"backend/python/tinygrad/backend.py","lineNumber":134,"sourceCode":"\ndef _load_hf_safetensors(model_dir: Path) -> dict[str, Any]:\n    \"\"\"Load sharded or single-file HF safetensors from a directory.\"\"\"\n    from tinygrad.nn.state import safe_load\n\n    index = model_dir / \"model.safetensors.index.json\"\n    if index.exists():\n        with open(index) as fp:\n            weight_map = json.load(fp)[\"weight_map\"]\n        shards: dict[str, Any] = {}\n        for shard_name in set(weight_map.values()):\n            shards[shard_name] = safe_load(str(model_dir / shard_name))\n        return {k: shards[n][k] for k, n in weight_map.items()}\n\n    single = model_dir / \"model.safetensors\"\n    if single.exists():\n        return safe_load(str(single))\n\n    raise FileNotFoundError(f\"No safetensors weights found under {model_dir}\")\n\n\ndef _auto_tool_parser(model_ref: Optional[str], config: dict) -> Optional[str]:\n    \"\"\"Pick a tool parser automatically from model family heuristics.\n\n    Order of precedence: architecture name from config.json, then model ref\n    string. Returns None to fall through to the passthrough parser.\n    \"\"\"\n    arches = \" \".join(a.lower() for a in config.get(\"architectures\", []))\n    ref = (model_ref or \"\").lower()\n    blob = f\"{arches} {ref}\"\n\n    if \"qwen3\" in blob:\n        return \"qwen3_xml\"\n    if \"hermes\" in blob or \"qwen2\" in blob or \"qwen\" in blob:\n        return \"hermes\"\n    if \"llama-3\" in blob or \"llama_3\" in blob or \"llama3\" in blob:\n        return \"llama3_json\"","sourceCodeStart":116,"sourceCodeEnd":152,"githubUrl":"https://github.com/mudler/LocalAI/blob/44413a9d06bf5bc52ce088ba8ca74e5a2e8bee26/backend/python/tinygrad/backend.py#L116-L152","documentation":"_load_hf_safetensors loads sharded weights via model.safetensors.index.json, or the single model.safetensors file, from a model directory. If neither the index nor the single-file weights exist in the directory, it raises FileNotFoundError. Only safetensors is supported — .bin (pickle) weights, or an index whose shards failed the allow-pattern download, produce this.","triggerScenarios":"Pointing the tinygrad backend at an HF directory that only ships pytorch_model*.bin weights; a snapshot_download with allow_patterns that skipped safetensors; a directory containing only config/tokenizer files because the weights are in a Git-LFS pointer state.","commonSituations":"Older HF repos that predate safetensors; partial/interrupted downloads; manually copying a repo dir and missing the large weight files; GGUF-only repos reaching this code path by mistake.","solutions":["Verify model.safetensors or model.safetensors.index.json + shard files are present in the directory: ls <model_dir>.","If the repo only has .bin weights, convert them to safetensors or pick a repo revision that ships safetensors.","Re-download with a clean cache (rm the HF cache entry) so interrupted snapshot downloads complete; ensure allow_patterns include '*.safetensors*'."],"exampleFix":"# before: repo with only pytorch_model.bin\nmodel_ref = \"some/old-llm\"\n# after: safetensors revision / converted repo\nmodel_ref = \"some/old-llm-safetensors\"","handlingStrategy":"validation","validationCode":"from pathlib import Path\n\ndef has_safetensors(model_dir: str) -> bool:\n    d = Path(model_dir)\n    return (d / \"model.safetensors\").is_file() or (d / \"model.safetensors.index.json\").is_file()","typeGuard":null,"tryCatchPattern":"try:\n    weights = _load_hf_safetensors(model_dir)\nexcept FileNotFoundError:\n    raise ModelLayoutError(\n        f\"{model_dir} has no safetensors; convert .bin weights or use a safetensors repo\"\n    )","preventionTips":["Pre-flight check model dirs for model.safetensors or the sharded index before load.","Prefer repos that ship safetensors natively.","Verify downloads completed (compare file sizes against HF metadata)."],"tags":["tinygrad","safetensors","weights-loading","file-not-found","localai"],"backgroundTag":null,"analyzedSha":"44413a9d06bf5bc52ce088ba8ca74e5a2e8bee26","analyzedAt":"2026-08-15T10:13:50.291Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}