{"record":{"id":"63ea11896cd277e1","repo":"unslothai/unsloth","slug":"model-r-has-cached-pickle-weights-that-cannot-be","errorCode":null,"errorMessage":"{model!r} has cached pickle weights that cannot be security-scanned offline and no safetensors alternative, so it cannot be used as the embedding model. Re-download it with safetensors weights while online.","messagePattern":"(.+?) has cached pickle weights that cannot be security-scanned offline and no safetensors alternative, so it cannot be used as the embedding model\\. Re-download it with safetensors weights while online\\.","errorType":"http","errorClass":"HTTPException","httpStatus":403,"severity":"error","filePath":"studio/backend/routes/settings.py","lineNumber":1625,"sourceCode":"            model,\n            hf_token = scan_token,\n            load_subdirs = load_subdirs,\n            local_only_load = local_only_load,\n        ).blocked:\n            # 403, not 409: the client routes every 409 into the forceable \"save anyway\"\n            # flow, but this block is a hard, non-forceable security refusal.\n            if local_only_load:\n                detail = (\n                    f\"{model!r} has cached pickle weights that cannot be security-scanned \"\n                    \"offline and no safetensors alternative, so it cannot be used as the \"\n                    \"embedding model. Re-download it with safetensors weights while online.\"\n                )\n            else:\n                detail = (\n                    f\"{model!r} is flagged as unsafe by Hugging Face's security scan and \"\n                    \"cannot be used as the embedding model.\"\n                )\n            raise HTTPException(status_code = 403, detail = detail)\n    if model != default_embedding_model() and not payload.force and not is_local_gguf:\n        from core.rag import config as rag_config\n\n        # A GGUF-named repo on the llama-server backend is loaded from its .gguf\n        # files, which rarely carry sentence-transformers metadata; verify the\n        # GGUF is available (below) rather than the ST embedding-metadata gate,\n        # which would wrongly 409 a valid online GGUF embedder.\n        gguf_named = _llama_backend_active() and rag_config._names_gguf(model)\n        if not gguf_named and not is_embedding_model(model, hf_token = hf_token):\n            # Offline, is_embedding_model can only confirm the ST layout (modules.json); a\n            # transformers-native embedder (e.g. gte-modernbert) is unverifiable without Hub\n            # metadata. If already cached and loadable, accept it rather than raising a 409 that\n            # online would not (ST can load any cached encoder). Uncached -> 409.\n            from utils.utils import hf_cache_snapshot_is_loadable\n\n            # Require a genuinely loadable cache (config + weights), not just a resolved refs/main,\n            # so a metadata-only partial cache still gets the forceable 409.\n            offline_cached = local_only_load and hf_cache_snapshot_is_loadable(model)","sourceCodeStart":1607,"sourceCodeEnd":1643,"githubUrl":"https://github.com/unslothai/unsloth/blob/203007d19051dcd2ae33876786d117c99f6b0368/studio/backend/routes/settings.py#L1607-L1643","documentation":"HTTP 403 (hard, non-forceable security refusal) from the embedding-model endpoint when local_only_load is true and the requested model has cached pickle-format weights (.bin/.pt) with no safetensors alternative. Pickle weights cannot be security-scanned offline, so the endpoint refuses to use them as the RAG embedding model; unlike the generic verification 409, no force flag bypasses this — you must obtain safetensors weights.","triggerScenarios":"PUT the embedding-model setting to a repo whose local cache only contains pickle weight files, while the server runs in local/offline-only load mode (no Hub access).","commonSituations":"Pre-2023 sentence-transformers snapshots downloaded before safetensors became standard; air-gapped machines; a cache populated by an older tool that skipped safetensors; partial downloads that kept the .bin files.","solutions":["Bring the server online once and re-download the model so safetensors weights are fetched alongside (or instead of) the pickle files.","Or delete the pickle-only cache entry and download the repo with safetensors on a connected machine, then copy the cache over.","Do not attempt to bypass with force:true — this branch intentionally ignores it.","Prefer repos that publish safetensors (most current sentence-transformers models do)."],"exampleFix":"# before (offline, pickle-only cache)\nPUT /settings/embedding-model {\"model\": \"old-model-with-.bin-only\"}  # 403\n\n# after (one-time online refresh)\nhuggingface-cli download old-model --include \"*.safetensors\" \"*.json\"\n# restart offline mode; the same PUT now passes the scan gate","handlingStrategy":"validation","validationCode":"const files = await listLocalCacheFiles(model);\nconst hasSafetensors = files.some(f => f.endsWith('.safetensors'));\nif (!hasSafetensors) throw new Error('Model lacks safetensors; refresh online before offline use');\nawait api.put('/settings/embedding-model', { model });","typeGuard":"function isOfflineSafeEmbedder(files: string[]): boolean {\n  return files.some(f => f.endsWith('.safetensors'));\n}","tryCatchPattern":"try { await api.put('/settings/embedding-model', { model }); }\ncatch (e) {\n  if (e.status === 403 && /pickle/i.test(e.detail)) { queueOnlineRedownload(model); return; } // not forceable\n  throw e;\n}","preventionTips":["Standardize on safetensors releases when seeding caches for offline machines.","Periodically refresh caches while online so pre-safetensors snapshots get updated.","Accept that this 403 ignores force:true by design; plan provisioning around it."],"tags":["fastapi","http-403","security","hugging-face","pickle","offline","embeddings"],"backgroundTag":null,"analyzedSha":"203007d19051dcd2ae33876786d117c99f6b0368","analyzedAt":"2026-08-15T02:48:39.846Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}