zylon-ai/private-gpt · error · FileNotFoundError

Local model files not found at '{model_id}'. Ensure the mode

Error message

Local model files not found at '{model_id}'. Ensure the model is downloaded locally.

What it means

When loading with local_files_only=True, AutoProcessor.from_pretrained raises OSError if the model files are not in the local cache/directory. The code converts that into FileNotFoundError telling you the model was never downloaded to '{model_id}', so offline mode cannot be honored.

Source

Thrown at private_gpt/components/llm/tokenizers/huggingface.py:100

                force_download=force_download,
                trust_remote_code=trust_remote_code,
                **kwargs,
            )

            # Extract tokenizer from multimodal processor if needed
            tokenizer: PreTrainedTokenizerBase
            if hasattr(loaded, "tokenizer"):
                processor = cast(ProcessorMixin, loaded)
                tokenizer = cast(PreTrainedTokenizerBase, loaded.tokenizer)
                is_multimodal = True
            else:
                tokenizer = cast(PreTrainedTokenizerBase, loaded)

            return cls(tokenizer, is_multimodal=is_multimodal, processor=processor)

        except OSError as e:
            if local_files_only:
                raise FileNotFoundError(
                    f"Local model files not found at '{model_id}'. "
                    f"Ensure the model is downloaded locally."
                ) from e
            raise ValueError(f"Could not load tokenizer from '{model_id}': {e}") from e
        except Exception as e:
            raise ValueError(f"Failed to load tokenizer: {e}") from e

    @classmethod
    def is_available(cls, model_id: str | Path | None, **kwargs: Any) -> bool:
        return bool(model_id)

    @property
    def all_special_tokens(self) -> list[str]:
        tokens: list[str] = self._tokenizer.all_special_tokens
        return tokens

    @property
    def all_special_ids(self) -> list[int]:

View on GitHub (pinned to 4a030776a3)

Solutions

Pre-download the model once with network access: huggingface-cli download <model_id> (into the cache the app uses), then keep local_files_only=True.
Verify the cache dir: ensure HF_HOME/HF_HUB_CACHE matches where models were stored, and that the path in the error actually contains config/tokenizer files.
If network is available, drop local_files_only=True to allow the download.

Example fix

# before
tok = HuggingFaceTokenizer.from_pretrained('mistralai/Mistral-7B-Instruct-v0.3', local_files_only=True)  # not cached

# after
# shell: huggingface-cli download mistralai/Mistral-7B-Instruct-v0.3
tok = HuggingFaceTokenizer.from_pretrained('mistralai/Mistral-7B-Instruct-v0.3', local_files_only=True)

Defensive patterns

Strategy: validation

Validate before calling

from pathlib import Path

def model_cached_locally(model_id: str, cache_dir: Path | None = None) -> bool:
    if Path(model_id).exists():
        return bool(list(Path(model_id).glob('tokenizer*')) or Path(model_id, 'config.json').exists())
    from huggingface_hub import scan_cache_dir
    return any(e.repo_id == model_id for e in scan_cache_dir(cache_dir).repos)

Try / catch

try:
    tok = HuggingFaceTokenizer.from_pretrained(model_id, local_files_only=True)
except FileNotFoundError as e:
    raise RuntimeError(f'model {model_id} not cached; pre-download before offline run') from e

Prevention

Pre-download models during image build and pin HF_HOME to the same directory used at runtime.
Fail fast at startup in offline mode by checking the cache before serving traffic.
Verify mounted volumes exist and are populated before enabling local_files_only.

When it happens

Trigger: Calling HuggingFaceTokenizer.from_pretrained(model_id, local_files_only=True) when the model id is a Hub name not present in the HF cache, or when a local path is wrong/empty (e.g. a mounted volume not mounted).

Common situations: Air-gapped or offline deployments expecting a pre-downloaded model that was never cached in the image; HF_HOME/HF_HUB_CACHE pointing to a different location than where models were downloaded; typos in model paths or volumes.

Related errors

AI-assisted analysis of zylon-ai/private-gpt@4a030776a3 (2026-08-15). Data as JSON: /api/errors/d059e22f4c059ce5. Report an issue: GitHub.