hiyouga/LlamaFactory · critical · OSError

Failed to load tokenizer.

Error message

Failed to load tokenizer.

What it means

Raised as OSError (wrapping the original exception) in _get_tokenizer when AutoTokenizer.from_pretrained fails with anything other than the specific ValueError that triggers the fast/slow retry. Typical root causes are a missing files.huggingface.co / network failure, an offline cache miss, a repo needing auth, or a path that has no tokenizer files; the original exception is chained via 'from e' for inspection.

Source

Thrown at src/llamafactory/model/loader.py:93

    """
    init_kwargs = _get_init_kwargs(model_args)
    try:
        tokenizer = AutoTokenizer.from_pretrained(
            model_args.model_name_or_path,
            use_fast=model_args.use_fast_tokenizer,
            split_special_tokens=model_args.split_special_tokens,
            padding_side="right",
            **init_kwargs,
        )
    except ValueError:  # try another one
        tokenizer = AutoTokenizer.from_pretrained(
            model_args.model_name_or_path,
            use_fast=not model_args.use_fast_tokenizer,
            padding_side="right",
            **init_kwargs,
        )
    except Exception as e:
        raise OSError("Failed to load tokenizer.") from e

    patch_tokenizer(tokenizer, model_args)

    try:
        processor = AutoProcessor.from_pretrained(
            model_args.model_name_or_path,
            use_fast=model_args.use_fast_tokenizer,
            **init_kwargs,
        )
    except ValueError:  # try another one
        processor = AutoProcessor.from_pretrained(
            model_args.model_name_or_path,
            use_fast=not model_args.use_fast_tokenizer,
            **init_kwargs,
        )
    except Exception as e:
        logger.info_rank0(f"Failed to load processor: {e}.")
        processor = None

View on GitHub (pinned to f28afaf635)

Solutions

1. Check the chained exception (raise ... from e) to see the real cause before retrying.
2. For auth issues: huggingface-cli login (or set HF_TOKEN) for gated repos.
3. For offline use: pre-download with huggingface-cli download <model> and/or set HF_HUB_OFFLINE=1 only after the cache is populated.
4. For local paths: verify tokenizer.json / tokenizer_config.json exist in the directory and the path is correct.

Example fix

# before
model_name_or_path: meta-llama/Llama-3-8B  # gated, not logged in

# after
huggingface-cli login
model_name_or_path: meta-llama/Llama-3-8B

import os
from huggingface_hub import snapshot_download

path = cfg["model_args"]["model_name_or_path"]
if not os.path.isdir(path):  # remote repo
    snapshot_download(path, allow_patterns=["tokenizer*", "*.model"])  # fails early with a clear hub error
else:
    assert any(f.startswith("tokenizer") or f == "special_tokens_map.json" for f in os.listdir(path)), \
        "local dir has no tokenizer files"

from transformers import AutoTokenizer
try:
    tok = AutoTokenizer.from_pretrained(model_path, padding_side="right")
except OSError as e:
    cause = e.__cause__
    if cause is not None and ("401" in str(cause) or "gated" in str(cause).lower()):
        print("Auth problem: run huggingface-cli login")
    elif cause is not None and ("offline" in str(cause).lower() or "Connection" in str(cause)):
        print("Network/offline problem: pre-download or unset HF_HUB_OFFLINE")
    raise

When it happens

Trigger: model_name_or_path points to a repo that 401/404s, HF_HUB_OFFLINE=1 without a cached tokenizer, a local dir lacking tokenizer_config.json, a revoked/gated model, or a transient network error during download.

Common situations: Corporate proxies blocking huggingface.co; expired or missing HF token for gated models (Llama etc.); typos in model names; partial local snapshots from interrupted downloads.

Related errors

• Access to private or reserved IP addresses is not allowed.
• Could not resolve hostname: {parsed_url.hostname}
• Stop words are required to replace the EOS token.
• YAML config must be a dictionary mapping tokens to descripti
• Cannot resize embedding layers of a quantized model.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/92720d3f75b0c126. Report an issue: GitHub.