hiyouga/LlamaFactory · critical · OSError

Failed to load tokenizer.

Error message

Failed to load tokenizer.

What it means

Raised as OSError (wrapping the original exception) in _get_tokenizer when AutoTokenizer.from_pretrained fails with anything other than the specific ValueError that triggers the fast/slow retry. Typical root causes are a missing files.huggingface.co / network failure, an offline cache miss, a repo needing auth, or a path that has no tokenizer files; the original exception is chained via 'from e' for inspection.

Solutions

1. Check the chained exception (raise ... from e) to see the real cause before retrying.
2. For auth issues: huggingface-cli login (or set HF_TOKEN) for gated repos.
3. For offline use: pre-download with huggingface-cli download <model> and/or set HF_HUB_OFFLINE=1 only after the cache is populated.
4. For local paths: verify tokenizer.json / tokenizer_config.json exist in the directory and the path is correct.

Example fix

# before
model_name_or_path: meta-llama/Llama-3-8B  # gated, not logged in

# after
huggingface-cli login
model_name_or_path: meta-llama/Llama-3-8B

import os
from huggingface_hub import snapshot_download

path = cfg["model_args"]["model_name_or_path"]
if not os.path.isdir(path):  # remote repo
    snapshot_download(path, allow_patterns=["tokenizer*", "*.model"])  # fails early with a clear hub error
else:
    assert any(f.startswith("tokenizer") or f == "special_tokens_map.json" for f in os.listdir(path)), \
        "local dir has no tokenizer files"

from transformers import AutoTokenizer
try:
    tok = AutoTokenizer.from_pretrained(model_path, padding_side="right")
except OSError as e:
    cause = e.__cause__
    if cause is not None and ("401" in str(cause) or "gated" in str(cause).lower()):
        print("Auth problem: run huggingface-cli login")
    elif cause is not None and ("offline" in str(cause).lower() or "Connection" in str(cause)):
        print("Network/offline problem: pre-download or unset HF_HUB_OFFLINE")
    raise

When it happens

Trigger: model_name_or_path points to a repo that 401/404s, HF_HUB_OFFLINE=1 without a cached tokenizer, a local dir lacking tokenizer_config.json, a revoked/gated model, or a transient network error during download.

Common situations: Corporate proxies blocking huggingface.co; expired or missing HF token for gated models (Llama etc.); typos in model names; partial local snapshots from interrupted downloads.

Related errors

• Access to private or reserved IP addresses is not allowed.
• Cannot resize embedding layers of a quantized model.
• Cannot resize embedding layers of a quantized model.
• Could not resolve hostname
• Current model does not support resizing embedding layers.

AI-assisted analysis of hiyouga/LlamaFactory@f28afaf635 (2026-08-14). Data as JSON: /api/errors/92720d3f75b0c126. Report an issue: GitHub.

Appendix: source

Thrown at src/llamafactory/model/loader.py:93

    """
    init_kwargs = _get_init_kwargs(model_args)
    try:
        tokenizer = AutoTokenizer.from_pretrained(
            model_args.model_name_or_path,
            use_fast=model_args.use_fast_tokenizer,
            split_special_tokens=model_args.split_special_tokens,
            padding_side="right",
            **init_kwargs,
        )
    except ValueError:  # try another one
        tokenizer = AutoTokenizer.from_pretrained(
            model_args.model_name_or_path,
            use_fast=not model_args.use_fast_tokenizer,
            padding_side="right",
            **init_kwargs,
        )
    except Exception as e:
        raise OSError("Failed to load tokenizer.") from e

    patch_tokenizer(tokenizer, model_args)

    try:
        processor = AutoProcessor.from_pretrained(
            model_args.model_name_or_path,
            use_fast=model_args.use_fast_tokenizer,
            **init_kwargs,
        )
    except ValueError:  # try another one
        processor = AutoProcessor.from_pretrained(
            model_args.model_name_or_path,
            use_fast=not model_args.use_fast_tokenizer,
            **init_kwargs,
        )
    except Exception as e:
        logger.info_rank0(f"Failed to load processor: {e}.")
        processor = None

View on GitHub (pinned to f28afaf635)