sgl-project/sglang · error · RuntimeError

Failed to load the tokenizer. If you are using a LLaMA V1 mo

Error message

Failed to load the tokenizer. If you are using a LLaMA V1 model consider using '{_FAST_LLAMA_TOKENIZER}' instead of the original tokenizer.

What it means

AutoTokenizer.from_pretrained raised TypeError while loading the tokenizer; the classic case is LLaMA V1 tokenizers whose slow loader has an incompatible signature. The hint suggests the fast (tokenizers-based) alternative.

Source

Thrown at python/sglang/srt/utils/hf_transformers/tokenizer.py:188

        tokenizer_name = client.get_local_dir()

    return tokenizer_name


def _auto_tokenizer_from_pretrained(tokenizer_name, *args, **common_kwargs):
    """Call ``AutoTokenizer.from_pretrained`` with error handling."""
    try:
        tokenizer = AutoTokenizer.from_pretrained(
            tokenizer_name, *args, **common_kwargs
        )
        return tokenizer
    except TypeError as e:
        err_msg = (
            "Failed to load the tokenizer. If you are using a LLaMA V1 model "
            f"consider using '{_FAST_LLAMA_TOKENIZER}' instead of the "
            "original tokenizer."
        )
        raise RuntimeError(err_msg) from e
    except ValueError as e:
        # MistralCommon tokenizers reject standard HF kwargs like
        # trust_remote_code, use_fast etc. Retry without them.
        if "are not supported by" in str(e) and "MistralCommon" in str(e):
            return retry_without_mistral_common_kwargs(
                tokenizer_name, *args, **common_kwargs
            )
        # If the error pertains to the tokenizer class not existing or not
        # currently being imported, suggest using the --trust-remote-code flag.
        if not common_kwargs.get("trust_remote_code") and (
            "does not exist or is not currently imported." in str(e)
            or "requires you to execute the tokenizer file" in str(e)
        ):
            err_msg = (
                "Failed to load the tokenizer. If the tokenizer is a custom "
                "tokenizer not yet available in the HuggingFace transformers "
                "library, consider setting `trust_remote_code=True` in LLM "
                "or using the `--trust-remote-code` flag in the CLI."

View on GitHub (pinned to 0132848349)

Solutions

1. Use the fast tokenizer variant indicated by _FAST_LLAMA_TOKENIZER (e.g. --tokenizer-mode mistramask-free fast variant / hf-internal-testing/llama-fast-tokenizer style name)
2. Upgrade transformers so the tokenizer class signature matches
3. Re-point tokenizer to a compatible tokenizer.json-based repo

# pre-check tokenizer type from tokenizer_config.json before loading

try:
    AutoTokenizer.from_pretrained(name)
except RuntimeError as e:
    if 'FAST_LLAMA' in str(e) or 'LLaMA V1' in str(e): load the fast LLaMA tokenizer variant instead

When it happens

Trigger: Loading an original LLaMA V1 HF repo whose tokenizer implementation raises TypeError with the current transformers version.

Common situations: Old LLaMA V1 models/met repos, or transformers API changes breaking old tokenizer classes.

Related errors

• Failed to load the tokenizer. If the tokenizer is a custom t
• Using a slow tokenizer. This might cause a significant slowd
• Cannot put argument inside a f-string. This is not compatibl
• Cannot load PE model: 'model_max_length' not found in {os.pa
• Cosmos3TokenizationStage requires a tokenizer; expected the

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/7ebf97e5079fea65. Report an issue: GitHub.