sgl-project/sglang · error · RuntimeError

Failed to load the tokenizer. If the tokenizer is a custom t

Error message

Failed to load the tokenizer. If the tokenizer is a custom tokenizer not yet available in the HuggingFace transformers library, consider setting `trust_remote_code=True` in LLM or using the `--trust-remote-code` flag in the CLI.

What it means

AutoTokenizer.from_pretrained failed with a non-recoverable error (not the auto-retried MistralCommon kwarg case): typically the tokenizer class can't be found because the model uses custom remote code and trust_remote_code was not enabled.

Source

Thrown at python/sglang/srt/utils/hf_transformers/tokenizer.py:208

        # MistralCommon tokenizers reject standard HF kwargs like
        # trust_remote_code, use_fast etc. Retry without them.
        if "are not supported by" in str(e) and "MistralCommon" in str(e):
            return retry_without_mistral_common_kwargs(
                tokenizer_name, *args, **common_kwargs
            )
        # If the error pertains to the tokenizer class not existing or not
        # currently being imported, suggest using the --trust-remote-code flag.
        if not common_kwargs.get("trust_remote_code") and (
            "does not exist or is not currently imported." in str(e)
            or "requires you to execute the tokenizer file" in str(e)
        ):
            err_msg = (
                "Failed to load the tokenizer. If the tokenizer is a custom "
                "tokenizer not yet available in the HuggingFace transformers "
                "library, consider setting `trust_remote_code=True` in LLM "
                "or using the `--trust-remote-code` flag in the CLI."
            )
            raise RuntimeError(err_msg) from e
        raise


def _resolve_tokenizers_backend(tokenizer_name, *args, **common_kwargs):
    """Resolve generic ``TokenizersBackend`` to a proper tokenizer class.

    In transformers v5, ``AutoTokenizer`` falls back to ``TokenizersBackend``
    when the model_type has no tokenizer mapping.  This retries with
    ``use_fast=False``, then attempts loading by the class declared in
    ``tokenizer_config.json``.  May still return a ``TokenizersBackend``
    if all retries fail (with a warning).
    """
    logger.debug(
        "Tokenizer loaded as generic TokenizersBackend for %s, "
        "retrying with use_fast=False",
        tokenizer_name,
    )
    common_kwargs = {**common_kwargs, "use_fast": False}

View on GitHub (pinned to 0132848349)

Solutions

1. Set trust_remote_code=True (LLM(...trust_remote_code=True)) or --trust-remote-code on the CLI
2. Upgrade transformers so the tokenizer is natively supported
3. Verify the model path/revision actually contains tokenizer files

Example fix

# before
llm = LLM(model=..., trust_remote_code=False)
# after
llm = LLM(model=..., trust_remote_code=True)

cfg = json.load(open(Path(model)/'tokenizer_config.json'))
needs_trust = 'auto_map' in cfg or cfg.get('tokenizer_class') not in KNOWN_HF_TOKENIZERS

try:
    AutoTokenizer.from_pretrained(name)
except RuntimeError as e:
    if 'trust_remote_code' in str(e): retry with trust_remote_code=True (only for trusted sources)

When it happens

Trigger: Loading a tokenizer whose class lives in the model repo's remote code without --trust-remote-code / trust_remote_code=True.

Common situations: Custom or newly published tokenizers not yet merged into transformers.

Related errors

• Failed to load the tokenizer. If you are using a LLaMA V1 mo
• Using a slow tokenizer. This might cause a significant slowd
• Cannot load PE model: 'model_max_length' not found in {os.pa
• Cosmos3TokenizationStage requires a tokenizer; expected the
• Unexpected return type from apply_chat_template: {type(resul

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1f15cfd883cc6099. Report an issue: GitHub.