sgl-project/sglang · error · RuntimeError

Retry with use_fast=False for {tokenizer_name} also failed (

Error message

Retry with use_fast=False for {tokenizer_name} also failed (initial load returned TokenizersBackend): {e}

What it means

When the first AutoTokenizer load returned a generic TokenizersBackend instead of a concrete class, sglang retries with use_fast=False; that retry also raised, and this RuntimeError wraps the retry failure with its cause.

Source

Thrown at python/sglang/srt/utils/hf_transformers/tokenizer.py:232

    In transformers v5, ``AutoTokenizer`` falls back to ``TokenizersBackend``
    when the model_type has no tokenizer mapping.  This retries with
    ``use_fast=False``, then attempts loading by the class declared in
    ``tokenizer_config.json``.  May still return a ``TokenizersBackend``
    if all retries fail (with a warning).
    """
    logger.debug(
        "Tokenizer loaded as generic TokenizersBackend for %s, "
        "retrying with use_fast=False",
        tokenizer_name,
    )
    common_kwargs = {**common_kwargs, "use_fast": False}
    try:
        tokenizer = AutoTokenizer.from_pretrained(
            tokenizer_name, *args, **common_kwargs
        )
    except (ValueError, TypeError, OSError, ImportError, RuntimeError) as e:
        raise RuntimeError(
            f"Retry with use_fast=False for {tokenizer_name} also failed "
            f"(initial load returned TokenizersBackend): {e}"
        ) from e

    if type(tokenizer).__name__ == _TOKENIZERS_BACKEND:
        tokenizer = (
            _load_tokenizer_by_declared_class(tokenizer_name, *args, **common_kwargs)
            or tokenizer
        )

    if type(tokenizer).__name__ == _TOKENIZERS_BACKEND:
        if common_kwargs.get("trust_remote_code"):
            logger.warning(
                "Tokenizer for %s is still TokenizersBackend after retries "
                "with --trust-remote-code. Model-specific tokenizer attributes "
                "may be missing.",
                tokenizer_name,
            )

View on GitHub (pinned to 0132848349)

Solutions

1. Upgrade (or pin) transformers to a version matching the model's tokenizer support
2. Prefer a repo with a proper tokenizer.json plus concrete tokenizer class
3. Inspect the chained exception e for the real cause (missing file, unsupported type, etc.)

try:
    get_tokenizer(model)
except RuntimeError as e:
    if 'use_fast=False' in str(e): pin/upgrade transformers to the model's recommended version, then retry

When it happens

Trigger: Loading a tokenizer whose fast path yields TokenizersBackend and whose slow (Python) path is also broken or unavailable for the installed transformers version.

Common situations: New tokenizers on transformers versions where the backend shim is incomplete, or missing slow tokenizer implementations.

Related errors

• Unexpected return type from apply_chat_template: {type(resul
• No pre-tokenizer regex known for tokenizer.ggml.pre={pre_nam
• Serve backend {name!r} uses API version {backend.api_version
• scalar_type_id {scalar_type_id} doesn't exists.
• f"flash attention version {fa_ver} is not supported."

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/b14a075a4d5313e5. Report an issue: GitHub.