sgl-project/sglang · error · ValueError

Cannot use the fast tokenizer in slow tokenizer mode.

Error message

Cannot use the fast tokenizer in slow tokenizer mode.

What it means

SGLang's get_tokenizer raises this when the server is configured with --tokenizer-mode=slow but the caller also passes use_fast=True in the tokenizer kwargs. The slow mode explicitly forces use_fast=False, and passing an explicit conflicting True is treated as a configuration error rather than silently ignored.

Source

Thrown at python/sglang/srt/utils/hf_transformers/tokenizer.py:491

    tokenizer_mode: str = "auto",
    trust_remote_code: bool = False,
    tokenizer_revision: Optional[str] = None,
    tokenizer_backend: str = "huggingface",
    **kwargs,
) -> Union[PreTrainedTokenizer, PreTrainedTokenizerFast]:
    """Gets a tokenizer for the given model name via Huggingface."""
    # Tiktoken format has its own backend — no fastokens patching needed.
    if tokenizer_name.endswith(".json"):
        from sglang.srt.tokenizer.tiktoken_tokenizer import TiktokenTokenizer

        return TiktokenTokenizer(tokenizer_name)

    if tokenizer_backend == "fastokens":
        _ensure_fastokens_patched()

    if tokenizer_mode == "slow":
        if kwargs.get("use_fast", False):
            raise ValueError("Cannot use the fast tokenizer in slow tokenizer mode.")
        kwargs["use_fast"] = False
    elif tokenizer_mode == "auto":
        # Transformers v5 AutoTokenizer ignores use_fast (always fast), but
        # some code paths pass kwargs to non-AutoTokenizer loaders where
        # use_fast still matters. Set explicitly for those fallback paths.
        if "use_fast" not in kwargs:
            kwargs["use_fast"] = True

    if (
        check_gguf_file(tokenizer_name)
        and gguf_sidecar_dir(tokenizer_name, "tokenizer_config.json") is None
        and has_native_gguf_support(tokenizer_name)
    ):
        _ensure_gguf_version()
        tokenizer = build_gguf_tokenizer(tokenizer_name)
        _fix_special_tokens_pattern(tokenizer)
        attach_additional_stop_token_ids(tokenizer)
        return patch_tokenizer(tokenizer)

View on GitHub (pinned to 0132848349)

Solutions

1. Remove use_fast=True from kwargs (or set use_fast=False) when tokenizer_mode is "slow".
2. Change --tokenizer-mode to "auto" or "fast" if the fast tokenizer is actually wanted.
3. If wrapping get_tokenizer, sanitize kwargs: kwargs.setdefault("use_fast", False) and never override it in slow mode.

Example fix

# before
get_tokenizer("meta-llama/Llama-3-8B", tokenizer_mode="slow", use_fast=True)

# after
get_tokenizer("meta-llama/Llama-3-8B", tokenizer_mode="slow", use_fast=False)

kwargs.pop("use_fast", None)
if tokenizer_mode == "slow":
    kwargs["use_fast"] = False

When it happens

Trigger: Calling get_tokenizer()/get_processor() with tokenizer_mode="slow" while kwargs contains {"use_fast": True} — e.g., a launcher or config that sets both --tokenizer-mode slow and a fast-tokenizer flag.

Common situations: Users switching to slow tokenizer mode (often to work around a fast-tokenizer bug or missing tokenizers library) while a config file, env-derived kwarg, or wrapper still sets use_fast=True. Custom model loaders that always pass use_fast=True.

Related errors

• KernelSpec.target must be 'module:attr', got {self.target!r}
• PR #{pr_num} revert is not registered; available: {sorted(_P
• Unknown stop criteria: {self.stop_criteria}
• Invalid quantization method: {quantization}. Available metho
• backend must be a non-empty string

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cf6106a3d0a6030d. Report an issue: GitHub.