sgl-project/sglang · critical · ValueError

--enable-strict-thinking requires a grammar backend with tok

Error message

--enable-strict-thinking requires a grammar backend with token filtering support, but XGrammar failed to initialize: {e}. Cannot fall back to grammar_backend='none' with strict thinking enabled.

What it means

With --enable-strict-thinking, SGLang needs a grammar backend that supports token filtering. If the XGrammar backend fails to initialize (TokenizerNotSupportedError), create_grammar_backend refuses to silently fall back to no grammar backend and raises this ValueError.

Source

Thrown at python/sglang/srt/constrained/base_grammar_backend.py:392

    elif name == "xgrammar":
        from sglang.srt.constrained.xgrammar_backend import (
            TokenizerNotSupportedError,
            XGrammarGrammarBackend,
        )

        # Convert Set[int] to List[int] if needed
        eos_list = list(eos_token_ids) if eos_token_ids else None

        try:
            grammar_backend = XGrammarGrammarBackend(
                tokenizer,
                vocab_size=vocab_size,
                model_eos_token_ids=eos_list,
                any_whitespace=not get_serving().constrained_json_disable_any_whitespace,
            )
        except TokenizerNotSupportedError as e:
            if get_serving().enable_strict_thinking:
                raise ValueError(
                    f"--enable-strict-thinking requires a grammar backend with "
                    f"token filtering support, but XGrammar failed to initialize: "
                    f"{e}. Cannot fall back to grammar_backend='none' with strict "
                    f"thinking enabled."
                ) from e
            logger.warning(
                f"Grammar backend disabled because tokenizer is not supported by XGrammar: {e}. "
                "Falling back to grammar_backend='none'. "
                "Structured outputs (JSON schema, regex, EBNF) will not be available."
            )
            get_context().override("grammar.import_fallback", grammar_backend="none")
            return None
    elif name == "llguidance":
        from sglang.srt.constrained.llguidance_backend import GuidanceBackend

        grammar_backend = GuidanceBackend(
            tokenizer=tokenizer,
            any_whitespace=not get_serving().constrained_json_disable_any_whitespace,

View on GitHub (pinned to 0132848349)

Solutions

1. Remove --enable-strict-thinking if strict reasoning-format enforcement is not required
2. Switch to a model whose tokenizer is supported by XGrammar (standard BPE/SentencePiece HF tokenizers)
3. Upgrade xgrammar and transformers; check the chained cause '{e}' for the underlying tokenizer conversion failure

Example fix

# before
python -m sglang.launch_server --model ... --enable-strict-thinking
# after
python -m sglang.launch_server --model ...  # no strict thinking, or use an xgrammar-compatible model

try:
    backend = create_grammar_backend(args)
except ValueError as e:
    if 'strict-thinking' in str(e):
        logging.error('Disable --enable-strict-thinking or use an xgrammar-compatible tokenizer')
    raise

When it happens

Trigger: Launching the server with --enable-strict-thinking (and default xgrammar grammar backend) where the model's tokenizer cannot be turned into an XGrammar TokenizerInfo (e.g. unusual tokenizer type or failed from_huggingface conversion).

Common situations: New or exotic tokenizer architectures unsupported by xgrammar; version mismatch between transformers/xgrammar; strict-thinking feature enabled on a model whose tokenizer lacks xgrammar support.

Related errors

• --enable-strict-thinking requires a grammar backend that sup
• Invalid grammar backend: {name}
• think_end_token '{reasoning_parser.detector.think_end_token}
• Failed to get server info. {error_data['error']['message']}
• world_size ({world_size}) is less than tensor_parallel_degre

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/55c424669a58968c. Report an issue: GitHub.