sgl-project/sglang · error · ValueError

stop={stop_strs!r} is unavailable when skip_tokenizer_init=T

Error message

stop={stop_strs!r} is unavailable when skip_tokenizer_init=True (requires tokenizer to decode tokens to text for matching).

What it means

When the server is started with --skip-tokenizer-init (tokenizer is None), stop strings cannot be matched because matching requires decoding generated tokens to text. normalize() calls raise_if_tokenizer_required, which rejects stop_strs in that mode.

Source

Thrown at python/sglang/srt/sampling/sampling_params.py:324

    return total


def raise_if_tokenizer_required(
    tokenizer, stop_strs, stop_regex_strs, min_new_tokens=0
):
    """Raise ValueError if tokenizer-dependent features are used without a tokenizer.

    String-based stop conditions (stop_strs, stop_regex_strs) require tokenizer.decode()
    to convert output token IDs to text for matching. min_new_tokens requires the
    tokenizer's eos_token_id to penalize. When skip_tokenizer_init=True, these cannot
    be used.
    """
    if tokenizer is not None:
        return

    if stop_strs:
        raise ValueError(
            f"stop={stop_strs!r} is unavailable when skip_tokenizer_init=True "
            "(requires tokenizer to decode tokens to text for matching)."
        )
    if stop_regex_strs:
        raise ValueError(
            f"stop_regex={stop_regex_strs!r} is unavailable when skip_tokenizer_init=True "
            "(requires tokenizer to decode tokens to text for matching)."
        )
    if min_new_tokens > 0:
        raise ValueError(
            f"min_new_tokens={min_new_tokens} is unavailable when skip_tokenizer_init=True "
            "(requires tokenizer for eos_token_id)."
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Remove stop strings; use stop_token_ids (integer ids) instead.
2. Or restart the server without --skip-tokenizer-init if text-level stopping is required.

Example fix

# before
SamplingParams(stop=["</answer>"])  # with --skip-tokenizer-init
# after
SamplingParams(stop_token_ids=[tokenizer.convert_tokens_to_ids("</answer>")])

if skip_tokenizer_init and params.get('stop'):
    params['stop_token_ids'] = [tid for tid in to_ids(params.pop('stop'))]
    params['stop'] = None

def stop_ok_without_tokenizer(params: SamplingParams) -> bool:
    return not params.stop_strs

When it happens

Trigger: Server launched with --skip-tokenizer-init (input_ids passed directly) and the request includes SamplingParams(stop=["<some text>"]) or stop_token_ids containing strings.

Common situations: Benchmarking/prefill-token pipelines that skip tokenization but reuse a client template with string stop conditions; migrating a workload to token-id-only serving without stripping text-based stop params.

Related errors

• stop_regex={stop_regex_strs!r} is unavailable when skip_toke
• min_new_tokens={min_new_tokens} is unavailable when skip_tok
• beam_width must be at least 1, got {self.beam_width}.
• temperature must be a non-negative finite number, got {self.
• top_p must be in (0, 1], got {self.top_p}.

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/cad82f4e0d957a07. Report an issue: GitHub.