sgl-project/sglang · error · ValueError

The input_ids {seq} contains values greater than the vocab s

Error message

The input_ids {seq} contains values greater than the vocab size ({vocab_size}).

What it means

Raised by _validate_input_ids_in_vocab when a batch of pre-tokenized sequences contains at least one id >= vocab_size. SGLang validates supplied input_ids before dispatch so the embedding lookup on the worker will not fail opaquely.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager.py:1327

        vocab_size = self.model_config.vocab_size
        for token_id in token_ids_logprob:
            if not isinstance(token_id, int):
                raise ValueError("token_ids_logprob must be a flat list of integers.")
            if token_id < 0 or token_id >= vocab_size:
                raise ValueError(
                    f"token_ids_logprob contains out-of-vocabulary token id "
                    f"{token_id}; valid range is [0, {vocab_size})."
                )

    def _validate_input_ids_in_vocab(
        self, input_ids: Union[List[int], List[List[int]]], vocab_size: int
    ) -> None:
        # Handle both single sequence and batch of sequences
        if isinstance(input_ids[0], list):
            # Batch of sequences
            for seq in input_ids:
                if any(id >= vocab_size for id in seq):
                    raise ValueError(
                        f"The input_ids {seq} contains values greater than the vocab size ({vocab_size})."
                    )
        else:
            # Single sequence
            if any(id >= vocab_size for id in input_ids):
                raise ValueError(
                    f"The input_ids {input_ids} contains values greater than the vocab size ({vocab_size})."
                )

    def _create_tokenized_object(
        self,
        obj: Union[GenerateReqInput, EmbeddingReqInput],
        input_text: str,
        input_ids: Optional[List[int]],
        input_embeds: Optional[List[List[float]]] = None,
        mm_inputs=None,
        token_type_ids: Optional[List[int]] = None,
    ) -> Union[TokenizedGenerateReqInput, TokenizedEmbeddingReqInput]:

View on GitHub (pinned to 0132848349)

Solutions

1. Re-tokenize the text with the served model's tokenizer
2. Check max(input_ids) against model_config.vocab_size and filter offending sequences
3. Regenerate any cached/cached-to-disk token ids after model changes

Example fix

# before
input_ids=[[999999, 2], [5, 6]]
# after
input_ids=[enc.ids for enc in tokenizer.encode_batch(texts)]

vocab = client.get_server_info()['model_config']['vocab_size']
assert all(all(i < vocab for i in seq) for seq in input_ids), 'id exceeds vocab'

def batch_ids_in_vocab(seqs, vocab): return all(all(0 <= i < vocab for i in s) for s in seqs)

except ValueError as e: if 'vocab size' in str(e): re-tokenize batch with served tokenizer and retry

When it happens

Trigger: Sending GenerateReqInput(input_ids=[[...],[...]]) as a batch where any sequence has an id greater than or equal to the served model's vocab size.

Common situations: Tokenizing with a mismatched tokenizer, merging ids from a different model, stale cached tokenizations after switching the served model.

Related errors

• The input_ids {input_ids} contains values greater than the v
• Given arguments mismatch the SGL function signature
• token_ids_logprob contains out-of-vocabulary token id {token
• borrow_torch_tensors requires MPS tensors, got {devices}
• Unknown serve backend {name!r}. Available values: {available

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/4d3da77d36d2f175. Report an issue: GitHub.