sgl-project/sglang · error · ValueError

The input_ids {input_ids} contains values greater than the v

Error message

The input_ids {input_ids} contains values greater than the vocab size ({vocab_size}).

What it means

Raised by _validate_input_ids_in_vocab when a single pre-tokenized input_ids sequence contains an id >= vocab_size. Same guard as the batch path, applied to the flat list form.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager.py:1333

                    f"token_ids_logprob contains out-of-vocabulary token id "
                    f"{token_id}; valid range is [0, {vocab_size})."
                )

    def _validate_input_ids_in_vocab(
        self, input_ids: Union[List[int], List[List[int]]], vocab_size: int
    ) -> None:
        # Handle both single sequence and batch of sequences
        if isinstance(input_ids[0], list):
            # Batch of sequences
            for seq in input_ids:
                if any(id >= vocab_size for id in seq):
                    raise ValueError(
                        f"The input_ids {seq} contains values greater than the vocab size ({vocab_size})."
                    )
        else:
            # Single sequence
            if any(id >= vocab_size for id in input_ids):
                raise ValueError(
                    f"The input_ids {input_ids} contains values greater than the vocab size ({vocab_size})."
                )

    def _create_tokenized_object(
        self,
        obj: Union[GenerateReqInput, EmbeddingReqInput],
        input_text: str,
        input_ids: Optional[List[int]],
        input_embeds: Optional[List[List[float]]] = None,
        mm_inputs=None,
        token_type_ids: Optional[List[int]] = None,
    ) -> Union[TokenizedGenerateReqInput, TokenizedEmbeddingReqInput]:
        """Create a tokenized request object from common parameters."""
        input_ids_arr: Optional[array[int]] = (
            array("q", input_ids) if input_ids is not None else None
        )
        # Parse sampling parameters
        # Note: if there are preferred sampling params, we use them if they are not

View on GitHub (pinned to 0132848349)

Solutions

Verify ids against the served model: assert max(input_ids) < vocab_size
Re-encode the original text with the current tokenizer
Avoid copying token ids between models

Example fix

# before
resp = client.generate(input_ids=[131072, 11], sampling_params={...})
# after
resp = client.generate(input_ids=tokenizer.encode(prompt), sampling_params={...})

Defensive patterns

Strategy: validation

Validate before calling

assert max(input_ids, default=0) < vocab_size, f'{max(input_ids)} >= {vocab_size}'

Type guard

def ids_in_vocab(ids, vocab): return all(0 <= i < vocab for i in ids)

Try / catch

except ValueError as e: if 'vocab size' in str(e): re-encode text and retry

Prevention

Never copy ids across models/tokenizers
Assert max id < vocab before submit

When it happens

Trigger: Sending GenerateReqInput(input_ids=[...]) where any id exceeds or equals the served model's vocab size (note: negative ids pass this check but typically fail earlier/elsewhere).

Common situations: Using input_ids produced by a different tokenizer version, manual id arithmetic overflowing, LoRA/merged-vocab mismatches.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/1149ee2928ea852b. Report an issue: GitHub.