{"record":{"id":"4d3da77d36d2f175","repo":"sgl-project/sglang","slug":"the-input-ids-seq-contains-values-greater-than-t","errorCode":null,"errorMessage":"The input_ids {seq} contains values greater than the vocab size ({vocab_size}).","messagePattern":"The input_ids (.+?) contains values greater than the vocab size \\((.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager.py","lineNumber":1327,"sourceCode":"        vocab_size = self.model_config.vocab_size\n        for token_id in token_ids_logprob:\n            if not isinstance(token_id, int):\n                raise ValueError(\"token_ids_logprob must be a flat list of integers.\")\n            if token_id < 0 or token_id >= vocab_size:\n                raise ValueError(\n                    f\"token_ids_logprob contains out-of-vocabulary token id \"\n                    f\"{token_id}; valid range is [0, {vocab_size}).\"\n                )\n\n    def _validate_input_ids_in_vocab(\n        self, input_ids: Union[List[int], List[List[int]]], vocab_size: int\n    ) -> None:\n        # Handle both single sequence and batch of sequences\n        if isinstance(input_ids[0], list):\n            # Batch of sequences\n            for seq in input_ids:\n                if any(id >= vocab_size for id in seq):\n                    raise ValueError(\n                        f\"The input_ids {seq} contains values greater than the vocab size ({vocab_size}).\"\n                    )\n        else:\n            # Single sequence\n            if any(id >= vocab_size for id in input_ids):\n                raise ValueError(\n                    f\"The input_ids {input_ids} contains values greater than the vocab size ({vocab_size}).\"\n                )\n\n    def _create_tokenized_object(\n        self,\n        obj: Union[GenerateReqInput, EmbeddingReqInput],\n        input_text: str,\n        input_ids: Optional[List[int]],\n        input_embeds: Optional[List[List[float]]] = None,\n        mm_inputs=None,\n        token_type_ids: Optional[List[int]] = None,\n    ) -> Union[TokenizedGenerateReqInput, TokenizedEmbeddingReqInput]:","sourceCodeStart":1309,"sourceCodeEnd":1345,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager.py#L1309-L1345","documentation":"Raised by _validate_input_ids_in_vocab when a batch of pre-tokenized sequences contains at least one id >= vocab_size. SGLang validates supplied input_ids before dispatch so the embedding lookup on the worker will not fail opaquely.","triggerScenarios":"Sending GenerateReqInput(input_ids=[[...],[...]]) as a batch where any sequence has an id greater than or equal to the served model's vocab size.","commonSituations":"Tokenizing with a mismatched tokenizer, merging ids from a different model, stale cached tokenizations after switching the served model.","solutions":["Re-tokenize the text with the served model's tokenizer","Check max(input_ids) against model_config.vocab_size and filter offending sequences","Regenerate any cached/cached-to-disk token ids after model changes"],"exampleFix":"# before\ninput_ids=[[999999, 2], [5, 6]]\n# after\ninput_ids=[enc.ids for enc in tokenizer.encode_batch(texts)]","handlingStrategy":"validation","validationCode":"vocab = client.get_server_info()['model_config']['vocab_size']\nassert all(all(i < vocab for i in seq) for seq in input_ids), 'id exceeds vocab'","typeGuard":"def batch_ids_in_vocab(seqs, vocab): return all(all(0 <= i < vocab for i in s) for s in seqs)","tryCatchPattern":"except ValueError as e: if 'vocab size' in str(e): re-tokenize batch with served tokenizer and retry","preventionTips":["Always tokenize with the served model's tokenizer","Invalidate cached tokenizations when the served model changes"],"tags":["sglang","input-ids","vocab","batch"],"backgroundTag":"token-id-out-of-vocabulary","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}