{"record":{"id":"ff286bc18e953f00","repo":"sgl-project/sglang","slug":"token-ids-logprob-contains-out-of-vocabulary-token","errorCode":null,"errorMessage":"token_ids_logprob contains out-of-vocabulary token id {token_id}; valid range is [0, {vocab_size}).","messagePattern":"token_ids_logprob contains out-of-vocabulary token id (.+?); valid range is \\[0, (.+?)\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager.py","lineNumber":1314,"sourceCode":"                f\"Provided dimensions are greater than max embedding dimension: {self.model_config.hidden_size}\"\n            )\n\n    def _validate_token_ids_logprob(self, obj: GenerateReqInput) -> None:\n        # Batch requests are split into per-request sub-objects before this\n        # runs (normalize_batch_and_arguments + __getitem__), so the only\n        # legal shape here is the per-request contract of\n        # TokenizedGenerateReqInput.token_ids_logprob: a flat list of ints.\n        token_ids_logprob = obj.token_ids_logprob\n        if not token_ids_logprob:\n            return\n        if not isinstance(token_ids_logprob, list):\n            raise ValueError(\"token_ids_logprob must be a flat list of integers.\")\n        vocab_size = self.model_config.vocab_size\n        for token_id in token_ids_logprob:\n            if not isinstance(token_id, int):\n                raise ValueError(\"token_ids_logprob must be a flat list of integers.\")\n            if token_id < 0 or token_id >= vocab_size:\n                raise ValueError(\n                    f\"token_ids_logprob contains out-of-vocabulary token id \"\n                    f\"{token_id}; valid range is [0, {vocab_size}).\"\n                )\n\n    def _validate_input_ids_in_vocab(\n        self, input_ids: Union[List[int], List[List[int]]], vocab_size: int\n    ) -> None:\n        # Handle both single sequence and batch of sequences\n        if isinstance(input_ids[0], list):\n            # Batch of sequences\n            for seq in input_ids:\n                if any(id >= vocab_size for id in seq):\n                    raise ValueError(\n                        f\"The input_ids {seq} contains values greater than the vocab size ({vocab_size}).\"\n                    )\n        else:\n            # Single sequence\n            if any(id >= vocab_size for id in input_ids):","sourceCodeStart":1296,"sourceCodeEnd":1332,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager.py#L1296-L1332","documentation":"Raised when a token id in token_ids_logprob falls outside [0, vocab_size) for the loaded model. SGLang validates requested logprob tokens against model_config.vocab_size so the sampler does not index nonexistent rows.","triggerScenarios":"Requesting logprobs for token ids from a different tokenizer/vocabulary than the served model, e.g. ids near 151k for a 32k-vocab model, or negative ids.","commonSituations":"Reusing ids computed with another tokenizer or model revision, hardcoding ids copied from a different model's output, off-by-one vocab size assumptions after a tokenizer upgrade.","solutions":["Re-derive the ids with the tokenizer of the served model (tokenizer.encode or decode round-trip)","Print model_config.vocab_size and clamp/filter ids to [0, vocab_size)","Drop negative or oversized ids from token_ids_logprob"],"exampleFix":"# before\ntoken_ids_logprob=[200000, 5]  # model vocab_size=32000\n# after\ntoken_ids_logprob=[t for t in candidates if 0 <= t < vocab_size]","handlingStrategy":"validation","validationCode":"vocab_size = client.get_server_info()['model_config']['vocab_size']\ntoken_ids_logprob = [t for t in token_ids_logprob if 0 <= t < vocab_size]","typeGuard":"def in_vocab(ids, vocab_size): return all(0 <= t < vocab_size for t in ids)","tryCatchPattern":"except ValueError as e: if 'out-of-vocabulary' in str(e): filter ids and retry","preventionTips":["Derive candidate ids from the served model's tokenizer only","Filter ids against vocab_size before sending"],"tags":["sglang","vocab","out-of-range","logprob"],"backgroundTag":"token-id-out-of-vocabulary","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}