{"record":{"id":"f8690d202da64a20","repo":"sgl-project/sglang","slug":"token-id-token-id-is-out-of-vocabulary-vocab-si","errorCode":null,"errorMessage":"Token ID {token_id} is out of vocabulary (vocab size: {vocab_size})","messagePattern":"Token ID (.+?) is out of vocabulary \\(vocab size: (.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager_score_mixin.py","lineNumber":507,"sourceCode":"            query_embed_overrides is not None or item_embed_overrides is not None\n        )\n        if has_embeds and embed_override_token_id is None:\n            raise ValueError(\n                \"embed_override_token_id is required when query_embed_overrides \"\n                \"or item_embed_overrides are supplied.\"\n            )\n        if item_first and has_embeds:\n            raise ValueError(\"item_first is not supported when embeddings are supplied\")\n        if item_embed_overrides is not None and len(item_embed_overrides) != len(items):\n            raise ValueError(\n                f\"item_embed_overrides length ({len(item_embed_overrides)}) \"\n                f\"must match items length ({len(items)}).\"\n            )\n        if self.tokenizer is not None and label_token_ids is not None:\n            vocab_size = self.tokenizer.vocab_size\n            for token_id in label_token_ids:\n                if token_id >= vocab_size:\n                    raise ValueError(\n                        f\"Token ID {token_id} is out of vocabulary (vocab size: {vocab_size})\"\n                    )\n\n        # Check if multi-item scoring is enabled\n        use_multi_item_scoring = self.server_args.enable_mis\n\n        input_ids = None\n        text_prompts = None\n        positional_embed_overrides = None\n        delimiter_indices = None\n\n        use_text_prompts = isinstance(query, str) and not has_embeds\n\n        if use_text_prompts:\n            # Both query and items are text\n            items_list = [items] if isinstance(items, str) else items\n            if use_multi_item_scoring:\n                # Tokenize separately, then combine at token level with placeholder","sourceCodeStart":489,"sourceCodeEnd":525,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager_score_mixin.py#L489-L525","documentation":"Raised by score_request when a token id in label_token_ids is >= tokenizer.vocab_size (note: negative ids are NOT checked here and fail elsewhere). Label tokens index the model vocabulary directly, so any id outside [0, vocab_size) is invalid.","triggerScenarios":"Calling score with label_token_ids containing ids from a different tokenizer/vocabulary, ids computed after special-token offsets, or hardcoded ids copied from another model.","commonSituations":"Switching the base model but keeping hardcoded label token ids from the previous model; using ids from an added-tokens table that extends past vocab_size; converting a token string with the wrong tokenizer instance.","solutions":["Re-derive label ids with self.tokenizer.convert_tokens_to_ids(label) for the loaded model","Print tokenizer.vocab_size and the offending ids to find the mismatch","If ids come from a config file, regenerate it for the current model checkpoint"],"exampleFix":"# before\nlabel_token_ids=[151665, 151666]  # hardcoded from another model\n# after\nlabel_token_ids=[tok.convert_tokens_to_ids(t) for t in [\"<|good|>\", \"<|bad|>\"]]","handlingStrategy":"validation","validationCode":"vocab = engine.tokenizer.vocab_size\nassert all(0 <= t < vocab for t in label_token_ids), f\"ids outside [0, {vocab})\"","typeGuard":"def valid_label_ids(ids: list[int], tokenizer) -> bool:\n    return all(0 <= t < tokenizer.vocab_size for t in ids)","tryCatchPattern":"try:\n    await engine.async_score(q, d, label_token_ids=ids)\nexcept ValueError as e:\n    if \"out of vocabulary\" in str(e):\n        ids = [tok.convert_tokens_to_ids(t) for t in labels]  # re-derive","preventionTips":["Always derive label ids via convert_tokens_to_ids on the served model's tokenizer","Never hardcode token ids across model versions"],"tags":["sglang","scoring","tokenizer","out-of-vocabulary"],"backgroundTag":"token-id-out-of-vocabulary","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}