{"record":{"id":"871bf0a004f10717","repo":"sgl-project/sglang","slug":"embed-override-token-id-is-required-when-query-emb","errorCode":null,"errorMessage":"embed_override_token_id is required when query_embed_overrides or item_embed_overrides are supplied.","messagePattern":"embed_override_token_id is required when query_embed_overrides or item_embed_overrides are supplied\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager_score_mixin.py","lineNumber":492,"sourceCode":"        return_pooled_hidden_states is only supported for non-generation models\n        (SequenceClassification, RewardModel); raises ValueError for CausalLM.\n        \"\"\"\n        is_generation = self.is_generation\n\n        if is_generation and label_token_ids is None:\n            raise ValueError(\n                \"label_token_ids is required for generation (CausalLM) models.\"\n            )\n        if items is None:\n            raise ValueError(\"items must be provided\")\n        if not items:\n            return ScoreResult(scores=[], prompt_tokens=0)\n\n        has_embeds = (\n            query_embed_overrides is not None or item_embed_overrides is not None\n        )\n        if has_embeds and embed_override_token_id is None:\n            raise ValueError(\n                \"embed_override_token_id is required when query_embed_overrides \"\n                \"or item_embed_overrides are supplied.\"\n            )\n        if item_first and has_embeds:\n            raise ValueError(\"item_first is not supported when embeddings are supplied\")\n        if item_embed_overrides is not None and len(item_embed_overrides) != len(items):\n            raise ValueError(\n                f\"item_embed_overrides length ({len(item_embed_overrides)}) \"\n                f\"must match items length ({len(items)}).\"\n            )\n        if self.tokenizer is not None and label_token_ids is not None:\n            vocab_size = self.tokenizer.vocab_size\n            for token_id in label_token_ids:\n                if token_id >= vocab_size:\n                    raise ValueError(\n                        f\"Token ID {token_id} is out of vocabulary (vocab size: {vocab_size})\"\n                    )\n","sourceCodeStart":474,"sourceCodeEnd":510,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager_score_mixin.py#L474-L510","documentation":"When query_embed_overrides or item_embed_overrides are supplied, embed_override_token_id must also be given so the server knows which token marks each override slot. The has_embeds check rejects embedding overrides without a placeholder token id.","triggerScenarios":"Calling score/score_request with query_embed_overrides (and/or item_embed_overrides) while omitting embed_override_token_id.","commonSituations":"Multimodal pipelines where embeddings are passed but the placeholder token constant isn't plumbed through a config; API wrappers with the token id optional and defaulted to None.","solutions":["Pass embed_override_token_id (the token id you inserted into query/items text as placeholder) alongside the overrides","If embeddings weren't intended, remove the *_embed_overrides arguments"],"exampleFix":"# before\nresult = engine.score_request(query=..., items=...,\n    query_embed_overrides=embs)\n\n# after\nresult = engine.score_request(query=..., items=...,\n    query_embed_overrides=embs, embed_override_token_id=PLACEHOLDER_ID)","handlingStrategy":"validation","validationCode":"has_embeds = (query_embed_overrides is not None or item_embed_overrides is not None)\nif has_embeds:\n    assert embed_override_token_id is not None","typeGuard":"def embed_kwargs_consistent(kwargs: dict) -> bool:\n    has = kwargs.get(\"query_embed_overrides\") is not None or kwargs.get(\"item_embed_overrides\") is not None\n    return (not has) or kwargs.get(\"embed_override_token_id\") is not None","tryCatchPattern":null,"preventionTips":["Bundle placeholder token id and embeddings in one config object so they travel together","Add a dataclass/TypedDict for embed-override args to force completeness","Smoke-test multimodal scoring calls in CI to catch missing token id early"],"tags":["scoring","embeddings","overrides","validation"],"backgroundTag":"missing-required-parameter","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}