{"record":{"id":"8eec1ef267b7a7d4","repo":"sgl-project/sglang","slug":"label-contains-len-positions-occurrences-of-e","errorCode":null,"errorMessage":"{label} contains {len(positions)} occurrences of embed_override_token_id={embed_override_token_id}, but {len(embeds)} override embeddings were provided.","messagePattern":"(.+?) contains (.+?) occurrences of embed_override_token_id=(.+?), but (.+?) override embeddings were provided\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager_score_mixin.py","lineNumber":288,"sourceCode":"    def _resolve_overrides_for_sequence(\n        self,\n        token_ids: List[int],\n        embeds: Optional[List[torch.Tensor]],\n        embed_override_token_id: int,\n        position_offset: int = 0,\n        label: str = \"input\",\n    ) -> Tuple[List[torch.Tensor], List[int]]:\n        \"\"\"Scan token_ids for placeholder occurrences and pair with embeddings.\n        Returns empty lists when embeds is None.\"\"\"\n        if embeds is None:\n            return [], []\n        positions = [\n            idx + position_offset\n            for idx, tok in enumerate(token_ids)\n            if tok == embed_override_token_id\n        ]\n        if len(positions) != len(embeds):\n            raise ValueError(\n                f\"{label} contains {len(positions)} occurrences of \"\n                f\"embed_override_token_id={embed_override_token_id}, \"\n                f\"but {len(embeds)} override embeddings were provided.\"\n            )\n        return embeds, positions\n\n    def _resolve_embed_overrides_for_request(\n        self,\n        query: List[int],\n        item: List[int],\n        embed_override_token_id: int,\n        query_embed_overrides: Optional[List[torch.Tensor]],\n        item_embeds: Optional[List[torch.Tensor]],\n        item_position_offset: int,\n        item_label: str,\n    ) -> Optional[PositionalEmbeds]:\n        \"\"\"Resolve embed overrides for a query+item pair; None when no overrides exist.\"\"\"\n        q_embeds, q_positions = self._resolve_overrides_for_sequence(","sourceCodeStart":270,"sourceCodeEnd":306,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager_score_mixin.py#L270-L306","documentation":"When embed overrides are used, every occurrence of embed_override_token_id in the tokenized sequence must have exactly one matching embedding vector. This error reports the count mismatch between placeholder token occurrences and provided embeddings, per label (query/item).","triggerScenarios":"Calling scoring APIs with query_embed_overrides or item_embed_overrides whose length differs from the number of embed_override_token_id placeholders present in query/items token ids; also when position_offset shifts or a tokenizer inserts extra placeholder tokens.","commonSituations":"Embedding lists built from a different tokenization than the one used server-side; forgetting that both query and item can contain the placeholder; off-by-one after truncation; multimodal embeddings pipelines updated independently of prompts.","solutions":["Count occurrences of embed_override_token_id in your tokenized query/items and match embeds length exactly","Re-tokenize with the same tokenizer/settings used to build the embeddings","Ensure separate overrides for query vs item each match their own segment counts"],"exampleFix":"# before\nembeds = torch.randn(3, 4096)\nresult = engine.score_request(..., query_embed_overrides=embeds,\n    items=[\"text <placeholder>\"], embed_override_token_id=PH_TOK)  # 1 occurrence != 3\n\n# after\nn = query_token_ids.count(PH_TOK)\nresult = engine.score_request(..., query_embed_overrides=embeds[:n], ...)","handlingStrategy":"validation","validationCode":"n_q = sum(1 for t in query_token_ids if t == EMBED_TOK)\nn_i = sum(1 for t in item_token_ids if t == EMBED_TOK) if items is not None else 0\nassert len(query_embeds or []) == n_q\nassert len(item_embeds or []) == n_i","typeGuard":"def overrides_match(query_ids, q_embeds, item_ids, i_embeds, tok) -> bool:\n    return (len(q_embeds or []) == sum(1 for t in query_ids if t == tok)\n            and len(i_embeds or []) == sum(1 for t in (item_ids or []) if t == tok))","tryCatchPattern":"try:\n    r = engine.score_request(...)\nexcept ValueError as e:\n    if \"override embeddings were provided\" in str(e):\n        raise ValueError(f\"embed count mismatch: {e}\") from e\n    raise","preventionTips":["Derive embedding counts from the exact tokenized text used in the request","Keep one source of truth for the placeholder token and its count in the pipeline","Add a unit test asserting placeholder count == embeds length for each sample"],"tags":["embeddings","overrides","count-mismatch","validation"],"backgroundTag":"embedding-override-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}