{"record":{"id":"a136d84d1ec7ef3c","repo":"sgl-project/sglang","slug":"input-contains-len-positions-occurrences-of-emb","errorCode":null,"errorMessage":"input contains {len(positions)} occurrences of embed_override_token_id={token_id}, but embed_overrides has {len(embeds)} entries.","messagePattern":"input contains (.+?) occurrences of embed_override_token_id=(.+?), but embed_overrides has (.+?) entries\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager.py","lineNumber":1461,"sourceCode":"                http_worker_ipc=obj.http_worker_ipc,\n                return_pooled_hidden_states=obj.return_pooled_hidden_states,\n                multi_item_delimiter_indices=obj.multi_item_delimiter_indices,\n            )\n\n        tokenized_obj.time_stats = self.rid_to_state[obj.rid].time_stats\n        self.rid_to_state[obj.rid].time_stats.set_tokenize_finish_time()\n\n        return tokenized_obj\n\n    @staticmethod\n    def _resolve_embed_overrides(\n        input_ids: array[int],\n        token_id: int,\n        embeds: List[torch.Tensor],\n    ) -> PositionalEmbeds:\n        positions = [idx for idx, tok in enumerate(input_ids) if tok == token_id]\n        if len(positions) != len(embeds):\n            raise ValueError(\n                f\"input contains {len(positions)} occurrences of \"\n                f\"embed_override_token_id={token_id}, \"\n                f\"but embed_overrides has {len(embeds)} entries.\"\n            )\n        return PositionalEmbeds(embeds=embeds, positions=positions)\n\n    async def _batch_tokenize_and_process(\n        self, batch_size: int, obj: Union[GenerateReqInput, EmbeddingReqInput]\n    ) -> List[Union[TokenizedGenerateReqInput, TokenizedEmbeddingReqInput]]:\n        \"\"\"Handle batch tokenization for text inputs only.\"\"\"\n        logger.debug(f\"Starting batch tokenization for {batch_size} text requests\")\n\n        # If batch does not have text nothing to tokenize\n        # so lets construct the return object\n        if not self._batch_has_text(batch_size, obj):\n            # All requests already have input_ids, no need to tokenize\n            return [await self._tokenize_one_request(obj[i]) for i in range(batch_size)]\n","sourceCodeStart":1443,"sourceCodeEnd":1479,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager.py#L1443-L1479","documentation":"Raised by _resolve_embed_overrides when the count of placeholder tokens equal to embed_override_token_id in input_ids does not match the number of tensors in embed_overrides. Each override tensor must replace exactly one placeholder occurrence, so counts must match 1:1.","triggerScenarios":"Calling generate with input_embeds overrides where embed_overrides has fewer/more tensors than the number of embed_override_token_id tokens present in the tokenized input, e.g. num image patches changed but the placeholder token count in the text did not.","commonSituations":"Editing prompt templates that contain a different number of placeholder tokens than the multimodal encoder produced embeddings for; chunking embeddings without adjusting placeholders; stale template after model changes.","solutions":["Count placeholders: input_ids.count(embed_override_token_id) must equal len(embed_overrides)","Adjust the prompt/template so placeholder count matches the number of override embeddings","Regenerate embed_overrides from the same source that produced the tokenized input"],"exampleFix":"# before\ninput_ids=[1, 999, 999, 2]; embed_overrides=[img_emb]  # 1 embed, 2 placeholders\n# after\ninput_ids=[1, 999, 2]; embed_overrides=[img_emb]","handlingStrategy":"validation","validationCode":"n_placeholders = sum(input_ids.count(t) if isinstance(input_ids[0], list) else input_ids.count(t) for t in [embed_override_token_id])\nassert n_placeholders == len(embed_overrides), f'{n_placeholders} placeholders vs {len(embed_overrides)} embeds'","typeGuard":null,"tryCatchPattern":"except ValueError as e: if 'embed_override_token_id' in str(e): recount placeholders and rebuild overrides","preventionTips":["Generate template placeholders and embeddings from the same preprocessing step","Unit-test placeholder count == len(embeds) for each template"],"tags":["sglang","embed-overrides","multimodal","count-mismatch"],"backgroundTag":"embedding-placeholder-count-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}