{"record":{"id":"80ef098b4e96e8e9","repo":"sgl-project/sglang","slug":"batch-tokenization-is-not-needed-for-pre-tokenized","errorCode":null,"errorMessage":"Batch tokenization is not needed for pre-tokenized input_ids. Do not set `enable_tokenizer_batch_encode`.","messagePattern":"Batch tokenization is not needed for pre-tokenized input_ids\\. Do not set `enable_tokenizer_batch_encode`\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager.py","lineNumber":1522,"sourceCode":"            tokenized_objs.append(\n                self._create_tokenized_object(\n                    req, req.text, input_ids_list[i], None, None, token_type_ids\n                )\n            )\n        logger.debug(f\"Completed batch processing for {batch_size} requests\")\n        return tokenized_objs\n\n    def _validate_batch_tokenization_constraints(\n        self, batch_size: int, obj: Union[GenerateReqInput, EmbeddingReqInput]\n    ) -> None:\n        \"\"\"Validate constraints for batch tokenization processing.\"\"\"\n        for i in range(batch_size):\n            if self.is_generation and obj[i].contains_mm_input():\n                raise ValueError(\n                    \"For multimodal input processing do not set `enable_tokenizer_batch_encode`.\"\n                )\n            if obj[i].input_ids is not None:\n                raise ValueError(\n                    \"Batch tokenization is not needed for pre-tokenized input_ids. Do not set `enable_tokenizer_batch_encode`.\"\n                )\n            if obj[i].input_embeds is not None:\n                raise ValueError(\n                    \"Batch tokenization is not needed for input_embeds. Do not set `enable_tokenizer_batch_encode`.\"\n                )\n\n    def _batch_has_text(\n        self, batch_size: int, obj: Union[GenerateReqInput, EmbeddingReqInput]\n    ) -> bool:\n        \"\"\"Check if any request in the batch contains text input.\"\"\"\n        for i in range(batch_size):\n            if obj[i].text:\n                return True\n            elif self.is_generation and obj[i].contains_mm_input():\n                return True\n\n        return False","sourceCodeStart":1504,"sourceCodeEnd":1540,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager.py#L1504-L1540","documentation":"Raised when enable_tokenizer_batch_encode is set but a request in the batch already supplies input_ids. Batch tokenization exists to accelerate text tokenization; pre-tokenized input makes it unnecessary and the combination is rejected.","triggerScenarios":"Server has enable_tokenizer_batch_encode=True and a batch request element has input_ids != None (client-side tokenized prompts).","commonSituations":"A client library pre-tokenizes prompts (caching) while the operator enabled batch encode for throughput; mixing cached-token requests with a batch-encoded deployment.","solutions":["Drop input_ids and send raw text so batch tokenization applies, or","Disable enable_tokenizer_batch_encode on the server to allow pre-tokenized input","Keep pre-tokenized traffic on a separate server without the flag"],"exampleFix":"# before\nGenerateReqInput(input_ids=cached_ids, sampling_params=sp)  # server has --enable-tokenizer-batch-encode\n# after\nGenerateReqInput(text=prompts, sampling_params=sp)","handlingStrategy":"validation","validationCode":"if any(getattr(r, 'input_ids', None) is not None for r in batch_requests):\n    assert not server_args.enable_tokenizer_batch_encode, 'pre-tokenized input incompatible with batch encode'","typeGuard":null,"tryCatchPattern":"except ValueError as e: if 'pre-tokenized' in str(e): send text instead of input_ids or disable the flag","preventionTips":["Choose one tokenization side: client (input_ids) or server (text + batch encode), not both","Document the flag decision per deployment"],"tags":["sglang","config","pretokenized","batch-encode"],"backgroundTag":"incompatible-server-config-option","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}