{"record":{"id":"aea0fb4d0f557ea0","repo":"sgl-project/sglang","slug":"for-multimodal-input-processing-do-not-set-enable","errorCode":null,"errorMessage":"For multimodal input processing do not set `enable_tokenizer_batch_encode`.","messagePattern":"For multimodal input processing do not set `enable_tokenizer_batch_encode`\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/managers/tokenizer_manager.py","lineNumber":1518,"sourceCode":"            self._validate_one_request(obj[i], input_ids_list[i])\n            token_type_ids = (\n                token_type_ids_list[i] if token_type_ids_list is not None else None\n            )\n            tokenized_objs.append(\n                self._create_tokenized_object(\n                    req, req.text, input_ids_list[i], None, None, token_type_ids\n                )\n            )\n        logger.debug(f\"Completed batch processing for {batch_size} requests\")\n        return tokenized_objs\n\n    def _validate_batch_tokenization_constraints(\n        self, batch_size: int, obj: Union[GenerateReqInput, EmbeddingReqInput]\n    ) -> None:\n        \"\"\"Validate constraints for batch tokenization processing.\"\"\"\n        for i in range(batch_size):\n            if self.is_generation and obj[i].contains_mm_input():\n                raise ValueError(\n                    \"For multimodal input processing do not set `enable_tokenizer_batch_encode`.\"\n                )\n            if obj[i].input_ids is not None:\n                raise ValueError(\n                    \"Batch tokenization is not needed for pre-tokenized input_ids. Do not set `enable_tokenizer_batch_encode`.\"\n                )\n            if obj[i].input_embeds is not None:\n                raise ValueError(\n                    \"Batch tokenization is not needed for input_embeds. Do not set `enable_tokenizer_batch_encode`.\"\n                )\n\n    def _batch_has_text(\n        self, batch_size: int, obj: Union[GenerateReqInput, EmbeddingReqInput]\n    ) -> bool:\n        \"\"\"Check if any request in the batch contains text input.\"\"\"\n        for i in range(batch_size):\n            if obj[i].text:\n                return True","sourceCodeStart":1500,"sourceCodeEnd":1536,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/managers/tokenizer_manager.py#L1500-L1536","documentation":"Raised by _validate_batch_tokenization_constraints when enable_tokenizer_batch_encode is on and a generation request in the batch contains multimodal input. Batch tokenization only handles plain text; multimodal inputs need per-request processing paths.","triggerScenarios":"Server started with --enable-tokenizer-batch-encode and a batch GenerateReqInput where any element has image/audio/video mm inputs (contains_mm_input() true).","commonSituations":"Turning on the perf flag globally then sending vision-language requests (e.g. images to a VLM endpoint) through the batch API.","solutions":["Disable enable_tokenizer_batch_encode (remove the server flag / set it False) when serving multimodal models","Route multimodal requests to a deployment without the flag","Split text-only and multimodal traffic into separate server instances"],"exampleFix":"# before\npython -m sglang.launch_server --model ... --enable-tokenizer-batch-encode  # then sending images\n# after\npython -m sglang.launch_server --model ...  # flag removed for VLM serving","handlingStrategy":"validation","validationCode":"if any(req.contains_mm_input() for req in batch_requests):\n    assert not server_args.enable_tokenizer_batch_encode, 'disable batch encode for mm input'","typeGuard":null,"tryCatchPattern":"except ValueError as e: if 'enable_tokenizer_batch_encode' in str(e): relaunch server without the flag and retry","preventionTips":["Don't enable batch encode on VLM/multimodal deployments","Segment text-only vs multimodal traffic"],"tags":["sglang","config","multimodal","batch-encode"],"backgroundTag":"incompatible-server-config-option","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}