sgl-project/sglang · error · ValueError

For multimodal input processing do not set `enable_tokenizer

Error message

For multimodal input processing do not set `enable_tokenizer_batch_encode`.

What it means

Raised by _validate_batch_tokenization_constraints when enable_tokenizer_batch_encode is on and a generation request in the batch contains multimodal input. Batch tokenization only handles plain text; multimodal inputs need per-request processing paths.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager.py:1518

            self._validate_one_request(obj[i], input_ids_list[i])
            token_type_ids = (
                token_type_ids_list[i] if token_type_ids_list is not None else None
            )
            tokenized_objs.append(
                self._create_tokenized_object(
                    req, req.text, input_ids_list[i], None, None, token_type_ids
                )
            )
        logger.debug(f"Completed batch processing for {batch_size} requests")
        return tokenized_objs

    def _validate_batch_tokenization_constraints(
        self, batch_size: int, obj: Union[GenerateReqInput, EmbeddingReqInput]
    ) -> None:
        """Validate constraints for batch tokenization processing."""
        for i in range(batch_size):
            if self.is_generation and obj[i].contains_mm_input():
                raise ValueError(
                    "For multimodal input processing do not set `enable_tokenizer_batch_encode`."
                )
            if obj[i].input_ids is not None:
                raise ValueError(
                    "Batch tokenization is not needed for pre-tokenized input_ids. Do not set `enable_tokenizer_batch_encode`."
                )
            if obj[i].input_embeds is not None:
                raise ValueError(
                    "Batch tokenization is not needed for input_embeds. Do not set `enable_tokenizer_batch_encode`."
                )

    def _batch_has_text(
        self, batch_size: int, obj: Union[GenerateReqInput, EmbeddingReqInput]
    ) -> bool:
        """Check if any request in the batch contains text input."""
        for i in range(batch_size):
            if obj[i].text:
                return True

View on GitHub (pinned to 0132848349)

Solutions

1. Disable enable_tokenizer_batch_encode (remove the server flag / set it False) when serving multimodal models
2. Route multimodal requests to a deployment without the flag
3. Split text-only and multimodal traffic into separate server instances

Example fix

# before
python -m sglang.launch_server --model ... --enable-tokenizer-batch-encode  # then sending images
# after
python -m sglang.launch_server --model ...  # flag removed for VLM serving

if any(req.contains_mm_input() for req in batch_requests):
    assert not server_args.enable_tokenizer_batch_encode, 'disable batch encode for mm input'

except ValueError as e: if 'enable_tokenizer_batch_encode' in str(e): relaunch server without the flag and retry

When it happens

Trigger: Server started with --enable-tokenizer-batch-encode and a batch GenerateReqInput where any element has image/audio/video mm inputs (contains_mm_input() true).

Common situations: Turning on the perf flag globally then sending vision-language requests (e.g. images to a VLM endpoint) through the batch API.

Related errors

• {selection_error}{component_suffix}
• Batch tokenization is not needed for pre-tokenized input_ids
• Batch tokenization is not needed for input_embeds. Do not se
• No compatible attention backend is available{component_suffi
• f"Unsupported patch_size type: {type(patch_size)}"

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/aea0fb4d0f557ea0. Report an issue: GitHub.