sgl-project/sglang · error · ValueError

Batch tokenization is not needed for pre-tokenized input_ids

Error message

Batch tokenization is not needed for pre-tokenized input_ids. Do not set `enable_tokenizer_batch_encode`.

What it means

Raised when enable_tokenizer_batch_encode is set but a request in the batch already supplies input_ids. Batch tokenization exists to accelerate text tokenization; pre-tokenized input makes it unnecessary and the combination is rejected.

Source

Thrown at python/sglang/srt/managers/tokenizer_manager.py:1522

            tokenized_objs.append(
                self._create_tokenized_object(
                    req, req.text, input_ids_list[i], None, None, token_type_ids
                )
            )
        logger.debug(f"Completed batch processing for {batch_size} requests")
        return tokenized_objs

    def _validate_batch_tokenization_constraints(
        self, batch_size: int, obj: Union[GenerateReqInput, EmbeddingReqInput]
    ) -> None:
        """Validate constraints for batch tokenization processing."""
        for i in range(batch_size):
            if self.is_generation and obj[i].contains_mm_input():
                raise ValueError(
                    "For multimodal input processing do not set `enable_tokenizer_batch_encode`."
                )
            if obj[i].input_ids is not None:
                raise ValueError(
                    "Batch tokenization is not needed for pre-tokenized input_ids. Do not set `enable_tokenizer_batch_encode`."
                )
            if obj[i].input_embeds is not None:
                raise ValueError(
                    "Batch tokenization is not needed for input_embeds. Do not set `enable_tokenizer_batch_encode`."
                )

    def _batch_has_text(
        self, batch_size: int, obj: Union[GenerateReqInput, EmbeddingReqInput]
    ) -> bool:
        """Check if any request in the batch contains text input."""
        for i in range(batch_size):
            if obj[i].text:
                return True
            elif self.is_generation and obj[i].contains_mm_input():
                return True

        return False

View on GitHub (pinned to 0132848349)

Solutions

Drop input_ids and send raw text so batch tokenization applies, or
Disable enable_tokenizer_batch_encode on the server to allow pre-tokenized input
Keep pre-tokenized traffic on a separate server without the flag

Example fix

# before
GenerateReqInput(input_ids=cached_ids, sampling_params=sp)  # server has --enable-tokenizer-batch-encode
# after
GenerateReqInput(text=prompts, sampling_params=sp)

Defensive patterns

Strategy: validation

Validate before calling

if any(getattr(r, 'input_ids', None) is not None for r in batch_requests):
    assert not server_args.enable_tokenizer_batch_encode, 'pre-tokenized input incompatible with batch encode'

Try / catch

except ValueError as e: if 'pre-tokenized' in str(e): send text instead of input_ids or disable the flag

Prevention

Choose one tokenization side: client (input_ids) or server (text + batch encode), not both
Document the flag decision per deployment

When it happens

Trigger: Server has enable_tokenizer_batch_encode=True and a batch request element has input_ids != None (client-side tokenized prompts).

Common situations: A client library pre-tokenizes prompts (caching) while the operator enabled batch encode for throughput; mixing cached-token requests with a batch-encoded deployment.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/80ef098b4e96e8e9. Report an issue: GitHub.