sgl-project/sglang · error · ValueError

prompt has {num_text_tokens} tokens, exceeds max_text_tokens

Error message

prompt has {num_text_tokens} tokens, exceeds max_text_tokens={max_text_tokens}

What it means

Raised by the Ideogram stage's _tokenize when the tokenized prompt exceeds the pipeline config's max_text_tokens cap. The stage tokenizes text with HF tokenizers and enforces the model's context budget before building inputs. It protects the downstream transformer from overflowing its text token positions.

Source

Thrown at python/sglang/multimodal_gen/runtime/pipelines_core/stages/model_specific_stages/ideogram.py:136

class Ideogram4TextEncodingStage(TextEncodingStage):
    deduplicated_extra_tensor_tree_output_keys = ("ideogram4",)

    def __init__(self, text_encoder, tokenizer) -> None:
        super().__init__([text_encoder], [tokenizer])

    def _tokenize(self, prompt: str, max_text_tokens: int):
        messages = [{"role": "user", "content": [{"type": "text", "text": prompt}]}]
        text = self.tokenizers[0].apply_chat_template(
            messages, add_generation_prompt=True, tokenize=False
        )
        encoded = self.tokenizers[0](
            text, return_tensors="pt", add_special_tokens=False
        )
        token_ids = encoded["input_ids"][0]
        num_text_tokens = int(token_ids.shape[0])
        if num_text_tokens > max_text_tokens:
            raise ValueError(
                f"prompt has {num_text_tokens} tokens, exceeds max_text_tokens={max_text_tokens}"
            )
        return token_ids, num_text_tokens

    def _build_inputs(self, prompts: list[str], height: int, width: int, server_args):
        cfg = server_args.pipeline_config
        tokenized = [self._tokenize(p, cfg.max_text_tokens) for p in prompts]
        batch_size = len(prompts)
        patch = cfg.patch_size * cfg.ae_scale_factor
        if height < 256 or height > 2048 or width < 256 or width > 2048:
            raise ValueError("height/width must be between 256 and 2048")
        if height % patch != 0 or width % patch != 0:
            raise ValueError(
                f"height/width must be divisible by patch_size*ae_scale_factor={patch}"
            )
        grid_h = height // patch
        grid_w = width // patch
        num_image_tokens = grid_h * grid_w

View on GitHub (pinned to 0132848349)

Solutions

1. Shorten the prompt below max_text_tokens
2. Raise cfg.max_text_tokens in pipeline_config if the model supports a larger text budget
3. Pre-tokenize and truncate/summarize the prompt before submitting

Example fix

// before
resp = pipeline.generate(prompt=very_long_prompt)
// after
ids = tokenizer(very_long_prompt, add_special_tokens=False)['input_ids']
if len(ids) > cfg.max_text_tokens:
    prompt = tokenizer.decode(ids[:cfg.max_text_tokens])
resp = pipeline.generate(prompt=prompt)

ids = tokenizer(prompt, add_special_tokens=False)['input_ids']
if len(ids) > cfg.max_text_tokens:
    prompt = tokenizer.decode(ids[:cfg.max_text_tokens])

When it happens

Trigger: Calling the Ideogram generate/forward path with a prompt whose token count (add_special_tokens=False) exceeds server_args.pipeline_config.max_text_tokens; longer prompts or batch entries with verbose descriptions trigger it.

Common situations: Passing very long descriptive prompts, concatenated style tags, or programmatically generated prompt strings; lowering max_text_tokens in pipeline_config; switching to a tokenizer with a larger vocabulary producing more tokens per word.

Related errors

• height/width must be between 256 and 2048
• height/width must be divisible by patch_size*ae_scale_factor
• Unknown Ideogram 4 preset {preset!r}; expected one of {sorte
• v_cache must be provided
• q can only be None when only_qv=True

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/fae65452592b0802. Report an issue: GitHub.