sgl-project/sglang · error · ValueError

prompt has image placeholder token(s) but image(s) were…

Error message

prompt has {num_placeholders} image placeholder token(s) but {len(counts)} image(s) were provided

What it means

_expand_input_ids verifies that the number of placeholder token ids in the already-tokenized prompt exactly equals the number of expansion 'counts' (per-image expanded token counts). A mismatch means the prompt text and the media list disagree, so retokenize-avoidance expansion cannot proceed safely.

Solutions

1. Regenerate the prompt with the model's chat template so placeholders match len(images) exactly
2. Ensure the tokenized ids correspond to the same prompt/media pairing used to compute counts
3. Avoid manually inserting or deleting placeholder tokens in the token ids

Example fix

// before
input_ids = tokenizer('<image> <image>')  # 2 placeholders
counts = [c for c in per_img]  # 3 images
// after
input_ids = tokenizer('<image>' * len(images))
counts = [c for c in per_img]  # len == len(images) == placeholders

num_ph = sum(1 for t in input_ids if t == placeholder_token_id)
assert num_ph == len(counts), f'{num_ph} placeholders vs {len(counts)} images'

When it happens

Trigger: Calling process_and_combine_mm_data where the tokenized prompt contains N placeholder tokens but the resolved per-image counts list has a different length — e.g. prompt with 2 <image> tags but 3 images provided, or custom text embedding extra placeholder tokens.

Common situations: Chat templates that add/remove placeholder tokens conditionally; offline tokenized prompts reused with different image counts; duplicated placeholder tokens from string interpolation.

Related errors

• InklingMultimodalProcessor
• processor image placeholder count mismatch: processor=
• An exception occurred while loading
• An exception occurred while loading multimodal data
• Audio placeholder count does not match audio_data

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/3d15c37e92e962a3. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/multimodal/processors/base_processor.py:1631

        counts: List[int],
        placeholder_token_id: Optional[int],
    ) -> List[int]:
        """Rebuild final input_ids for a pre-tokenized (list[int]) prompt.

        Keep the user's ORIGINAL tokens verbatim and expand the i-th image
        placeholder into ``counts[i]`` copies of ``placeholder_token_id``. The HF
        processor's re-tokenization is discarded, so non-media tokens cannot
        drift.

        """
        if placeholder_token_id is None:
            raise ValueError("placeholder_token_id is not set for this processor")

        num_placeholders = sum(
            1 for token_id in original_ids if token_id == placeholder_token_id
        )
        if num_placeholders != len(counts):
            raise ValueError(
                f"prompt has {num_placeholders} image placeholder token(s) but "
                f"{len(counts)} image(s) were provided"
            )

        rebuilt: List[int] = []
        next_image_idx = 0
        for token_id in original_ids:
            if token_id == placeholder_token_id:
                rebuilt.extend([placeholder_token_id] * counts[next_image_idx])
                next_image_idx += 1
            else:
                rebuilt.append(token_id)
        return rebuilt

    def process_and_combine_mm_data(
        self,
        base_output: BaseMultiModalProcessorOutput,
        mm_tokens: MultimodalSpecialTokens,

View on GitHub (pinned to 0132848349)