sgl-project/sglang · error · ValueError

processor image placeholder count mismatch: processor={proce

Error message

processor image placeholder count mismatch: processor={processor_placeholder_count}, resolved={sum(counts)}

What it means

After running an overridden processor, process_and_combine_mm_data recounts image placeholder tokens in the returned input_ids and requires the total to equal sum(counts) from resolved per-image expansions. A mismatch means the override emitted a different number of placeholder tokens than the resolved media implies, which would corrupt token offsets.

Source

Thrown at python/sglang/srt/multimodal/processors/base_processor.py:1727

                and raw_images
                and not raw_audios
                and not raw_videos
            ):
                assert isinstance(
                    base_output.input_ids, list
                ), f"expected list[int] input_ids, got {type(base_output.input_ids)}"
                try:
                    counts = self.resolve_image_token_counts(raw_images)
                    image_placeholder_token_id = mm_tokens.image_token_id
                    if image_placeholder_token_id is None:
                        raise ValueError(
                            "image placeholder token id is not set for this processor"
                        )
                    processor_placeholder_count = int(
                        (input_ids == image_placeholder_token_id).sum().item()
                    )
                    if processor_placeholder_count != sum(counts):
                        raise ValueError(
                            "processor image placeholder count mismatch: "
                            f"processor={processor_placeholder_count}, "
                            f"resolved={sum(counts)}"
                        )
                    input_ids = torch.tensor(
                        self._expand_input_ids(
                            base_output.input_ids,
                            counts,
                            image_placeholder_token_id,
                        ),
                        dtype=input_ids.dtype,
                    )
                except Exception as e:
                    logger.warning(
                        f"Due to {e}, falling back to decode+retokenize, which may change prompt length (token drift)."
                    )
        else:
            ret = None

View on GitHub (pinned to 0132848349)

Solutions

1. Make the override preserve exactly one placeholder token per image (or align its expansion with counts)
2. Recompute counts with the same rules the override uses to emit placeholders
3. If the override cannot comply, disable the retokenize-avoidance fast path for that processor

Example fix

// before
override = MyProcessor(...)  # retokenizes, changes placeholder count
// after
class MyProcessor(...):
    def __call__(self, text, images, **kw):
        out = super().__call__(text, images, **kw)
        # ensure placeholder token count == len(images); fix text if not
        return out

ph = int((input_ids == image_placeholder_token_id).sum().item())
assert ph == sum(counts), f'override emitted {ph} placeholders, expected {sum(counts)}'

When it happens

Trigger: Using processor_override whose tokenization expands/collapses image placeholder tokens differently (e.g. emits 3 tokens per image while counts assume 1), or an override that drops placeholder tokens for some images.

Common situations: Custom HF processor wrappers that retokenize or normalize text; models whose placeholder token maps to multi-token sequences; mismatch between the override's config (e.g. num_image_tokens) and the resolver's counts.

Related errors

• prompt has {num_placeholders} image placeholder token(s) but
• An exception occurred while loading multimodal data: {e}
• Image placeholder count does not match image_data
• Audio placeholder count does not match audio_data
• Video placeholder count does not match video_data: {len(vide

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2bc2960f121818a9. Report an issue: GitHub.