{"record":{"id":"2bc2960f121818a9","repo":"sgl-project/sglang","slug":"processor-image-placeholder-count-mismatch-proces","errorCode":null,"errorMessage":"processor image placeholder count mismatch: processor={processor_placeholder_count}, resolved={sum(counts)}","messagePattern":"processor image placeholder count mismatch: processor=(.+?), resolved=(.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/multimodal/processors/base_processor.py","lineNumber":1727,"sourceCode":"                and raw_images\n                and not raw_audios\n                and not raw_videos\n            ):\n                assert isinstance(\n                    base_output.input_ids, list\n                ), f\"expected list[int] input_ids, got {type(base_output.input_ids)}\"\n                try:\n                    counts = self.resolve_image_token_counts(raw_images)\n                    image_placeholder_token_id = mm_tokens.image_token_id\n                    if image_placeholder_token_id is None:\n                        raise ValueError(\n                            \"image placeholder token id is not set for this processor\"\n                        )\n                    processor_placeholder_count = int(\n                        (input_ids == image_placeholder_token_id).sum().item()\n                    )\n                    if processor_placeholder_count != sum(counts):\n                        raise ValueError(\n                            \"processor image placeholder count mismatch: \"\n                            f\"processor={processor_placeholder_count}, \"\n                            f\"resolved={sum(counts)}\"\n                        )\n                    input_ids = torch.tensor(\n                        self._expand_input_ids(\n                            base_output.input_ids,\n                            counts,\n                            image_placeholder_token_id,\n                        ),\n                        dtype=input_ids.dtype,\n                    )\n                except Exception as e:\n                    logger.warning(\n                        f\"Due to {e}, falling back to decode+retokenize, which may change prompt length (token drift).\"\n                    )\n        else:\n            ret = None","sourceCodeStart":1709,"sourceCodeEnd":1745,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/multimodal/processors/base_processor.py#L1709-L1745","documentation":"After running an overridden processor, process_and_combine_mm_data recounts image placeholder tokens in the returned input_ids and requires the total to equal sum(counts) from resolved per-image expansions. A mismatch means the override emitted a different number of placeholder tokens than the resolved media implies, which would corrupt token offsets.","triggerScenarios":"Using processor_override whose tokenization expands/collapses image placeholder tokens differently (e.g. emits 3 tokens per image while counts assume 1), or an override that drops placeholder tokens for some images.","commonSituations":"Custom HF processor wrappers that retokenize or normalize text; models whose placeholder token maps to multi-token sequences; mismatch between the override's config (e.g. num_image_tokens) and the resolver's counts.","solutions":["Make the override preserve exactly one placeholder token per image (or align its expansion with counts)","Recompute counts with the same rules the override uses to emit placeholders","If the override cannot comply, disable the retokenize-avoidance fast path for that processor"],"exampleFix":"// before\noverride = MyProcessor(...)  # retokenizes, changes placeholder count\n// after\nclass MyProcessor(...):\n    def __call__(self, text, images, **kw):\n        out = super().__call__(text, images, **kw)\n        # ensure placeholder token count == len(images); fix text if not\n        return out","handlingStrategy":"validation","validationCode":"ph = int((input_ids == image_placeholder_token_id).sum().item())\nassert ph == sum(counts), f'override emitted {ph} placeholders, expected {sum(counts)}'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Custom processor overrides must emit exactly one placeholder token per image","Add a unit test for the override asserting placeholder count == len(images)"],"tags":["multimodal","processor-override","tokenization","placeholder-mismatch"],"backgroundTag":"placeholder-count-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}