sgl-project/sglang · error · ValueError

InklingMultimodalProcessor: {n_img_ph} image placeholder tok

Error message

InklingMultimodalProcessor: {n_img_ph} image placeholder token(s) in input_ids but {len(image_data)} image(s) provided; counts must match.

What it means

During Inkling request assembly, the number of image placeholder tokens found in input_ids must equal len(image_data). A mismatch means the chat template rendered a different number of <image> placeholders than the images supplied — typically caused by sending input_ids (pre-tokenized) that were rendered with the wrong template or wrong placeholder token id.

Source

Thrown at python/sglang/srt/multimodal/processors/inkling.py:210

        ``image_data`` / ``audio_data`` in encounter order.
        """
        image_data = image_data or []
        audio_data = audio_data or []

        # One placeholder per media item (expanded below); a count mismatch (incl. a
        # None token id absent from config) must fail loudly, not drop media silently.
        n_img_ph = (
            sum(1 for t in input_ids if t == self.IMAGE_TOKEN_ID)
            if self.IMAGE_TOKEN_ID is not None
            else 0
        )
        n_aud_ph = (
            sum(1 for t in input_ids if t == self.AUDIO_TOKEN_ID)
            if self.AUDIO_TOKEN_ID is not None
            else 0
        )
        if n_img_ph != len(image_data):
            raise ValueError(
                f"InklingMultimodalProcessor: {n_img_ph} image placeholder token(s) in "
                f"input_ids but {len(image_data)} image(s) provided; counts must match."
            )
        if n_aud_ph != len(audio_data):
            raise ValueError(
                f"InklingMultimodalProcessor: {n_aud_ph} audio placeholder token(s) in "
                f"input_ids but {len(audio_data)} audio(s) provided; counts must match."
            )

        img_feat = (
            self.inkling_processor.process_images(image_data) if image_data else None
        )
        aud_feat = (
            self.inkling_processor.process_audios(audio_data) if audio_data else None
        )

        # Rust processor returns content_hashes; original processor does not.
        img_hashes = img_feat.get("content_hashes") if img_feat else None

View on GitHub (pinned to 0132848349)

Solutions

Regenerate input_ids with the Inkling chat template so exactly one image placeholder per provided image is emitted
Verify the placeholder token id used when rendering matches processor.IMAGE_TOKEN_ID
Assert counts before submit: len([t for t in input_ids if t==IMAGE_TOKEN_ID]) == len(image_data)

Example fix

# before
req.input_ids = base_tokenizer(text)['input_ids']  # no image placeholder
# after
req.input_ids = inkling_processor.apply_chat_template(msgs_with_image_tokens)['input_ids']  # 1 placeholder per image

Defensive patterns

Strategy: validation

Validate before calling

n_ph = sum(1 for t in input_ids if t == processor.IMAGE_TOKEN_ID)
assert n_ph == len(image_data or []), f'{n_ph} placeholders vs {len(image_data)} images'

Type guard

def placeholders_match(input_ids: list[int], token_id: int, data: list) -> bool:
    return sum(1 for t in input_ids if t == token_id) == len(data or [])

Try / catch

try:
    out = await processor.process_mm_data_async(None, im, request_obj)
except ValueError as e:
    if 'placeholder' in str(e):
        request_obj.input_ids = re-render_with_template(msgs, n_images=len(im))
        out = await processor.process_mm_data_async(None, im, request_obj)
    else: raise

Prevention

Always render input_ids through the model's chat template
Unit-test template rendering with N images before batch runs

When it happens

Trigger: Passing request_obj.input_ids tokenized with a template that emits 0 or 2 image tokens while image_data has 1 image; using a placeholder token id that differs from InklingProcessor's IMAGE_TOKEN_ID so the counting pass finds none; image list containing None entries after a filter step.

Common situations: Offline batch pipelines pre-tokenizing prompts with the base tokenizer instead of the processor's chat template; template version drift after model upgrade; duplicated or dropped images during data loading.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/0488affc02212deb. Report an issue: GitHub.