sgl-project/sglang · error · ValueError

[internvl][internlm2] image_data provided but no images pars

Error message

[internvl][internlm2] image_data provided but no images parsed from prompt placeholders

What it means

Raised by SGLang's InternVL multimodal processor when the request carries image_data but no InternLM2-style image placeholders were found and parsed from the prompt text, leaving pixel_values_list empty. The processor locates images by scanning the prompt for placeholder tokens, so image data alone is not enough. It aborts rather than silently dropping the images.

Source

Thrown at python/sglang/srt/multimodal/processors/internvl.py:677

        for image in base_output.images:
            if isinstance(image, Image.Image):
                img_np = np.array(image.convert("RGB"))
                tensor = (
                    torch.from_numpy(img_np).permute(2, 0, 1).to(get_device()).float()
                    / 255.0
                )
            else:
                tensor = image.to(get_device())

            tensor = (tensor - mean) / std
            tiles = self.dynamic_preprocess(
                tensor, image_size=448, max_num=12, use_thumbnail=True
            )
            pixel_values_list.append(tiles)
            num_patches_list.append(int(tiles.shape[0]))

        if image_data and not pixel_values_list:
            raise ValueError(
                "[internvl][internlm2] image_data provided but no images parsed from prompt placeholders"
            )

        pixel_values = (
            torch.cat(pixel_values_list, dim=0) if pixel_values_list else None
        )

        # Expand each <IMG_CONTEXT> into <img> + <IMG_CONTEXT>*N + </img>
        ph = "<<<__IMG_CONTEXT_PLACEHOLDER__>>>"
        input_text_base = (base_output.input_text or prompt).replace(
            self.IMG_CONTEXT, ph
        )

        input_text_updated = input_text_base
        for num_patches in num_patches_list:
            image_tokens = (
                self.IMG_START
                + (self.IMG_CONTEXT * (self.num_image_token * int(num_patches)))

View on GitHub (pinned to 0132848349)

Solutions

Ensure the prompt is produced by the correct chat template so image placeholder tokens are inserted for each image
Verify the number of placeholder tokens in the prompt matches len(image_data)
If building prompts manually, insert the model's image placeholder token once per image before calling process_mm_data_async
If images are optional, do not pass image_data when the prompt has no placeholders

Example fix

// before
resp = await client.generate(prompt="describe this", image_data=[img])

// after
placeholder = "<image>"  # model's image placeholder token
resp = await client.generate(prompt=f"{placeholder}describe this", image_data=[img])

Defensive patterns

Strategy: validation

Validate before calling

placeholder = "<image>"  # model-specific token
n_img = len(image_data or [])
assert input_text.count(placeholder) == n_img, (
    f"prompt has {input_text.count(placeholder)} placeholders for {n_img} images"
)

Prevention

Always build prompts through the model's chat template rather than string concatenation
Assert placeholder count equals image count before each request
Log the final prompt when debugging multimodal requests

When it happens

Trigger: Calling process_mm_data_async (directly or via the server /generate API) with a non-empty image_data list while the prompt string contains no InternLM2 image placeholder tokens, or placeholders in a format the scanner does not recognize.

Common situations: Chat template not inserting image placeholder tokens (template/config mismatch), sending images with a hand-built prompt that omits placeholders, or a model/config combination where the internlm2 path expects a different placeholder string than the one emitted.

Related errors

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/924797e63d795dfd. Report an issue: GitHub.