sgl-project/sglang · error · RuntimeError

An exception occurred while loading multimodal data

Error message

An exception occurred while loading multimodal data: {e}

What it means

In the legacy template-based loader, a StopIteration while advancing the media iterator is expected only when a precomputed input is supplied with multi-image prompts; otherwise it means the prompt contained more media placeholders than available media items, re-raised as RuntimeError.

Solutions

1. Make placeholder count in the prompt equal the number of media items
2. Let the model's chat template generate the prompt instead of hand-writing placeholders
3. If using precomputed embeddings with multi-image, ensure the precomputed input path is correctly detected (has_precomputed_input)

Example fix

// before
prompt = '<image> and <image>'  # 2 placeholders, 1 image
// after
prompt = '<image>'  # matches len(images)

n_placeholders = prompt.count('<image>') + prompt.count('<audio>') + prompt.count('<video>')
n_media = sum(len(v) for v in (images or []) + (audios or []) + (videos or []))
assert n_placeholders == n_media or has_precomputed, (n_placeholders, n_media)

When it happens

Trigger: Using legacy_load_mm_data where the prompt's <image>/<audio> placeholder count exceeds the number of supplied media items, and no precomputed input path is active.

Common situations: Hand-written prompt templates with extra placeholder tags; chat templates that inject a placeholder per turn while the user attached fewer files; multi-image prompts built by string concatenation.

Related errors

• The number of placeholders does not match the number of…
• Audio placeholder count does not match audio_data
• Expected image placeholder token(s), found .
• Image placeholder count does not match image_data
• InklingMultimodalProcessor

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2d11ce6cf5c8e523. Report an issue: GitHub.

Appendix: source

Thrown at python/sglang/srt/multimodal/processors/base_processor.py:1386

                        new_text_parts += mm_tokens
                    elif modality == Modality.AUDIO:
                        # audio
                        mm_tokens = (
                            text_part
                            if is_precomputed
                            else multimodal_tokens.audio_token
                        )
                        new_text_parts += mm_tokens
                else:
                    # normal text
                    new_text_parts += [text_part]

            except StopIteration as e:
                # when precomputed_input is presented with multi-images, StopIteration is expected
                if has_precomputed_input:
                    new_text_parts += [text_part]
                    continue
                raise RuntimeError(
                    f"An exception occurred while loading multimodal data: {e}"
                )
            except ValueError as e:
                raise ValueError(
                    f"An exception occurred while loading multimodal data: {e}"
                ) from e
            except Exception as e:
                raise RuntimeError(
                    f"An exception occurred while loading multimodal data: {e}"
                )
        return BaseMultiModalProcessorOutput(
            images=images,
            audios=audios,
            videos=videos,
            input_text="".join(new_text_parts),
            input_ids=input_ids,
        )

View on GitHub (pinned to 0132848349)