sgl-project/sglang · error · RuntimeError

An exception occurred while loading multimodal data: {e}

Error message

An exception occurred while loading multimodal data: {e}

What it means

In the legacy template-based loader, a StopIteration while advancing the media iterator is expected only when a precomputed input is supplied with multi-image prompts; otherwise it means the prompt contained more media placeholders than available media items, re-raised as RuntimeError.

Source

Thrown at python/sglang/srt/multimodal/processors/base_processor.py:1386

                        new_text_parts += mm_tokens
                    elif modality == Modality.AUDIO:
                        # audio
                        mm_tokens = (
                            text_part
                            if is_precomputed
                            else multimodal_tokens.audio_token
                        )
                        new_text_parts += mm_tokens
                else:
                    # normal text
                    new_text_parts += [text_part]

            except StopIteration as e:
                # when precomputed_input is presented with multi-images, StopIteration is expected
                if has_precomputed_input:
                    new_text_parts += [text_part]
                    continue
                raise RuntimeError(
                    f"An exception occurred while loading multimodal data: {e}"
                )
            except ValueError as e:
                raise ValueError(
                    f"An exception occurred while loading multimodal data: {e}"
                ) from e
            except Exception as e:
                raise RuntimeError(
                    f"An exception occurred while loading multimodal data: {e}"
                )
        return BaseMultiModalProcessorOutput(
            images=images,
            audios=audios,
            videos=videos,
            input_text="".join(new_text_parts),
            input_ids=input_ids,
        )

View on GitHub (pinned to 0132848349)

Solutions

1. Make placeholder count in the prompt equal the number of media items
2. Let the model's chat template generate the prompt instead of hand-writing placeholders
3. If using precomputed embeddings with multi-image, ensure the precomputed input path is correctly detected (has_precomputed_input)

Example fix

// before
prompt = '<image> and <image>'  # 2 placeholders, 1 image
// after
prompt = '<image>'  # matches len(images)

n_placeholders = prompt.count('<image>') + prompt.count('<audio>') + prompt.count('<video>')
n_media = sum(len(v) for v in (images or []) + (audios or []) + (videos or []))
assert n_placeholders == n_media or has_precomputed, (n_placeholders, n_media)

When it happens

Trigger: Using legacy_load_mm_data where the prompt's <image>/<audio> placeholder count exceeds the number of supplied media items, and no precomputed input path is active.

Common situations: Hand-written prompt templates with extra placeholder tags; chat templates that inject a placeholder per turn while the user attached fewer files; multi-image prompts built by string concatenation.

Related errors

• The number of placeholders does not match the number of repl
• prompt has {num_placeholders} image placeholder token(s) but
• processor image placeholder count mismatch: processor={proce
• Image placeholder count does not match image_data
• Audio placeholder count does not match audio_data

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/2d11ce6cf5c8e523. Report an issue: GitHub.