sgl-project/sglang · error · ValueError

Multi-output conditioning requires prompt text so the prompt

Error message

Multi-output conditioning requires prompt text so the prompt batch size is unambiguous.

What it means

Thrown by ConditionExpansion.from_batch when num_outputs > 1 but batch.prompt is None — with multiple outputs per prompt, the runtime needs prompt text so it knows how many prompts are in the batch and can expand each per-prompt conditioning num_outputs times.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/condition_expansion.py:25

@dataclass(frozen=True)
class PromptToSampleBatchExpander:
    """Expand selected conditioning from prompt order to sample order."""

    prompt_batch_size: int
    sample_batch_size: int

    @classmethod
    def from_batch(cls, batch):
        num_outputs = int(batch.num_outputs_per_prompt or 1)
        if num_outputs <= 1:
            return None
        if isinstance(batch.prompt, list):
            prompt_batch_size = len(batch.prompt)
        elif batch.prompt is not None:
            prompt_batch_size = 1
        else:
            raise ValueError(
                "Multi-output conditioning requires prompt text so the prompt "
                "batch size is unambiguous."
            )
        if prompt_batch_size <= 0:
            raise ValueError("Multi-output conditioning requires at least one prompt.")
        return cls(prompt_batch_size, prompt_batch_size * num_outputs)

    def _expand_tensor(self, value: torch.Tensor, name: str) -> torch.Tensor:
        current_batch_size = value.shape[0]
        if current_batch_size == self.sample_batch_size:
            return value
        if current_batch_size != self.prompt_batch_size:
            raise ValueError(
                f"{name} has batch dim {current_batch_size} (shape "
                f"{tuple(value.shape)}); expected {self.prompt_batch_size} "
                f"(per-prompt) or {self.sample_batch_size} (per-sample)."
            )
        repeats = self.sample_batch_size // self.prompt_batch_size

View on GitHub (pinned to 0132848349)

Solutions

1. Populate batch.prompt (list of strings, or a single string) before requesting multiple outputs per prompt
2. If prompts are genuinely unavailable, set num_outputs/n back to 1 so expansion is trivial
3. Ensure the tokenizer/front-end attaches the original text to the batch object

Example fix

# before
batch = SampleBatch(input_ids=ids, prompt=None)  # n=4
# after
batch = SampleBatch(input_ids=ids, prompt=["describe this image"])  # n=4

if num_outputs > 1:
    assert batch.prompt is not None and (not isinstance(batch.prompt, list) or len(batch.prompt) > 0), "multi-output needs prompt text"

def supports_multi_output(batch) -> bool:
    return batch.prompt is not None and (not isinstance(batch.prompt, list) or len(batch.prompt) > 0)

try:
    expansion = expand_conditioning_to_sample_batch(batch, num_outputs)
except ValueError as e:
    if "requires prompt text" in str(e):
        batch.prompt = tokenizer.decode(batch.input_ids[0])
        expansion = expand_conditioning_to_sample_batch(batch, num_outputs)
    else:
        raise

When it happens

Trigger: Calling expand_conditioning_to_sample_batch on a batch with sampling params requesting n/num_samples > 1 while batch.prompt is None (e.g. a token-ids-only or image-only batch).

Common situations: Switching a pipeline to multi-output sampling (n>1) while feeding token ids instead of text prompts; refactor that stopped populating batch.prompt.

Related errors

• batching config rule requires max_batch_size
• Multi-output conditioning requires at least one prompt.
• kv-canary: RealKvSource.read_bytes must be a positive multip
• top_p must be scalar or have one value per row, got {top_ps.
• top_p values must be in (0, 1]

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/5444b5c77c116bc3. Report an issue: GitHub.