{"record":{"id":"5444b5c77c116bc3","repo":"sgl-project/sglang","slug":"multi-output-conditioning-requires-prompt-text-so","errorCode":null,"errorMessage":"Multi-output conditioning requires prompt text so the prompt batch size is unambiguous.","messagePattern":"Multi-output conditioning requires prompt text so the prompt batch size is unambiguous\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/condition_expansion.py","lineNumber":25,"sourceCode":"\n@dataclass(frozen=True)\nclass PromptToSampleBatchExpander:\n    \"\"\"Expand selected conditioning from prompt order to sample order.\"\"\"\n\n    prompt_batch_size: int\n    sample_batch_size: int\n\n    @classmethod\n    def from_batch(cls, batch):\n        num_outputs = int(batch.num_outputs_per_prompt or 1)\n        if num_outputs <= 1:\n            return None\n        if isinstance(batch.prompt, list):\n            prompt_batch_size = len(batch.prompt)\n        elif batch.prompt is not None:\n            prompt_batch_size = 1\n        else:\n            raise ValueError(\n                \"Multi-output conditioning requires prompt text so the prompt \"\n                \"batch size is unambiguous.\"\n            )\n        if prompt_batch_size <= 0:\n            raise ValueError(\"Multi-output conditioning requires at least one prompt.\")\n        return cls(prompt_batch_size, prompt_batch_size * num_outputs)\n\n    def _expand_tensor(self, value: torch.Tensor, name: str) -> torch.Tensor:\n        current_batch_size = value.shape[0]\n        if current_batch_size == self.sample_batch_size:\n            return value\n        if current_batch_size != self.prompt_batch_size:\n            raise ValueError(\n                f\"{name} has batch dim {current_batch_size} (shape \"\n                f\"{tuple(value.shape)}); expected {self.prompt_batch_size} \"\n                f\"(per-prompt) or {self.sample_batch_size} (per-sample).\"\n            )\n        repeats = self.sample_batch_size // self.prompt_batch_size","sourceCodeStart":7,"sourceCodeEnd":43,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/condition_expansion.py#L7-L43","documentation":"Thrown by ConditionExpansion.from_batch when num_outputs > 1 but batch.prompt is None — with multiple outputs per prompt, the runtime needs prompt text so it knows how many prompts are in the batch and can expand each per-prompt conditioning num_outputs times.","triggerScenarios":"Calling expand_conditioning_to_sample_batch on a batch with sampling params requesting n/num_samples > 1 while batch.prompt is None (e.g. a token-ids-only or image-only batch).","commonSituations":"Switching a pipeline to multi-output sampling (n>1) while feeding token ids instead of text prompts; refactor that stopped populating batch.prompt.","solutions":["Populate batch.prompt (list of strings, or a single string) before requesting multiple outputs per prompt","If prompts are genuinely unavailable, set num_outputs/n back to 1 so expansion is trivial","Ensure the tokenizer/front-end attaches the original text to the batch object"],"exampleFix":"# before\nbatch = SampleBatch(input_ids=ids, prompt=None)  # n=4\n# after\nbatch = SampleBatch(input_ids=ids, prompt=[\"describe this image\"])  # n=4","handlingStrategy":"validation","validationCode":"if num_outputs > 1:\n    assert batch.prompt is not None and (not isinstance(batch.prompt, list) or len(batch.prompt) > 0), \"multi-output needs prompt text\"","typeGuard":"def supports_multi_output(batch) -> bool:\n    return batch.prompt is not None and (not isinstance(batch.prompt, list) or len(batch.prompt) > 0)","tryCatchPattern":"try:\n    expansion = expand_conditioning_to_sample_batch(batch, num_outputs)\nexcept ValueError as e:\n    if \"requires prompt text\" in str(e):\n        batch.prompt = tokenizer.decode(batch.input_ids[0])\n        expansion = expand_conditioning_to_sample_batch(batch, num_outputs)\n    else:\n        raise","preventionTips":["Always attach decoded prompt text to batches when n>1 sampling may be requested","Default num_outputs to 1 in pipelines that can't guarantee prompts"],"tags":["batching","sampling","multimodal","validation"],"backgroundTag":"batch-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}