{"record":{"id":"ecd582a38033629d","repo":"sgl-project/sglang","slug":"name-has-batch-dim-current-batch-size-shape","errorCode":null,"errorMessage":"{name} has batch dim {current_batch_size} (shape {tuple(value.shape)}); expected {self.prompt_batch_size} (per-prompt) or {self.sample_batch_size} (per-sample).","messagePattern":"(.+?) has batch dim (.+?) \\(shape (.+?)\\); expected (.+?) \\(per-prompt\\) or (.+?) \\(per-sample\\)\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"python/sglang/multimodal_gen/runtime/utils/condition_expansion.py","lineNumber":38,"sourceCode":"        if isinstance(batch.prompt, list):\n            prompt_batch_size = len(batch.prompt)\n        elif batch.prompt is not None:\n            prompt_batch_size = 1\n        else:\n            raise ValueError(\n                \"Multi-output conditioning requires prompt text so the prompt \"\n                \"batch size is unambiguous.\"\n            )\n        if prompt_batch_size <= 0:\n            raise ValueError(\"Multi-output conditioning requires at least one prompt.\")\n        return cls(prompt_batch_size, prompt_batch_size * num_outputs)\n\n    def _expand_tensor(self, value: torch.Tensor, name: str) -> torch.Tensor:\n        current_batch_size = value.shape[0]\n        if current_batch_size == self.sample_batch_size:\n            return value\n        if current_batch_size != self.prompt_batch_size:\n            raise ValueError(\n                f\"{name} has batch dim {current_batch_size} (shape \"\n                f\"{tuple(value.shape)}); expected {self.prompt_batch_size} \"\n                f\"(per-prompt) or {self.sample_batch_size} (per-sample).\"\n            )\n        repeats = self.sample_batch_size // self.prompt_batch_size\n        return value.repeat_interleave(repeats, dim=0)\n\n    def _expand_tensors(self, value, name: str):\n        \"\"\"Expand a tensor or each tensor in a list, preserving its container.\"\"\"\n        if value is None:\n            return None\n        if isinstance(value, torch.Tensor):\n            return self._expand_tensor(value, name)\n        if not isinstance(value, list):\n            raise TypeError(f\"{name} must be a tensor, list of tensors, or None.\")\n        if any(\n            item is not None and not isinstance(item, torch.Tensor) for item in value\n        ):","sourceCodeStart":20,"sourceCodeEnd":56,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/multimodal_gen/runtime/utils/condition_expansion.py#L20-L56","documentation":"Thrown by ConditionExpansion._expand_tensor when a conditioning tensor's leading dimension matches neither sample_batch_size (already per-sample, returned as-is) nor prompt_batch_size (per-prompt, to be repeated). The tensor cannot be mapped onto the batch layout.","triggerScenarios":"Calling expand_field on a tensor whose shape[0] differs from both prompt_batch_size and sample_batch_size — e.g. a per-token tensor, a stale tensor built for a different batch, or a list-length mismatch baked into dim 0.","commonSituations":"Mixing tensors from a previous batch after the prompt list changed; passing hidden states or per-image features whose batch dim doesn't align; prompt list edited between steps.","solutions":["Rebuild the offending tensor so dim 0 equals the number of prompts (it will be repeat_interleave'd) or the number of samples","Verify the tensor wasn't carried over from a previous, differently-sized batch","If it's per-token/per-step data, exclude it from expand_field and handle it separately"],"exampleFix":"# before\n# prompt_batch_size=2, num_outputs=4, but guidance has dim 0 == 3\nexpand.expand_field(guidance, \"guidance\")\n# after\nguidance = guidance[:2]  # one entry per prompt\nexpand.expand_field(guidance, \"guidance\")","handlingStrategy":"validation","validationCode":"assert value.shape[0] in (expansion.prompt_batch_size, expansion.sample_batch_size), f\"bad batch dim {value.shape[0]}\"","typeGuard":"def expandable_batch_dim(t: \"torch.Tensor\", exp) -> bool:\n    return t.dim() > 0 and t.shape[0] in (exp.prompt_batch_size, exp.sample_batch_size)","tryCatchPattern":"try:\n    out = exp.expand_field(value, name)\nexcept ValueError as e:\n    if \"batch dim\" in str(e):\n        raise RuntimeError(f\"stale conditioning tensor {name}; rebuild it for the current batch\") from e\n    raise","preventionTips":["Never reuse conditioning tensors across differently-sized batches","Keep one source of truth for prompt count and build all conditioning from it"],"tags":["batching","tensor-shape","validation"],"backgroundTag":"batch-shape-mismatch","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}