sgl-project/sglang · error · ValueError

{name} has batch dim {current_batch_size} (shape {tuple(valu

Error message

{name} has batch dim {current_batch_size} (shape {tuple(value.shape)}); expected {self.prompt_batch_size} (per-prompt) or {self.sample_batch_size} (per-sample).

What it means

Thrown by ConditionExpansion._expand_tensor when a conditioning tensor's leading dimension matches neither sample_batch_size (already per-sample, returned as-is) nor prompt_batch_size (per-prompt, to be repeated). The tensor cannot be mapped onto the batch layout.

Source

Thrown at python/sglang/multimodal_gen/runtime/utils/condition_expansion.py:38

        if isinstance(batch.prompt, list):
            prompt_batch_size = len(batch.prompt)
        elif batch.prompt is not None:
            prompt_batch_size = 1
        else:
            raise ValueError(
                "Multi-output conditioning requires prompt text so the prompt "
                "batch size is unambiguous."
            )
        if prompt_batch_size <= 0:
            raise ValueError("Multi-output conditioning requires at least one prompt.")
        return cls(prompt_batch_size, prompt_batch_size * num_outputs)

    def _expand_tensor(self, value: torch.Tensor, name: str) -> torch.Tensor:
        current_batch_size = value.shape[0]
        if current_batch_size == self.sample_batch_size:
            return value
        if current_batch_size != self.prompt_batch_size:
            raise ValueError(
                f"{name} has batch dim {current_batch_size} (shape "
                f"{tuple(value.shape)}); expected {self.prompt_batch_size} "
                f"(per-prompt) or {self.sample_batch_size} (per-sample)."
            )
        repeats = self.sample_batch_size // self.prompt_batch_size
        return value.repeat_interleave(repeats, dim=0)

    def _expand_tensors(self, value, name: str):
        """Expand a tensor or each tensor in a list, preserving its container."""
        if value is None:
            return None
        if isinstance(value, torch.Tensor):
            return self._expand_tensor(value, name)
        if not isinstance(value, list):
            raise TypeError(f"{name} must be a tensor, list of tensors, or None.")
        if any(
            item is not None and not isinstance(item, torch.Tensor) for item in value
        ):

View on GitHub (pinned to 0132848349)

Solutions

1. Rebuild the offending tensor so dim 0 equals the number of prompts (it will be repeat_interleave'd) or the number of samples
2. Verify the tensor wasn't carried over from a previous, differently-sized batch
3. If it's per-token/per-step data, exclude it from expand_field and handle it separately

Example fix

# before
# prompt_batch_size=2, num_outputs=4, but guidance has dim 0 == 3
expand.expand_field(guidance, "guidance")
# after
guidance = guidance[:2]  # one entry per prompt
expand.expand_field(guidance, "guidance")

assert value.shape[0] in (expansion.prompt_batch_size, expansion.sample_batch_size), f"bad batch dim {value.shape[0]}"

def expandable_batch_dim(t: "torch.Tensor", exp) -> bool:
    return t.dim() > 0 and t.shape[0] in (exp.prompt_batch_size, exp.sample_batch_size)

try:
    out = exp.expand_field(value, name)
except ValueError as e:
    if "batch dim" in str(e):
        raise RuntimeError(f"stale conditioning tensor {name}; rebuild it for the current batch") from e
    raise

When it happens

Trigger: Calling expand_field on a tensor whose shape[0] differs from both prompt_batch_size and sample_batch_size — e.g. a per-token tensor, a stale tensor built for a different batch, or a list-length mismatch baked into dim 0.

Common situations: Mixing tensors from a previous batch after the prompt list changed; passing hidden states or per-image features whose batch dim doesn't align; prompt list edited between steps.

Related errors

• `dt_bias` must have {HV * K} elements (got {dt_bias.numel()}
• `mixed_qkv` must be 2D (got ndim={mixed_qkv.ndim}).
• num_token_non_padded must be a single-element tensor, got sh
• Z-Image text embeddings must have shape [seq, dim] or [batch
• f"Expected camera embedding shape [B, C, F, H, W], got {tupl

AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28). Data as JSON: /api/errors/ecd582a38033629d. Report an issue: GitHub.