hpcaitech/Open-Sora · error · ValueError

Input img and txt tensors must have 3 dimensions.

Error message

Input img and txt tensors must have 3 dimensions.

What it means

prepare_block_inputs expects img (noisy latent sequence) and txt (T5 text embeddings) as rank-3 [B, L, C] tensors — MMDiT operates on token sequences. If either tensor is 4D (e.g. raw [B, C, T, H, W] latents not patchified) or 2D (unbatched), it raises.

Source

Thrown at opensora/models/mmdit/model.py:173

        self,
        img: Tensor,
        img_ids: Tensor,
        txt: Tensor,  # t5 encoded vec
        txt_ids: Tensor,
        timesteps: Tensor,
        y_vec: Tensor,  # clip encoded vec
        cond: Tensor = None,
        guidance: Tensor | None = None,
    ):
        """
        obtain the processed:
            img: projected noisy img latent,
            txt: text context (from t5),
            vec: clip encoded vector,
            pe: the positional embeddings for concatenated img and txt
        """
        if img.ndim != 3 or txt.ndim != 3:
            raise ValueError("Input img and txt tensors must have 3 dimensions.")

        # running on sequences img
        img = self.img_in(img)
        if self.config.cond_embed:
            if cond is None:
                raise ValueError("Didn't get conditional input for conditional model.")
            img = img + self.cond_in(cond)

        vec = self.time_in(timestep_embedding(timesteps, 256))
        if self.config.guidance_embed:
            if guidance is None:
                raise ValueError(
                    "Didn't get guidance strength for guidance distilled model."
                )
            vec = vec + self.guidance_in(timestep_embedding(guidance, 256))
        vec = vec + self.vector_in(y_vec)

        txt = self.txt_in(txt)

View on GitHub (pinned to 7ad6a96a13)

Solutions

1. Patchify/flatten img to [B, seq_len, channels] (use the model's patch embed / arrange '(b t h w) (c p1 p2 p3)')
2. Ensure txt has shape [B, L_txt, txt_channels]; re-insert the batch dim with .unsqueeze(0) if needed
3. Print img.shape and txt.shape right before the call and confirm both are 3-D

Example fix

# before
out = model(img=latents_bcthw, txt=t5_emb)  # img.ndim == 4 → error
# after
img = patchify(latents_bcthw)  # → [B, L, C]
txt = t5_emb if t5_emb.ndim == 3 else t5_emb.unsqueeze(0)
out = model(img=img, txt=txt, ...)

assert img.ndim == 3 and txt.ndim == 3, f"img {tuple(img.shape)} txt {tuple(txt.shape)} must be [B, L, C]"

def is_seq3(t: torch.Tensor) -> bool:
    return torch.is_tensor(t) and t.dim() == 3

When it happens

Trigger: Calling the model forward with img that still has spatial dims (missing patchify/flatten step) or txt embeddings with a squeezed/missing batch dimension; also affects forward_ckpt and forward_selective_ckpt paths.

Common situations: Custom pipelines that pass VAE latents straight to MMDit without the patchifier; accidentally squeezing text embeddings; batch-of-one code that drops the batch dim.

Related errors

• The last dimension D must be even.
• Hidden size {config.hidden_size} must be divisible by num_he
• Got {config.axes_dim} but expected positional dim {pe_dim}
• Didn't get conditional input for conditional model.
• Didn't get guidance strength for guidance distilled model.

AI-assisted analysis of hpcaitech/Open-Sora@7ad6a96a13 (2026-08-28). Data as JSON: /api/errors/a363cacb5df2eb15. Report an issue: GitHub.