invoke-ai/InvokeAI · error · ValueError

Text embedding y must be [B, L, D]

Error message

Text embedding y must be [B, L, D]

What it means

forward() expects the text conditioning tensor y to be a 3-D tensor of shape [batch, sequence, dim]. Any other rank (e.g. a flattened [B*L, D] tensor or a 2-D [B, D] pooled embedding) cannot be indexed as [B, L, D], so the model raises ValueError immediately. This protects the downstream y_embedder and positional-embedding add, which assume exactly three dimensions.

Source

Thrown at invokeai/backend/pid/_src/networks/pixeldit_official.py:1428

        nn.init.zeros_(self.final_layer.linear.weight)
        nn.init.zeros_(self.final_layer.linear.bias)

    def forward(self, x, t, y, s=None, mask=None):
        B, _, H, W = x.shape
        # Derive grid token count deterministically from spatial size
        Hs = H // self.patch_size
        Ws = W // self.patch_size
        L = Hs * Ws

        # Patch tokens for condition pathway
        pos = self.fetch_pos(Hs, Ws, x.device)
        x_patches = torch.nn.functional.unfold(x, kernel_size=self.patch_size, stride=self.patch_size).transpose(1, 2)

        t_emb = self.t_embedder(t.view(-1)).view(B, -1, self.hidden_size)

        # Text tokens -> project to hidden_size and add learned pos
        if y.dim() != 3:
            raise ValueError("Text embedding y must be [B, L, D]")
        Ltxt = min(y.shape[1], self.txt_max_length)
        y = y[:, :Ltxt, :]
        y_emb = self.y_embedder(y).view(B, Ltxt, self.hidden_size)
        y_emb = y_emb + self.y_pos_embedding[:, :Ltxt, :].to(y_emb.dtype)

        # PixDiT design: no AdaLN modulation applied on text stream
        condition = torch.nn.functional.silu(t_emb)

        # Condition blocks on patch tokens with MM-DiT joint attention to text tokens
        pad = None
        pos_txt = self.fetch_pos_text(Ltxt, x.device) if self.use_text_rope else None
        if mask is not None and isinstance(mask, torch.Tensor):
            m = mask
            while m.dim() > 2 and m.size(1) == 1:
                m = m.squeeze(1)
            if m.dim() == 3 and m.size(1) == 1:
                m = m.squeeze(1)
            if m.dim() == 2:

View on GitHub (pinned to 0b6a024f2f)

Solutions

  1. Unsqueeze a missing sequence dimension: y = y.unsqueeze(1) for [B, D] -> [B, 1, D]
  2. If y is [B*L, D], reshape to [B, L, D] using the known sequence length
  3. Verify the text encoder output shape before passing it as conditioning

Example fix

// before
model(x, t, text_emb.squeeze(0))
// after
if text_emb.dim() == 2:
    text_emb = text_emb.unsqueeze(1)
model(x, t, text_emb)
Defensive patterns

Strategy: type-guard

Validate before calling

assert isinstance(y, torch.Tensor) and y.dim() == 3, f"y must be [B, L, D], got {tuple(y.shape)}"

Type guard

def is_text_emb(y) -> bool:
    return isinstance(y, torch.Tensor) and y.dim() == 3

Try / catch

try:
    out = model(x, t, y)
except ValueError as e:
    if "must be [B, L, D]" in str(e):
        y = y.unsqueeze(1) if y.dim() == 2 else y.view(B, -1, y.shape[-1])
        out = model(x, t, y)
    else:
        raise

Prevention

When it happens

Trigger: Passing y with y.dim() != 3 to PixDiT_T2I.forward, e.g. after an accidental squeeze(), from an encoder that returns pooled 2-D embeddings, or after batching/reshaping mistakes.

Common situations: Swapping text encoders (one returns [B, D], another [B, L, D]), calling forward with pre-pooled CLS embeddings, or denoising-loop wrappers that reshape the conditioning tensor.

Understand the failure class

Background: Tensor shape mismatch errors ("must have shape", "expected shape ... got ..."): when tensor dimensions disagree with what an op or layer was told to expect — this error's family across 6 libraries.

Related errors

AI-assisted analysis of invoke-ai/InvokeAI@0b6a024f2f (2026-08-29). Data as JSON: /api/errors/01fa1a3711522d3d. Report an issue: GitHub.