rohitg00/ai-engineering-from-scratch · error · ValueError

expected (B, N, D), got {tuple(x.shape)}

Error message

expected (B, N, D), got {tuple(x.shape)}

What it means

Error "expected (B, N, D), got {tuple(x.shape)}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/59-vit-transformer/code/main.py:86

            patch_size=self.patch_size,
            in_channels=self.in_channels,
            hidden=self.hidden,
        )


class MultiHeadSelfAttention(nn.Module):
    def __init__(self, cfg: ViTConfig) -> None:
        super().__init__()
        self.cfg = cfg
        self.qkv = nn.Linear(cfg.hidden, cfg.hidden * 3, bias=True)
        self.out = nn.Linear(cfg.hidden, cfg.hidden, bias=True)
        self.drop = nn.Dropout(cfg.dropout)
        self.scale = 1.0 / math.sqrt(cfg.head_dim)
        self.last_attn: torch.Tensor | None = None

    def forward(self, x: torch.Tensor, store_attn: bool = False) -> torch.Tensor:
        if x.dim() != 3:
            raise ValueError(f"expected (B, N, D), got {tuple(x.shape)}")
        b, n, d = x.shape
        h = self.cfg.heads
        hd = self.cfg.head_dim

        qkv = self.qkv(x).reshape(b, n, 3, h, hd).permute(2, 0, 3, 1, 4)
        q, k, v = qkv[0], qkv[1], qkv[2]

        scores = (q @ k.transpose(-2, -1)) * self.scale
        attn = F.softmax(scores, dim=-1)
        if store_attn:
            self.last_attn = attn.detach()

        out = attn @ v
        out = out.transpose(1, 2).reshape(b, n, d)
        out = self.out(out)
        out = self.drop(out)
        return out

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/59-vit-transformer/code/main.py:86 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/7f6ad7b64ebed2ec. Report an issue: GitHub.