rohitg00/ai-engineering-from-scratch · error · ValueError

expected (B, L), got {tuple(ids.shape)}

Error message

expected (B, L), got {tuple(ids.shape)}

What it means

Error "expected (B, L), got {tuple(ids.shape)}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/62-vision-language-pretraining/code/main.py:123

    """
    if logits.dim() != 3 or target_ids.dim() != 2:
        raise ValueError(f"logits must be 3D and targets 2D, got {logits.shape} {target_ids.shape}")
    b, l, v = logits.shape
    flat_logits = logits.reshape(b * l, v)
    flat_target = target_ids.reshape(b * l)
    return F.cross_entropy(flat_logits, flat_target, ignore_index=padding_id)


class TextSideEncoder(nn.Module):
    """Tiny text encoder: embedding lookup + mean pool over non-padding tokens."""

    def __init__(self, vocab_size: int, embed_dim: int) -> None:
        super().__init__()
        self.embed = nn.Embedding(vocab_size, embed_dim, padding_idx=PAD_ID)

    def forward(self, ids: torch.Tensor) -> torch.Tensor:
        if ids.dim() != 2:
            raise ValueError(f"expected (B, L), got {tuple(ids.shape)}")
        x = self.embed(ids)
        mask = (ids != PAD_ID).float().unsqueeze(-1)
        denom = mask.sum(dim=1).clamp(min=1.0)
        return (x * mask).sum(dim=1) / denom


class MultimodalModel(nn.Module):
    """Encoder + projection + text side + cross-attention decoder, all trainable."""

    def __init__(self, cfg: PretrainConfig) -> None:
        super().__init__()
        self.cfg = cfg

        vit_cfg = ViTConfig(
            image_size=32,
            patch_size=16,
            hidden=cfg.vision_hidden,
            depth=2,

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/62-vision-language-pretraining/code/main.py:123 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/2d05572c43d651f7. Report an issue: GitHub.