rohitg00/ai-engineering-from-scratch · error · ValueError

expected (B, L) ids, got {tuple(ids.shape)}

Error message

expected (B, L) ids, got {tuple(ids.shape)}

What it means

Error "expected (B, L) ids, got {tuple(ids.shape)}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/60-projection-layer-modality-align/code/main.py:90

class MockTextEmbedding(nn.Module):
    """Frozen text table used as alignment targets.

    Captions are sequences of token ids; the caption embedding is the mean of
    the embedded ids. Deterministic given seed.
    """

    def __init__(self, vocab_size: int, dim: int, seed: int) -> None:
        super().__init__()
        gen = torch.Generator().manual_seed(seed)
        weight = torch.randn(vocab_size, dim, generator=gen) * (1.0 / dim ** 0.5)
        self.table = nn.Embedding(vocab_size, dim, _weight=weight)
        for p in self.table.parameters():
            p.requires_grad_(False)

    def forward(self, ids: torch.Tensor) -> torch.Tensor:
        if ids.dim() != 2:
            raise ValueError(f"expected (B, L) ids, got {tuple(ids.shape)}")
        embed = self.table(ids)
        mask = (ids != 0).float().unsqueeze(-1)
        denom = mask.sum(dim=1).clamp(min=1.0)
        pooled = (embed * mask).sum(dim=1) / denom
        return pooled


def make_pair(seed: int, vocab_size: int, max_len: int) -> tuple[torch.Tensor, torch.Tensor]:
    """One synthetic (image, caption_ids) pair.

    Image is the deterministic 224x224x3 fixture from lesson 58 with a per-pair
    seed. Caption is a length-`max_len` sequence of token ids, again
    deterministic in seed. Token id 0 is reserved as padding.
    """
    img = synthesize_image(seed=seed)
    rng = np.random.default_rng(seed + 10_000)
    length = int(rng.integers(4, max_len + 1))
    ids = np.zeros((max_len,), dtype=np.int64)

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/60-projection-layer-modality-align/code/main.py:90 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/3dd8c7f6cd31f44c. Report an issue: GitHub.