rohitg00/ai-engineering-from-scratch · error · ValueError

shape mismatch image {tuple(image_emb.shape)} vs text {tuple

Error message

shape mismatch image {tuple(image_emb.shape)} vs text {tuple(text_emb.shape)}

What it means

Error "shape mismatch image {tuple(image_emb.shape)} vs text {tuple(text_emb.shape)}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/62-vision-language-pretraining/code/main.py:82

    n_pairs: int = 200
    batch_size: int = 16
    steps: int = 50
    lr: float = 5e-4
    lm_weight: float = 1.0
    init_log_tau: float = math.log(1.0 / 0.07)
    seed: int = 0


def info_nce_loss(image_emb: torch.Tensor, text_emb: torch.Tensor,
                  log_tau: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
    """Bidirectional InfoNCE used in CLIP and friends.

    Returns (loss, similarity_matrix). image_emb and text_emb must have the
    same shape (N, D). The similarity matrix is symmetric in semantics but not
    in values (rows are images, columns are texts).
    """
    if image_emb.shape != text_emb.shape:
        raise ValueError(
            f"shape mismatch image {tuple(image_emb.shape)} vs text {tuple(text_emb.shape)}"
        )
    n = image_emb.shape[0]
    img_n = F.normalize(image_emb, dim=-1)
    txt_n = F.normalize(text_emb, dim=-1)

    scale = log_tau.exp().clamp(min=1e-3, max=100.0)
    sim = (img_n @ txt_n.T) * scale

    targets = torch.arange(n, device=sim.device)
    loss_i2t = F.cross_entropy(sim, targets)
    loss_t2i = F.cross_entropy(sim.T, targets)
    return (loss_i2t + loss_t2i) * 0.5, sim


def lm_loss(logits: torch.Tensor, target_ids: torch.Tensor,
            padding_id: int = PAD_ID) -> torch.Tensor:
    """Next-token cross-entropy with padding masked.

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/62-vision-language-pretraining/code/main.py:82 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/d08a6e62c730cfaa. Report an issue: GitHub.