rohitg00/ai-engineering-from-scratch · error · ValueError

d_model must be even, got {d_model}

Error message

d_model must be even, got {d_model}

What it means

Error "d_model must be even, got {d_model}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/33-multihead-self-attention/code/main.py:124

        self.embedding = nn.Embedding(vocab_size, d_model)
        with torch.no_grad():
            self.embedding.weight.normal_(0.0, 0.02)

    def forward(self, ids: torch.Tensor) -> torch.Tensor:
        return self.embedding(ids)


class SinusoidalPositionalEmbedding(nn.Module):
    """Parameter-free sin/cos positional table (compact copy of lesson 32)."""

    def __init__(self, max_context_length: int, d_model: int, base: float = 10000.0) -> None:
        super().__init__()
        if max_context_length < 1:
            raise ValueError(f"max_context_length must be >= 1, got {max_context_length}")
        if d_model < 1:
            raise ValueError(f"d_model must be >= 1, got {d_model}")
        if d_model % 2 != 0:
            raise ValueError(f"d_model must be even, got {d_model}")
        self.max_context_length = max_context_length
        pos = torch.arange(max_context_length, dtype=torch.float32).unsqueeze(1)
        i = torch.arange(d_model // 2, dtype=torch.float32)
        denom = base ** (2 * i / d_model)
        angle = pos / denom
        pe = torch.zeros(max_context_length, d_model, dtype=torch.float32)
        pe[:, 0::2] = torch.sin(angle)
        pe[:, 1::2] = torch.cos(angle)
        self.register_buffer("pe", pe, persistent=False)

    def forward(self, seq_len: int) -> torch.Tensor:
        if seq_len < 1:
            raise ValueError(f"seq_len must be >= 1, got {seq_len}")
        if seq_len > self.max_context_length:
            raise ValueError(
                f"seq_len {seq_len} exceeds max_context_length {self.max_context_length}"
            )
        return self.pe[:seq_len]

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/33-multihead-self-attention/code/main.py:124 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/39c2215ed65eace0. Report an issue: GitHub.