rohitg00/ai-engineering-from-scratch · error · ValueError

d_model must be even for sinusoidal, got {d_model}

Error message

d_model must be even for sinusoidal, got {d_model}

What it means

Error "d_model must be even for sinusoidal, got {d_model}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/32-token-positional-embeddings/code/main.py:98

        positions = torch.arange(seq_len, device=self.embedding.weight.device)
        return self.embedding(positions)


class SinusoidalPositionalEmbedding(nn.Module):
    """Parameter-free position-to-vector mapping.

    pe[p, 2k]     = sin(p / 10000^(2k/d_model))
    pe[p, 2k+1]   = cos(p / 10000^(2k/d_model))
    """

    def __init__(self, max_context_length: int, d_model: int, base: float = 10000.0) -> None:
        super().__init__()
        if max_context_length < 1:
            raise ValueError(f"max_context_length must be >= 1, got {max_context_length}")
        if d_model < 1:
            raise ValueError(f"d_model must be >= 1, got {d_model}")
        if d_model % 2 != 0:
            raise ValueError(f"d_model must be even for sinusoidal, got {d_model}")
        self.max_context_length = max_context_length
        self.d_model = d_model
        self.base = base
        pe = self._build_table(max_context_length, d_model, base)
        self.register_buffer("pe", pe, persistent=False)

    @staticmethod
    def _build_table(max_context_length: int, d_model: int, base: float) -> torch.Tensor:
        pos = torch.arange(max_context_length, dtype=torch.float32).unsqueeze(1)
        i = torch.arange(d_model // 2, dtype=torch.float32)
        denom = base ** (2 * i / d_model)
        angle = pos / denom
        pe = torch.zeros(max_context_length, d_model, dtype=torch.float32)
        pe[:, 0::2] = torch.sin(angle)
        pe[:, 1::2] = torch.cos(angle)
        return pe

    def forward(self, seq_len: int) -> torch.Tensor:

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/32-token-positional-embeddings/code/main.py:98 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/b30f9ee0adee55c3. Report an issue: GitHub.