rohitg00/ai-engineering-from-scratch · error · ValueError

seq_len {seq_len} exceeds max_context_length {self.max_conte

Error message

seq_len {seq_len} exceeds max_context_length {self.max_context_length}

What it means

Error "seq_len {seq_len} exceeds max_context_length {self.max_context_length}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/32-token-positional-embeddings/code/main.py:77

        max_context_length: int,
        d_model: int,
        init_std: float = DEFAULT_INIT_STD,
    ) -> None:
        super().__init__()
        if max_context_length < 1:
            raise ValueError(f"max_context_length must be >= 1, got {max_context_length}")
        if d_model < 1:
            raise ValueError(f"d_model must be >= 1, got {d_model}")
        self.max_context_length = max_context_length
        self.d_model = d_model
        self.embedding = nn.Embedding(max_context_length, d_model)
        _init_normal(self.embedding.weight, std=init_std)

    def forward(self, seq_len: int) -> torch.Tensor:
        if seq_len < 1:
            raise ValueError(f"seq_len must be >= 1, got {seq_len}")
        if seq_len > self.max_context_length:
            raise ValueError(
                f"seq_len {seq_len} exceeds max_context_length {self.max_context_length}"
            )
        positions = torch.arange(seq_len, device=self.embedding.weight.device)
        return self.embedding(positions)


class SinusoidalPositionalEmbedding(nn.Module):
    """Parameter-free position-to-vector mapping.

    pe[p, 2k]     = sin(p / 10000^(2k/d_model))
    pe[p, 2k+1]   = cos(p / 10000^(2k/d_model))
    """

    def __init__(self, max_context_length: int, d_model: int, base: float = 10000.0) -> None:
        super().__init__()
        if max_context_length < 1:
            raise ValueError(f"max_context_length must be >= 1, got {max_context_length}")
        if d_model < 1:

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/32-token-positional-embeddings/code/main.py:77 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/83fb82664d264b44. Report an issue: GitHub.