rohitg00/ai-engineering-from-scratch · error · TypeError

ids must be long tensor, got {ids.dtype}

Error message

ids must be long tensor, got {ids.dtype}

What it means

Error "ids must be long tensor, got {ids.dtype}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/32-token-positional-embeddings/code/main.py:48

class TokenEmbedding(nn.Module):
    """Vocabulary-id to vector lookup."""

    def __init__(self, vocab_size: int, d_model: int, init_std: float = DEFAULT_INIT_STD) -> None:
        super().__init__()
        if vocab_size < 1:
            raise ValueError(f"vocab_size must be >= 1, got {vocab_size}")
        if d_model < 1:
            raise ValueError(f"d_model must be >= 1, got {d_model}")
        self.vocab_size = vocab_size
        self.d_model = d_model
        self.embedding = nn.Embedding(vocab_size, d_model)
        _init_normal(self.embedding.weight, std=init_std)

    def forward(self, ids: torch.Tensor) -> torch.Tensor:
        if ids.dtype != torch.long:
            raise TypeError(f"ids must be long tensor, got {ids.dtype}")
        if ids.dim() != 2:
            raise ValueError(f"ids must be (B, T), got shape {tuple(ids.shape)}")
        return self.embedding(ids)


class LearnedPositionalEmbedding(nn.Module):
    """Position-id to vector lookup with learned parameters."""

    def __init__(
        self,
        max_context_length: int,
        d_model: int,
        init_std: float = DEFAULT_INIT_STD,
    ) -> None:
        super().__init__()
        if max_context_length < 1:
            raise ValueError(f"max_context_length must be >= 1, got {max_context_length}")
        if d_model < 1:

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/32-token-positional-embeddings/code/main.py:48 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/a53b7c2823406b1a. Report an issue: GitHub.