rohitg00/ai-engineering-from-scratch · error · ValueError

context_length must be >= 1, got {context_length}

Error message

context_length must be >= 1, got {context_length}

What it means

Error "context_length must be >= 1, got {context_length}" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/31-tokenized-dataset-sliding-window/code/main.py:165

        out.extend(symbols)
    return out


class SlidingWindowDataset(Dataset):
    """PyTorch Dataset over a flat id stream.

    Each example is a window of size T+1. __getitem__ returns
    (input_ids, target_ids) where target = input shifted left by one.
    """

    def __init__(
        self,
        ids: list[int],
        context_length: int,
        stride: int | None = None,
    ) -> None:
        if context_length < 1:
            raise ValueError(f"context_length must be >= 1, got {context_length}")
        if not ids:
            raise ValueError("ids must be non-empty")
        if stride is None:
            stride = context_length
        if stride < 1:
            raise ValueError(f"stride must be >= 1, got {stride}")
        self.ids = torch.tensor(ids, dtype=torch.long)
        self.context_length = context_length
        self.stride = stride

    @staticmethod
    def count_windows(num_ids: int, context_length: int, stride: int) -> int:
        usable = num_ids - (context_length + 1)
        if usable < 0:
            return 0
        return 1 + usable // stride

    def __len__(self) -> int:

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/31-tokenized-dataset-sliding-window/code/main.py:165 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/10c89d65a1158b75. Report an issue: GitHub.