rohitg00/ai-engineering-from-scratch · error · ValueError

num_shards must be >= 1

Error message

num_shards must be >= 1

What it means

Error "num_shards must be >= 1" thrown in rohitg00/ai-engineering-from-scratch.

Source

Thrown at phases/19-capstone-projects/47-checkpoint-save-resume/code/main.py:223

    restore_rng_state(payload["rng"])
    s = payload["state"]
    return TrainState(
        step=int(s["step"]),
        epoch=int(s["epoch"]),
        batch_in_epoch=int(s["batch_in_epoch"]),
        losses=list(s["losses"]),
    )


def shard_keys_by_prefix(state_dict: Dict[str, torch.Tensor], num_shards: int) -> Dict[int, List[str]]:
    """Round-robin allocate parameter keys across shards.

    Production sharding usually goes by parameter group or by layer. The
    round robin keeps the shards roughly the same size for the demo and
    keeps the index easy to read.
    """
    if num_shards < 1:
        raise ValueError("num_shards must be >= 1")
    keys = sorted(state_dict.keys())
    shards: Dict[int, List[str]] = {i: [] for i in range(num_shards)}
    for i, k in enumerate(keys):
        shards[i % num_shards].append(k)
    return shards


def save_sharded_checkpoint(
    model: nn.Module,
    optimizer: torch.optim.Optimizer,
    scheduler: torch.optim.lr_scheduler._LRScheduler,
    state: TrainState,
    out_dir: Path,
    *,
    num_shards: int,
    extras: Optional[Dict[str, Any]] = None,
) -> Dict[str, Any]:
    out_dir.mkdir(parents=True, exist_ok=True)

View on GitHub (pinned to 39ea8a1c6d)

When it happens

Trigger: Thrown at phases/19-capstone-projects/47-checkpoint-save-resume/code/main.py:223 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of rohitg00/ai-engineering-from-scratch@39ea8a1c6d (2026-08-26). Data as JSON: /api/errors/f4be5fc22365b7b2. Report an issue: GitHub.