Lightning-AI/pytorch-lightning · error · ValueError

Expected lengths ({lengths}) to be greater or equal than sam

Error message

Expected lengths ({lengths}) to be greater or equal than samples ({samples})

What it means

ThroughputMonitor.update() validates that the reported tensor lengths (e.g. token counts for variable-length sequences) are at least as large as the sample count, since each sample must have at least one element of length. It raises ValueError when lengths < samples because the stats would be internally inconsistent.

Source

Thrown at src/lightning/fabric/utilities/throughput.py:161

            time: Total elapsed time in seconds. It should monotonically increase by the iteration time with each
                call.
            batches: Total batches seen per device. It should monotonically increase with each call.
            samples: Total samples seen per device. It should monotonically increase by the batch size with each call.
            lengths: Total length of the samples seen. It should monotonically increase by the lengths of a batch with
                each call.
            flops: Flops elapased per device since last ``update()`` call. You can easily compute this by using
                :func:`measure_flops` and multiplying it by the number of batches that have been processed.
                The value might be different in each device if the batch size is not the same.

        """
        self._time.append(time)
        if samples < batches:
            raise ValueError(f"Expected samples ({samples}) to be greater or equal than batches ({batches})")
        self._batches.append(batches)
        self._samples.append(samples)
        if lengths is not None:
            if lengths < samples:
                raise ValueError(f"Expected lengths ({lengths}) to be greater or equal than samples ({samples})")
            self._lengths.append(lengths)
            if len(self._samples) != len(self._lengths):
                raise RuntimeError(
                    f"If lengths are passed ({len(self._lengths)}), there needs to be the same number of samples"
                    f" ({len(self._samples)})"
                )
        if flops is not None:
            # sum of flops across ranks
            self._flops.append(flops * self.world_size)

    def compute(self) -> _THROUGHPUT_METRICS:
        """Compute throughput metrics."""
        metrics = {
            "time": self._time[-1],
            "batches": self._batches[-1],
            "samples": self._samples[-1],
        }
        if self._lengths:

View on GitHub (pinned to 9fed5c27d2)

Solutions

1. Check the values you pass: lengths must be >= samples (e.g. sum of sequence lengths over the batch, not per-sample length)
2. If sequences are variable length, pass lengths as the total number of tokens in the batch, not the max/mean sequence length
3. Verify you did not swap the samples and lengths arguments

Example fix

# before
throughput.update(batch=(x := next(dl))[0].shape[0], samples=x[0].shape[0], lengths=seq_len)  # seq_len < batch size

# after
throughput.update(batch=batch_idx, samples=batch_size, lengths=int(lengths_tensor.sum().item()))

samples = batch_size
lengths = int(lengths_tensor.sum().item()) if lengths_tensor is not None else None
assert lengths is None or lengths >= samples, f"lengths {lengths} < samples {samples}"

When it happens

Trigger: Calling throughput.update(batch=..., samples=N, lengths=M) with M < N, e.g. samples=64 but lengths=32 (lengths not reduced per-rank consistently, or passing padded token counts smaller than batch size, or mixing up argument order between samples and lengths).

Common situations: User computes lengths as an int that lost its batch dimension (e.g. passing seq_len instead of seq_len * batch_size), or aggregates lengths only over a subset of the batch, or confuses samples (batch size) with total elements when using variable-length data (packed sequences, tokenized corpora).

Related errors

• Device should be CUDA, got {device} instead.
• You requested to find {num_devices} devices but there are no
• `setup_optimizers` requires at least one optimizer as input.
• `setup_dataloaders` requires at least one dataloader as inpu
• Passed `{type(self).__name__}(precision={precision!r})`. Pre

AI-assisted analysis of Lightning-AI/pytorch-lightning@9fed5c27d2 (2026-08-28). Data as JSON: /api/errors/1b6bd1d605152343. Report an issue: GitHub.