huggingface/transformers · error · ValueError

Ngrams should be of shape (batch_size, num_ngrams, ngram_len

Error message

Ngrams should be of shape (batch_size, num_ngrams, ngram_len), where ngram_len is {self.ngram_len}, but is {ngrams.shape}

What it means

Error "Ngrams should be of shape (batch_size, num_ngrams, ngram_len), where ngram_len is {self.ngram_len}, but is {ngrams.shape}" thrown in huggingface/transformers.

Source

Thrown at src/transformers/generation/logits_process.py:2813

            current_hash = torch.add(current_hash, data[..., i])
            current_hash = torch.mul(current_hash, multiplier)
            current_hash = torch.add(current_hash, increment)
        return current_hash

    def compute_ngram_keys(self, ngrams: torch.LongTensor) -> torch.LongTensor:
        """Computes random keys for each ngram and depth.

        Args:
            ngrams (`torch.LongTensor`):
                Ngrams (batch_size, num_ngrams, ngram_len).

        Returns:
            ngram keys (batch_size, num_ngrams, depth).
        """
        if len(ngrams.shape) != 3:
            raise ValueError(f"Ngrams should be of shape (batch_size, num_ngrams, ngram_len), but is {ngrams.shape}")
        if ngrams.shape[2] != self.ngram_len:
            raise ValueError(
                "Ngrams should be of shape (batch_size, num_ngrams, ngram_len),"
                f" where ngram_len is {self.ngram_len}, but is {ngrams.shape}"
            )
        batch_size, _, _ = ngrams.shape

        hash_result = torch.ones(batch_size, device=self.device, dtype=torch.long)
        # hash_result shape [batch_size,]
        # ngrams shape [batch_size, num_ngrams, ngram_len]
        hash_result = torch.vmap(self.accumulate_hash, in_dims=(None, 1), out_dims=1)(hash_result, ngrams)
        # hash_result shape [batch_size, num_ngrams]

        keys = self.keys[None, None, :, None]
        # hash_result shape [batch_size, num_ngrams]
        # keys shape [1, 1, depth, 1]
        hash_result = torch.vmap(self.accumulate_hash, in_dims=(None, 2), out_dims=2)(hash_result, keys)
        # hash_result shape [batch_size, num_ngrams, depth]

        return hash_result

View on GitHub (pinned to a597f97485)

Solutions

1. Ensure the last dimension of ngrams equals the processor's `ngram_len`.
2. Recreate the processor with the correct `ngram_len` for your ngram tensor.

When it happens

Trigger: Raised in an n-gram repetition penalty processor when the ngrams tensor's last dimension does not equal the configured ngram_len.

Common situations: Supplying n-grams of a different length than the processor's ngram_len, e.g. trigrams to a bigram processor.

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/81cdde48e83da11c. Report an issue: GitHub.