What causes "Error with input length {len(input_ids)} vs {batch_length}" in huggingface/transformers?

Enable truncation when tokenizing so inputs are cut to max_length (e.g. tokenizer(..., truncation=True, max_length=max_seq_length)).

huggingface/transformers · error · ValueError

Error with input length {len(input_ids)} vs {batch_length}

Error message

Error with input length {len(input_ids)} vs {batch_length}

What it means

While padding tokenized inputs to batch_length (the max sequence length in the batch or max_length), the featurizer computes padding_length = batch_length - len(input_ids) and pads. If input_ids is LONGER than batch_length, padding_length is negative, the 'padding' shrinks nothing (slicing a negative count of pad tokens appends/removes nothing with * negative -> empty list), and the final length check fires. In practice this means an input exceeded max_seq_length and no truncation strategy was applied.

Source

Thrown at src/transformers/data/processors/utils.py:292

        features = []
        for ex_index, (input_ids, example) in enumerate(zip(all_input_ids, self.examples)):
            if ex_index % 10000 == 0:
                logger.info(f"Writing example {ex_index}/{len(self.examples)}")
            # The mask has 1 for real tokens and 0 for padding tokens. Only real
            # tokens are attended to.
            attention_mask = [1 if mask_padding_with_zero else 0] * len(input_ids)

            # Zero-pad up to the sequence length.
            padding_length = batch_length - len(input_ids)
            if pad_on_left:
                input_ids = ([pad_token] * padding_length) + input_ids
                attention_mask = ([0 if mask_padding_with_zero else 1] * padding_length) + attention_mask
            else:
                input_ids = input_ids + ([pad_token] * padding_length)
                attention_mask = attention_mask + ([0 if mask_padding_with_zero else 1] * padding_length)

            if len(input_ids) != batch_length:
                raise ValueError(f"Error with input length {len(input_ids)} vs {batch_length}")
            if len(attention_mask) != batch_length:
                raise ValueError(f"Error with input length {len(attention_mask)} vs {batch_length}")

            if self.mode == "classification":
                label = label_map[example.label]
            elif self.mode == "regression":
                label = float(example.label)
            else:
                raise ValueError(self.mode)

            if ex_index < 5 and self.verbose:
                logger.info("*** Example ***")
                logger.info(f"guid: {example.guid}")
                logger.info(f"input_ids: {' '.join([str(x) for x in input_ids])}")
                logger.info(f"attention_mask: {' '.join([str(x) for x in attention_mask])}")
                logger.info(f"label: {example.label} (id = {label})")

            features.append(InputFeatures(input_ids=input_ids, attention_mask=attention_mask, label=label))

View on GitHub (pinned to a597f97485)

Solutions

Enable truncation when tokenizing so inputs are cut to max_length (e.g. tokenizer(..., truncation=True, max_length=max_seq_length)).
Raise max_seq_length / batch_length above the longest sequence in your data (check max(len(tokenizer.encode(t)) for t in texts)).
Pre-filter or split over-long examples before featurization.

Example fix

# before
features = featurizer.get_features(texts, max_length=128)  # some texts longer

# after
features = featurizer.get_features(texts, max_length=128, truncation=True)
# or: raise the limit after measuring the longest sequence

Defensive patterns

Strategy: validation

Validate before calling

max_len = max(len(tokenizer.encode(t)) for t in texts)
if max_len > max_seq_length:
    raise ValueError(f"Longest example {max_len} > max_seq_length {max_seq_length}; enable truncation or raise the limit")

Prevention

Always tokenize with truncation=True and an explicit max_length.
Measure the longest tokenized example before choosing max_seq_length.
Log distribution of token lengths during preprocessing to catch outliers.

When it happens

Trigger: Tokenizing examples longer than max_length with no truncation (the legacy featurizer's tokenizer settings do not truncate); setting max_length smaller than some sequences in the batch; enabling pad_on_left with an over-long sequence (same arithmetic).

Common situations: Reusing a max_seq_length tuned for short texts (e.g. 128) on longer documents; a few outlier documents in the dataset; switching a tokenizer/pipeline that previously truncated automatically to the low-level featurizer API which does not.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/798505a1526ac2be. Report an issue: GitHub.