huggingface/transformers · error · ValueError

Error with input length {len(attention_mask)} vs {batch_leng

Error message

Error with input length {len(attention_mask)} vs {batch_length}

What it means

The attention_mask is built as a list of the same length as input_ids and padded identically, so this length check fails exactly under the same conditions as the input_ids check: an input longer than batch_length makes padding_length negative and the padded attention_mask ends up not matching batch_length. Seeing this message means the example length exceeded max_seq_length without truncation.

Source

Thrown at src/transformers/data/processors/utils.py:294

            if ex_index % 10000 == 0:
                logger.info(f"Writing example {ex_index}/{len(self.examples)}")
            # The mask has 1 for real tokens and 0 for padding tokens. Only real
            # tokens are attended to.
            attention_mask = [1 if mask_padding_with_zero else 0] * len(input_ids)

            # Zero-pad up to the sequence length.
            padding_length = batch_length - len(input_ids)
            if pad_on_left:
                input_ids = ([pad_token] * padding_length) + input_ids
                attention_mask = ([0 if mask_padding_with_zero else 1] * padding_length) + attention_mask
            else:
                input_ids = input_ids + ([pad_token] * padding_length)
                attention_mask = attention_mask + ([0 if mask_padding_with_zero else 1] * padding_length)

            if len(input_ids) != batch_length:
                raise ValueError(f"Error with input length {len(input_ids)} vs {batch_length}")
            if len(attention_mask) != batch_length:
                raise ValueError(f"Error with input length {len(attention_mask)} vs {batch_length}")

            if self.mode == "classification":
                label = label_map[example.label]
            elif self.mode == "regression":
                label = float(example.label)
            else:
                raise ValueError(self.mode)

            if ex_index < 5 and self.verbose:
                logger.info("*** Example ***")
                logger.info(f"guid: {example.guid}")
                logger.info(f"input_ids: {' '.join([str(x) for x in input_ids])}")
                logger.info(f"attention_mask: {' '.join([str(x) for x in attention_mask])}")
                logger.info(f"label: {example.label} (id = {label})")

            features.append(InputFeatures(input_ids=input_ids, attention_mask=attention_mask, label=label))

        if return_tensors is None:

View on GitHub (pinned to a597f97485)

Solutions

1. Turn on truncation at tokenization time so len(input_ids) never exceeds max_length.
2. Increase max_seq_length to at least the longest tokenized example.
3. Measure token lengths first: max(map(len, tokenizer(list_of_texts)['input_ids'])) to pick a safe limit.

Example fix

# before
features = featurizer.get_features(texts, max_length=64)  # docs exceed 64

# after
features = featurizer.get_features(texts, max_length=64, truncation=True)

lengths = [len(ids) for ids in tokenizer(texts, add_special_tokens=True)["input_ids"]]
assert max(lengths) <= max_seq_length, f"max token length {max(lengths)} exceeds {max_seq_length}"

When it happens

Trigger: Same root cause as the input_ids check: max_length/batch_length smaller than a tokenized example and no truncation; mixing pad_on_left settings between runs while reusing cached lengths.

Common situations: Long outlier documents exceeding max_seq_length; lowering max_seq_length for memory reasons without adding truncation; datasets with mixed-length domains (tweets vs articles) sharing one limit.

Related errors

• Error with input length {len(input_ids)} vs {batch_length}
• You are attempting to pad samples but the tokenizer you are
• This collator requires that sequence lengths be even to crea
• You should supply an instance of `transformers.BatchFeature`
• Some items in the output dictionary have a different batch s

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/6e48340ac030544d. Report an issue: GitHub.