huggingface/transformers · error · ValueError

Text and ids have mismatched lengths {len(texts_or_text_and_

Error message

Text and ids have mismatched lengths {len(texts_or_text_and_labels)} and {len(ids)}

What it means

SeqClassificationFeaturizer.add_examples requires that an explicit ids list (per-example GUIDs) match the length of texts_or_text_and_labels exactly. As with the labels check, this guards against zip() silently dropping unpaired examples, which would make dataset indices and GUIDs inconsistent.

Source

Thrown at src/transformers/data/processors/utils.py:201

            if column_id is not None:
                ids.append(line[column_id])
            else:
                guid = f"{split_name}-{i}" if split_name else str(i)
                ids.append(guid)

        return self.add_examples(
            texts, labels, ids, overwrite_labels=overwrite_labels, overwrite_examples=overwrite_examples
        )

    def add_examples(
        self, texts_or_text_and_labels, labels=None, ids=None, overwrite_labels=False, overwrite_examples=False
    ):
        if labels is not None and len(texts_or_text_and_labels) != len(labels):
            raise ValueError(
                f"Text and labels have mismatched lengths {len(texts_or_text_and_labels)} and {len(labels)}"
            )
        if ids is not None and len(texts_or_text_and_labels) != len(ids):
            raise ValueError(f"Text and ids have mismatched lengths {len(texts_or_text_and_labels)} and {len(ids)}")
        if ids is None:
            ids = [None] * len(texts_or_text_and_labels)
        if labels is None:
            labels = [None] * len(texts_or_text_and_labels)
        examples = []
        added_labels = set()
        for text_or_text_and_label, label, guid in zip(texts_or_text_and_labels, labels, ids):
            if isinstance(text_or_text_and_label, (tuple, list)) and label is None:
                text, label = text_or_text_and_label
            else:
                text = text_or_text_and_label
            added_labels.add(label)
            examples.append(InputExample(guid=guid, text_a=text, text_b=None, label=label))

        # Update examples
        if overwrite_examples:
            self.examples = examples
        else:

View on GitHub (pinned to a597f97485)

Solutions

Ensure ids is built from the same list comprehension/loop as texts (e.g. ids = [f'train-{i}' for i in range(len(texts)]).
Pass ids=None and let the featurizer auto-generate GUIDs if you do not need stable ids.
Add an assert len(texts) == len(ids) guard in your data-loading code before calling the API.

Example fix

# before
featurizer.add_examples(texts, labels=labels, ids=old_ids)  # stale ids

# after
ids = [f"train-{i}" for i in range(len(texts))]
featurizer.add_examples(texts, labels=labels, ids=ids)

Defensive patterns

Strategy: validation

Validate before calling

if ids is not None:
    assert len(texts) == len(ids), f"texts={len(texts)} ids={len(ids)}"
featurizer.add_examples(texts, labels=labels, ids=ids)

Prevention

Generate ids from the same enumeration that builds texts.
Pass ids=None unless stable GUIDs are required; the API will synthesize them.
Never reuse ids from a previous preprocessing run with new data.

When it happens

Trigger: Calling add_examples(texts, labels=labels, ids=ids) where len(ids) != len(texts); reusing GUIDs generated for a previous, larger or smaller, batch of texts; passing ids=[...] while texts_or_text_and_labels is a list of (text, label) tuples.

Common situations: Precomputed ids from a prior preprocessing run applied to new data; slicing or subsampling texts without applying the same slice to ids; ids built with enumerate over a filtered list while texts came from the unfiltered list.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/529a0d5f43e96b29. Report an issue: GitHub.