huggingface/transformers · error · ValueError

Text and ids have mismatched lengths

Error message

Text and ids have mismatched lengths {len(texts_or_text_and_labels)} and {len(ids)}

What it means

SeqClassificationFeaturizer.add_examples requires that an explicit ids list (per-example GUIDs) match the length of texts_or_text_and_labels exactly. As with the labels check, this guards against zip() silently dropping unpaired examples, which would make dataset indices and GUIDs inconsistent.

Solutions

Ensure ids is built from the same list comprehension/loop as texts (e.g. ids = [f'train-{i}' for i in range(len(texts)]).
Pass ids=None and let the featurizer auto-generate GUIDs if you do not need stable ids.
Add an assert len(texts) == len(ids) guard in your data-loading code before calling the API.

Example fix

# before
featurizer.add_examples(texts, labels=labels, ids=old_ids)  # stale ids

# after
ids = [f"train-{i}" for i in range(len(texts))]
featurizer.add_examples(texts, labels=labels, ids=ids)

Defensive patterns

Strategy: validation

Validate before calling

if ids is not None:
    assert len(texts) == len(ids), f"texts={len(texts)} ids={len(ids)}"
featurizer.add_examples(texts, labels=labels, ids=ids)

Prevention

Generate ids from the same enumeration that builds texts.
Pass ids=None unless stable GUIDs are required; the API will synthesize them.
Never reuse ids from a previous preprocessing run with new data.

When it happens

Trigger: Calling add_examples(texts, labels=labels, ids=ids) where len(ids) != len(texts); reusing GUIDs generated for a previous, larger or smaller, batch of texts; passing ids=[...] while texts_or_text_and_labels is a list of (text, label) tuples.

Common situations: Precomputed ids from a prior preprocessing run applied to new data; slicing or subsampling texts without applying the same slice to ids; ids built with enumerate over a filtered list while texts came from the unfiltered list.

Related errors

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/529a0d5f43e96b29. Report an issue: GitHub.

Appendix: source

Thrown at src/transformers/data/processors/utils.py:201

            if column_id is not None:
                ids.append(line[column_id])
            else:
                guid = f"{split_name}-{i}" if split_name else str(i)
                ids.append(guid)

        return self.add_examples(
            texts, labels, ids, overwrite_labels=overwrite_labels, overwrite_examples=overwrite_examples
        )

    def add_examples(
        self, texts_or_text_and_labels, labels=None, ids=None, overwrite_labels=False, overwrite_examples=False
    ):
        if labels is not None and len(texts_or_text_and_labels) != len(labels):
            raise ValueError(
                f"Text and labels have mismatched lengths {len(texts_or_text_and_labels)} and {len(labels)}"
            )
        if ids is not None and len(texts_or_text_and_labels) != len(ids):
            raise ValueError(f"Text and ids have mismatched lengths {len(texts_or_text_and_labels)} and {len(ids)}")
        if ids is None:
            ids = [None] * len(texts_or_text_and_labels)
        if labels is None:
            labels = [None] * len(texts_or_text_and_labels)
        examples = []
        added_labels = set()
        for text_or_text_and_label, label, guid in zip(texts_or_text_and_labels, labels, ids):
            if isinstance(text_or_text_and_label, (tuple, list)) and label is None:
                text, label = text_or_text_and_label
            else:
                text = text_or_text_and_label
            added_labels.add(label)
            examples.append(InputExample(guid=guid, text_a=text, text_b=None, label=label))

        # Update examples
        if overwrite_examples:
            self.examples = examples
        else:

View on GitHub (pinned to a597f97485)