huggingface/transformers · error · KeyError

mode is not a valid split name

Error message

mode is not a valid split name

What it means

Raised by the deprecated GlueDataset constructor when the mode argument is a string that is not a member of the Split enum (train/dev). The constructor looks the string up in Split[mode]; a miss raises this bare KeyError with no echo of the bad value. The whole class also emits a FutureWarning because it is scheduled for removal in favor of the datasets library.

Source

Thrown at src/transformers/data/datasets/glue.py:96

        tokenizer: PreTrainedTokenizerBase,
        limit_length: int | None = None,
        mode: str | Split = Split.train,
        cache_dir: str | None = None,
    ):
        warnings.warn(
            "This dataset will be removed from the library soon, preprocessing should be handled with the Hugging Face Datasets "
            "library. You can have a look at this example script for pointers: "
            "https://github.com/huggingface/transformers/blob/main/examples/pytorch/text-classification/run_glue.py",
            FutureWarning,
        )
        self.args = args
        self.processor = glue_processors[args.task_name]()
        self.output_mode = glue_output_modes[args.task_name]
        if isinstance(mode, str):
            try:
                mode = Split[mode]
            except KeyError:
                raise KeyError("mode is not a valid split name")
        # Load data features from cache or dataset file
        cached_features_file = os.path.join(
            cache_dir if cache_dir is not None else args.data_dir,
            f"cached_{mode.value}_{tokenizer.__class__.__name__}_{args.max_seq_length}_{args.task_name}",
        )
        label_list = self.processor.get_labels()
        if args.task_name in ["mnli", "mnli-mm"] and tokenizer.__class__.__name__ in (
            "RobertaTokenizer",
            "XLMRobertaTokenizer",
            "BartTokenizer",
            "BartTokenizerFast",
        ):
            # HACK(label indices are swapped in RoBERTa pretrained model)
            label_list[1], label_list[2] = label_list[2], label_list[1]
        self.label_list = label_list

        # Make sure only the first process in distributed training processes the dataset,
        # and the others will use the cache.

View on GitHub (pinned to a597f97485)

Solutions

1. Use mode='dev' for evaluation data (or the Split.dev enum member) and mode='train' for training.
2. Pass the enum directly: from transformers.data.datasets.glue import Split; mode=Split.dev.
3. Better: migrate off GlueDataset entirely and use the datasets library with load_dataset('glue', task), as the FutureWarning advises.

Example fix

# before
dataset = GlueDataset(args, tokenizer=tok, mode='validation')

# after
dataset = GlueDataset(args, tokenizer=tok, mode='dev')
# or modern replacement
from datasets import load_dataset
ds = load_dataset('glue', args.task_name, split='validation')

from transformers.data.datasets.glue import Split

VALID = {s.name for s in Split}
mode = mode if mode in VALID else {'validation': 'dev', 'test': 'dev'}.get(mode, mode)
assert mode in VALID, f'mode must be one of {sorted(VALID)}, got {mode!r}'

def is_glue_split(mode) -> bool:
    from transformers.data.datasets.glue import Split
    return mode in Split.__members__

try:
    ds = GlueDataset(args, tokenizer=tok, mode=mode)
except KeyError:
    ds = GlueDataset(args, tokenizer=tok, mode={'validation': 'dev', 'train': 'train'}.get(mode, 'dev'))

When it happens

Trigger: GlueDataset(args, tokenizer=tokenizer, mode='validation') or mode='test' — the enum only contains train and dev, so common split names from the datasets ecosystem fail.

Common situations: Migrating old run_glue.py-style scripts and passing modern split names ('validation', 'test') instead of 'dev'; typos like 'Train' (case-sensitive lookup).

Related errors

• mode is not a valid split name
• The `layer_types` entries must be in {ALLOWED_LAYER_TYPES}
• `num_hidden_layers` ({num_hidden_layers}) must be equal to t
• Predictions and labels have mismatched lengths {len(preds)}

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/bdfbb471ea081d06. Report an issue: GitHub.