huggingface/transformers · error · KeyError

mode is not a valid split name

Error message

mode is not a valid split name

What it means

Raised by the deprecated SquadDataset constructor when the mode string is not a key of its Split enum (train/dev). Like GlueDataset, it resolves strings via Split[mode] and raises this bare KeyError for anything else, including the widely used 'validation'/'test' names. The class is legacy and intended to be replaced by the datasets library.

Source

Thrown at src/transformers/data/datasets/squad.py:131

    def __init__(
        self,
        args: SquadDataTrainingArguments,
        tokenizer: PreTrainedTokenizer,
        limit_length: int | None = None,
        mode: str | Split = Split.train,
        is_language_sensitive: bool = False,
        cache_dir: str | None = None,
        dataset_format: str = "pt",
    ):
        self.args = args
        self.is_language_sensitive = is_language_sensitive
        self.processor = SquadV2Processor() if args.version_2_with_negative else SquadV1Processor()
        if isinstance(mode, str):
            try:
                mode = Split[mode]
            except KeyError:
                raise KeyError("mode is not a valid split name")
        self.mode = mode
        # Load data features from cache or dataset file
        version_tag = "v2" if args.version_2_with_negative else "v1"
        cached_features_file = os.path.join(
            cache_dir if cache_dir is not None else args.data_dir,
            f"cached_{mode.value}_{tokenizer.__class__.__name__}_{args.max_seq_length}_{version_tag}",
        )

        # Make sure only the first process in distributed training processes the dataset,
        # and the others will use the cache.
        lock_path = cached_features_file + ".lock"
        with FileLock(lock_path):
            if os.path.exists(cached_features_file) and not args.overwrite_cache:
                start = time.time()
                check_torch_load_is_safe()
                self.old_features = torch.load(cached_features_file, weights_only=True)

                # Legacy cache files have only features, while new cache files

View on GitHub (pinned to a597f97485)

Solutions

1. Use mode='dev' for evaluation and mode='train' for training.
2. Pass the enum member directly (Split.dev / Split.train) to bypass string lookup.
3. Migrate to datasets.load_dataset('squad_v2' or 'squad') and the processors in transformers.data.processors.squad for feature conversion.

Example fix

# before
dataset = SquadDataset(args, tokenizer=tok, mode='validation')

# after
dataset = SquadDataset(args, tokenizer=tok, mode='dev')

from transformers.data.datasets.squad import Split

if isinstance(mode, str):
    mode = {'validation': 'dev'}.get(mode, mode)
    assert mode in Split.__members__, f'mode must be train/dev, got {mode!r}'
    mode = Split[mode]

def is_squad_split(mode) -> bool:
    from transformers.data.datasets.squad import Split
    return mode in Split.__members__

When it happens

Trigger: SquadDataset(args, tokenizer=tokenizer, mode='test') or mode='validation'; only 'train' and 'dev' are valid string names.

Common situations: Adapting legacy run_squad.py pipelines; using split names from the HF datasets hub ('validation') against this older API; case or whitespace mismatches in the mode string.

Related errors

• mode is not a valid split name
• The `layer_types` entries must be in {ALLOWED_LAYER_TYPES}
• `num_hidden_layers` ({num_hidden_layers}) must be equal to t
• Predictions and labels have mismatched lengths {len(preds)}
• No valid predictions

AI-assisted analysis of huggingface/transformers@a597f97485 (2026-08-14). Data as JSON: /api/errors/d953c9327848e815. Report an issue: GitHub.