What causes "'datasets' must be a list of dataset configurations" in huggingface/open-r1?

Make 'datasets' a list of dicts: {"datasets": [{"id": "org/name", "split": "train"}]}.

'datasets' must be a list of dataset configurations

Error message

'datasets' must be a list of dataset configurations

What it means

Inside dataset_mixture, the value under the 'datasets' key must be a list of per-dataset configuration dicts. If it is any other type (string, dict, number), __post_init__ raises this ValueError because it iterates the list and reads keys like id/config/split/columns/weight from each element.

Source

Thrown at src/open_r1/configs.py:104

                    "Expected format: {'datasets': [...], 'seed': int}"
                )

            datasets_list = []
            datasets_data = self.dataset_mixture.get("datasets", [])

            if isinstance(datasets_data, list):
                for dataset_config in datasets_data:
                    datasets_list.append(
                        DatasetConfig(
                            id=dataset_config.get("id"),
                            config=dataset_config.get("config"),
                            split=dataset_config.get("split", "train"),
                            columns=dataset_config.get("columns"),
                            weight=dataset_config.get("weight", 1.0),
                        )
                    )
            else:
                raise ValueError("'datasets' must be a list of dataset configurations")

            self.dataset_mixture = DatasetMixtureConfig(
                datasets=datasets_list,
                seed=self.dataset_mixture.get("seed", 0),
                test_split_size=self.dataset_mixture.get("test_split_size", None),
            )

            # Check that column names are consistent across all dataset configs
            columns_sets = [set(dataset.columns) for dataset in datasets_list if dataset.columns is not None]
            if columns_sets:
                first_columns = columns_sets[0]
                if not all(columns == first_columns for columns in columns_sets):
                    raise ValueError(
                        "Column names must be consistent across all dataset configurations in a mixture. "
                        f"Found different column sets: {[list(cols) for cols in columns_sets]}"
                    )

View on GitHub (pinned to 1416fa0cf2)

Solutions

Make 'datasets' a list of dicts: {"datasets": [{"id": "org/name", "split": "train"}]}.
If you only have one dataset, wrap it in a single-element list.
Validate the JSON/YAML structure parses to a list, not a string or object.

Example fix

// before
dataset_mixture={"datasets": {"id": "org/name"}}  # dict, not list
// after
dataset_mixture={"datasets": [{"id": "org/name", "config": "default", "split": "train", "weight": 1.0}]}

Defensive patterns

Strategy: type-guard

Validate before calling

ds = (mixture or {}).get("datasets")
if ds is not None and not (isinstance(ds, list) and all(isinstance(d, dict) for d in ds)):
    raise ValueError("'datasets' must be a list of dicts")

Type guard

def is_dataset_spec_list(v) -> bool:
    return isinstance(v, list) and all(isinstance(d, dict) and "id" in d for d in v)

Try / catch

try:
    cfg = DatasetConfig(dataset_mixture=mixture)
except ValueError as e:
    if "must be a list of dataset configurations" in str(e):
        sys.exit("Fix dataset_mixture.datasets: wrap each dataset spec in a list of dicts")
    raise

Prevention

Wrap single datasets in a list: [{"id": ...}]
Each spec needs at least an 'id' key; optional config/split/columns/weight
Validate JSON/YAML structure with a schema (e.g. jsonschema) before training

When it happens

Trigger: dataset_mixture={"datasets": "my_dataset"} or {"datasets": {"id": ...}} — 'datasets' present but not a list of dicts.

Common situations: Passing a single dataset id string instead of a list; nesting the mixture dict one level too deep; JSON configs where datasets was a mapping of name->config.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of huggingface/open-r1@1416fa0cf2 (2026-08-30). Data as JSON: /api/errors/80f1601c2f9901e4. Report an issue: GitHub.