huggingface/open-r1 · error

Column names must be consistent across all dataset configura

Error message

Column names must be consistent across all dataset configurations in a mixture. Found different column sets: {[list(cols) for cols in columns_sets]}

What it means

When building a dataset mixture, if individual dataset configs declare a 'columns' subset, all declared column sets must be identical across the mixture. This guarantees the concatenated/mixed dataset has a uniform schema; __post_init__ compares the sets and raises on mismatch.

Source

Thrown at src/open_r1/configs.py:117

                            columns=dataset_config.get("columns"),
                            weight=dataset_config.get("weight", 1.0),
                        )
                    )
            else:
                raise ValueError("'datasets' must be a list of dataset configurations")

            self.dataset_mixture = DatasetMixtureConfig(
                datasets=datasets_list,
                seed=self.dataset_mixture.get("seed", 0),
                test_split_size=self.dataset_mixture.get("test_split_size", None),
            )

            # Check that column names are consistent across all dataset configs
            columns_sets = [set(dataset.columns) for dataset in datasets_list if dataset.columns is not None]
            if columns_sets:
                first_columns = columns_sets[0]
                if not all(columns == first_columns for columns in columns_sets):
                    raise ValueError(
                        "Column names must be consistent across all dataset configurations in a mixture. "
                        f"Found different column sets: {[list(cols) for cols in columns_sets]}"
                    )


# TODO: add the shared options with a mixin to reduce code duplication
@dataclass
class GRPOConfig(trl.GRPOConfig):
    """
    args for callbacks, benchmarks etc
    """

    benchmarks: list[str] = field(
        default_factory=lambda: [],
        metadata={"help": "The benchmarks to run after training."},
    )
    callbacks: list[str] = field(
        default_factory=lambda: [],

View on GitHub (pinned to 1416fa0cf2)

Solutions

1. Make the 'columns' lists identical (same names) across every dataset entry in the mixture.
2. Remove the 'columns' key from all entries and instead normalize schemas in a dataset.map preprocessing step before training.
3. Align upstream datasets so the columns you select exist with the same names in each source.

Example fix

// before
{"datasets": [{"id": "a", "columns": ["prompt", "chosen"]}, {"id": "b", "columns": ["prompt", "completion"]}]}
// after
{"datasets": [{"id": "a", "columns": ["prompt", "completion"]}, {"id": "b", "columns": ["prompt", "completion"]}]}

cols = [set(d["columns"]) for d in mixture.get("datasets", []) if d.get("columns")]
if cols and any(c != cols[0] for c in cols[1:]):
    raise ValueError(f"Inconsistent mixture columns: {[sorted(c) for c in cols]}")

def columns_consistent(specs) -> bool:
    sets = [set(d["columns"]) for d in specs if d.get("columns") is not None]
    return len({frozenset(s) for s in sets}) <= 1

try:
    cfg = DatasetConfig(dataset_mixture=mixture)
except ValueError as e:
    if "Column names must be consistent" in str(e):
        for d in mixture["datasets"]:
            d.pop("columns", None)  # fall back to full-schema union
        cfg = DatasetConfig(dataset_mixture=mixture)
    else:
        raise

When it happens

Trigger: dataset_mixture where dataset A specifies columns=["prompt","completion"] and dataset B specifies columns=["prompt","chosen"] (or one declares columns and another declares different ones).

Common situations: Mixing chat and completion-formatted datasets with hand-picked columns; renaming a column in one dataset config but not the others; copy-paste between mixture entries with stale column lists.

Related errors

• dataset_mixture must be a dictionary with a 'datasets' key.
• 'datasets' must be a list of dataset configurations
• Either `dataset_name` or `dataset_mixture` must be provided
• CF_TESTS_FOLDER environment variable not set! Please downloa
• Dataset Question Field Error: {prompt_column} is not support

AI-assisted analysis of huggingface/open-r1@1416fa0cf2 (2026-08-30). Data as JSON: /api/errors/488155d47902eee2. Report an issue: GitHub.