p-e-w/heretic · error · ValueError

The "split" field is required for datasets: {path}

Error message

The "split" field is required for datasets: {path}

What it means

When loading prompts from a dataset source (as opposed to an inline source), the configuration must explicitly specify which split of the dataset to use. Datasets can have many splits and the library refuses to guess, so load_prompts raises this ValueError when the `split` field is missing.

Source

Thrown at src/heretic/utils.py:189

) -> list[Prompt]:
    path = specification.dataset
    split_str = specification.split

    if os.path.isfile(path):
        # Plain text file with one prompt per line. Empty lines are ignored.
        with open(path, encoding="utf-8") as file:
            prompts = [line.strip() for line in file if line.strip()]

        # The split is optional for text files. When given, it selects a subset
        # of the lines using slice notation (e.g. "[:400]"). A synthetic split
        # name is prepended because ReadInstruction expects a named split.
        if split_str is not None:
            start, end = get_split_slice(f"_{split_str}", len(prompts))
            prompts = prompts[start:end]
    else:
        # All dataset sources require an explicit split and column.
        if split_str is None:
            raise ValueError(f'The "split" field is required for datasets: {path}')

        if specification.column is None:
            raise ValueError(f'The "column" field is required for datasets: {path}')

        if is_hf_path(path):
            # Pin to the latest commit if not already set, so the exact dataset
            # version is recorded for reproducibility.
            if specification.commit is None:
                try:
                    specification.commit = huggingface_hub.dataset_info(path).sha
                except Exception as error:
                    # Fetching the commit hash requires internet access, but the
                    # dataset itself may be fully cached locally. Proceed without
                    # pinning; an unpinned dataset disables the reproducibility
                    # offer during upload.
                    print(
                        f"[yellow]Warning: Could not fetch the latest commit hash for dataset [bold]{path}[/] ({error}). "
                        "The dataset version will not be pinned.[/]"

View on GitHub (pinned to bedb94ef11)

Solutions

1. Add a `split` field to the dataset specification in your config (e.g. "train").
2. If the data is a small inline set, use an inline prompts source instead of a dataset path.
3. Verify the chosen split name exists in the dataset (e.g. train/validation/test).

Example fix

// before
{"dataset": {"path": "openai/gsm8k"}}
// after
{"dataset": {"path": "openai/gsm8k", "split": "test", "column": "question"}}

cfg = spec["dataset"]
assert cfg.get("split"), 'dataset config must include a "split" field'

def dataset_config_is_complete(cfg: dict) -> bool:
    return bool(cfg.get("split")) and bool(cfg.get("column"))

try:
    prompts = load_prompts(specification)
except ValueError as e:
    if '"split" field is required' in str(e):
        print(f"Add split to dataset config for {specification.path}")
    else:
        raise

When it happens

Trigger: Configuring a prompt load whose path resolves to a dataset (HuggingFace or file dataset) without a `split` field in the specification.

Common situations: Copy-pasting an inline-prompts config that needed no split and pointing it at a HF dataset; assuming "train" is the default split.

Related errors

• The "column" field is required for datasets: {path}
• Plugin '{name}' does not export a class named '{class_name}'
• You must append the plugin class name to the filepath like t
• File-based plugin must use the form 'path/to/plugin.py:Class
• Import-based plugin must use the form 'fully.qualified.modul

AI-assisted analysis of p-e-w/heretic@bedb94ef11 (2026-08-29). Data as JSON: /api/errors/04e48f361c1567af. Report an issue: GitHub.