{"record":{"id":"e7d556ca8f567c6d","repo":"p-e-w/heretic","slug":"the-column-field-is-required-for-datasets-path","errorCode":null,"errorMessage":"The \"column\" field is required for datasets: {path}","messagePattern":"The \"column\" field is required for datasets: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"src/heretic/utils.py","lineNumber":192,"sourceCode":"\n    if os.path.isfile(path):\n        # Plain text file with one prompt per line. Empty lines are ignored.\n        with open(path, encoding=\"utf-8\") as file:\n            prompts = [line.strip() for line in file if line.strip()]\n\n        # The split is optional for text files. When given, it selects a subset\n        # of the lines using slice notation (e.g. \"[:400]\"). A synthetic split\n        # name is prepended because ReadInstruction expects a named split.\n        if split_str is not None:\n            start, end = get_split_slice(f\"_{split_str}\", len(prompts))\n            prompts = prompts[start:end]\n    else:\n        # All dataset sources require an explicit split and column.\n        if split_str is None:\n            raise ValueError(f'The \"split\" field is required for datasets: {path}')\n\n        if specification.column is None:\n            raise ValueError(f'The \"column\" field is required for datasets: {path}')\n\n        if is_hf_path(path):\n            # Pin to the latest commit if not already set, so the exact dataset\n            # version is recorded for reproducibility.\n            if specification.commit is None:\n                try:\n                    specification.commit = huggingface_hub.dataset_info(path).sha\n                except Exception as error:\n                    # Fetching the commit hash requires internet access, but the\n                    # dataset itself may be fully cached locally. Proceed without\n                    # pinning; an unpinned dataset disables the reproducibility\n                    # offer during upload.\n                    print(\n                        f\"[yellow]Warning: Could not fetch the latest commit hash for dataset [bold]{path}[/] ({error}). \"\n                        \"The dataset version will not be pinned.[/]\"\n                    )\n            dataset = load_dataset(\n                path,","sourceCodeStart":174,"sourceCodeEnd":210,"githubUrl":"https://github.com/p-e-w/heretic/blob/bedb94ef117a271532ac2058447fbc165d5051bd/src/heretic/utils.py#L174-L210","documentation":"Dataset sources must declare which column of the dataset contains the prompts. load_prompts raises this ValueError when the specification names a dataset but leaves `column` unset, because the library cannot infer which field holds the prompt text.","triggerScenarios":"Loading a dataset path in the prompt config without a `column` field, even when `split` is provided.","commonSituations":"Dataset schemas where the prompt column is named question/prompt/text/something custom; switching datasets without updating the column name.","solutions":["Add a `column` field naming the prompt column (e.g. \"question\").","Inspect the dataset's schema/features to confirm the exact column name.","Re-run after aligning column with the actual dataset fields."],"exampleFix":"// before\n{\"dataset\": {\"path\": \"openai/gsm8k\", \"split\": \"test\"}}\n// after\n{\"dataset\": {\"path\": \"openai/gsm8k\", \"split\": \"test\", \"column\": \"question\"}}","handlingStrategy":"validation","validationCode":"cfg = spec[\"dataset\"]\nassert cfg.get(\"column\"), 'dataset config must include a \"column\" field'","typeGuard":"def dataset_config_is_complete(cfg: dict) -> bool:\n    return bool(cfg.get(\"split\")) and bool(cfg.get(\"column\"))","tryCatchPattern":"try:\n    prompts = load_prompts(specification)\nexcept ValueError as e:\n    if '\"column\" field is required' in str(e):\n        print(f\"Add column to dataset config for {specification.path}\")\n    else:\n        raise","preventionTips":["Inspect dataset features to find the prompt column name.","Always include split and column together for datasets.","Validate config against a schema (e.g. pydantic) before calling load_prompts."],"tags":["python","configuration","datasets"],"backgroundTag":"missing-required-config-field","analyzedSha":"bedb94ef117a271532ac2058447fbc165d5051bd","analyzedAt":"2026-08-29T08:38:06.692Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}