{"record":{"id":"259f56a9c4c601d7","repo":"headroomlabs-ai/headroom","slug":"huggingface-datasets-required-for-this-loader-ins","errorCode":null,"errorMessage":"HuggingFace datasets required for this loader. Install with: pip install headroom-ai[evals]","messagePattern":"HuggingFace datasets required for this loader\\. Install with: pip install headroom-ai\\[evals\\]","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"headroom/evals/datasets.py","lineNumber":42,"sourceCode":"Custom:\n- Tool output samples: Built-in realistic tool outputs\n\"\"\"\n\nfrom __future__ import annotations\n\nimport json\nfrom pathlib import Path\nfrom typing import Any\n\nfrom headroom.evals.core import EvalCase, EvalSuite\n\n\ndef _check_datasets_installed() -> None:\n    \"\"\"Check if HuggingFace datasets is installed.\"\"\"\n    try:\n        import datasets  # noqa: F401\n    except ImportError as e:\n        raise ImportError(\n            \"HuggingFace datasets required for this loader. \"\n            \"Install with: pip install headroom-ai[evals]\"\n        ) from e\n\n\n# =============================================================================\n# RAG / RETRIEVAL DATASETS\n# =============================================================================\n\n\ndef load_hotpotqa(\n    n: int = 100,\n    split: str = \"validation\",\n) -> EvalSuite:\n    \"\"\"Load HotpotQA dataset for multi-hop QA evaluation.\n\n    HotpotQA contains questions requiring reasoning over multiple\n    Wikipedia passages, with verified ground truth answers.","sourceCodeStart":24,"sourceCodeEnd":60,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/evals/datasets.py#L24-L60","documentation":"Raised by _check_datasets_installed() in headroom.evals.datasets: the HuggingFace `datasets` library is an optional dependency (shipped under the 'evals' extra), and every loader in this module (e.g. load_hotpotqa) calls the guard before importing it. The ImportError is chained from the original failure and names the exact extra to install: headroom-ai[evals].","triggerScenarios":"Calling any dataset loader (load_hotpotqa and siblings) with a plain `pip install headroom-ai` installation that omitted the [evals] extra, so `import datasets` fails. The guard fires before any network or parsing work.","commonSituations":"Minimal installs in Docker/CI for the proxy-only use case, then later running evals; stale environments from before the extra existed (datasets was perhaps a hard dep); interpreter mismatch where datasets lives in another venv.","solutions":["Install the extra in the same interpreter: `pip install 'headroom-ai[evals]'`","Or install the dependency directly: `pip install datasets`","Verify with `python -c \"import datasets\"` in the environment that runs the evals; fix venv mismatch if needed"],"exampleFix":"# before\nfrom headroom.evals.datasets import load_hotpotqa\ncases = load_hotpotqa(n=100)  # ImportError: HuggingFace datasets required...\n\n# after\n# pip install 'headroom-ai[evals]'\nfrom headroom.evals.datasets import load_hotpotqa\ncases = load_hotpotqa(n=100)","handlingStrategy":"validation","validationCode":"try:\n    import datasets  # noqa: F401\nexcept ImportError:\n    raise SystemExit(\"Eval datasets need the extra: pip install 'headroom-ai[evals]'\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Install with the [evals] extra on machines that run dataset loaders","Add `python -c 'import datasets'` to CI for eval pipelines","Keep eval dependencies in the same interpreter as headroom to avoid venv mismatches"],"tags":["dependency","import","evals","datasets","huggingface","extras"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}