headroomlabs-ai/headroom · error · ValueError

Failed to load CodeSearchNet for '{language}': {e}

Error message

Failed to load CodeSearchNet for '{language}': {e}

What it means

Error "Failed to load CodeSearchNet for '{language}': {e}" thrown in headroomlabs-ai/headroom.

Source

Thrown at headroom/evals/datasets.py:767

    Dataset: https://huggingface.co/datasets/code_search_net

    Languages: python, java, javascript, go, ruby, php

    Args:
        n: Number of samples to load
        language: Programming language
        split: Dataset split

    Returns:
        EvalSuite with CodeSearchNet cases
    """
    _check_datasets_installed()
    from datasets import load_dataset

    try:
        ds = load_dataset("code_search_net", language, split=split)
    except Exception as e:
        raise ValueError(f"Failed to load CodeSearchNet for '{language}': {e}") from e

    cases: list[EvalCase] = []
    for i, item in enumerate(ds):
        if len(cases) >= n:
            break

        code = item.get("func_code_string", "") or item.get("whole_func_string", "")
        if not code:
            continue

        # Use docstring as query (find code from description)
        docstring = item.get("func_documentation_string", "")
        if not docstring:
            continue

        cases.append(
            EvalCase(
                id=f"codesearchnet_{language}_{i}",

View on GitHub (pinned to 322425c43b)

Solutions

1. Verify the language '{language}' is one supported by CodeSearchNet (go, java, javascript, php, python, ruby)
2. Install the datasets package: pip install headroom-ai[evals]
3. Check network connectivity to the HuggingFace Hub and retry
4. Inspect the underlying error {e} for details

When it happens

Trigger: Raised when loading the CodeSearchNet dataset for a given programming language fails, due to an unsupported language, missing `datasets` dependency, or network/Hub errors.

Common situations: See trigger scenarios.

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/b218a691360a6122. Report an issue: GitHub.