{"record":{"id":"f375a692565ad972","repo":"headroomlabs-ai/headroom","slug":"failed-to-load-toolbench-category-category-e","errorCode":null,"errorMessage":"Failed to load ToolBench category '{category}': {e}","messagePattern":"Failed to load ToolBench category '(.+?)': (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"headroom/evals/datasets.py","lineNumber":684,"sourceCode":"    Categories:\n    - G1: Single-tool single-step\n    - G2: Single-tool multi-step\n    - G3: Multi-tool single-step\n\n    Args:\n        n: Number of samples to load\n        category: ToolBench category (G1, G2, G3)\n\n    Returns:\n        EvalSuite with ToolBench cases\n    \"\"\"\n    _check_datasets_installed()\n    from datasets import load_dataset\n\n    try:\n        ds = load_dataset(\"ToolBench/ToolBench\", category, split=\"test\")\n    except Exception as e:\n        raise ValueError(f\"Failed to load ToolBench category '{category}': {e}\") from e\n\n    cases: list[EvalCase] = []\n    for i, item in enumerate(ds):\n        if len(cases) >= n:\n            break\n\n        # Get tool definitions as context\n        tools = item.get(\"api_list\", [])\n        if not tools:\n            continue\n\n        # Format tools as JSON context\n        tool_defs = []\n        for tool in tools:\n            tool_defs.append(\n                {\n                    \"name\": tool.get(\"api_name\", \"\"),\n                    \"description\": tool.get(\"api_description\", \"\"),","sourceCodeStart":666,"sourceCodeEnd":702,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/evals/datasets.py#L666-L702","documentation":"Error \"Failed to load ToolBench category '{category}': {e}\" thrown in headroomlabs-ai/headroom.","triggerScenarios":"Raised when loading a ToolBench category fails, due to an unknown category name, missing local data files, or a network/parse failure.","commonSituations":"See trigger scenarios.","solutions":["Verify the ToolBench category '{category}' is spelled correctly and exists in the dataset","Check that the ToolBench data files are present/downloaded in the expected location","Check network connectivity if the loader fetches data remotely","Inspect the underlying error {e} for the specific failure (missing file, parse error, network)"],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}