{"record":{"id":"6aafb06c2550b734","repo":"NousResearch/hermes-agent","slug":"no-valid-entries-found-in-dataset-file-self-data","errorCode":null,"errorMessage":"No valid entries found in dataset file: {self.dataset_file}","messagePattern":"No valid entries found in dataset file: (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"batch_runner.py","lineNumber":672,"sourceCode":"        dataset = []\n        with open(self.dataset_file, 'r', encoding='utf-8') as f:\n            for line_num, line in enumerate(f, 1):\n                line = line.strip()\n                if not line:\n                    continue\n                \n                try:\n                    entry = json.loads(line)\n                    if 'prompt' not in entry:\n                        print(f\"⚠️  Warning: Line {line_num} missing 'prompt' field, skipping\")\n                        continue\n                    dataset.append(entry)\n                except json.JSONDecodeError as e:\n                    print(f\"⚠️  Warning: Invalid JSON on line {line_num}: {e}\")\n                    continue\n        \n        if not dataset:\n            raise ValueError(f\"No valid entries found in dataset file: {self.dataset_file}\")\n        \n        return dataset\n    \n    def _create_batches(self) -> List[List[Tuple[int, Dict[str, Any]]]]:\n        \"\"\"\n        Split dataset into batches with indices.\n        \n        Returns:\n            List of batches, where each batch is a list of (index, entry) tuples\n        \"\"\"\n        batches = []\n        for i in range(0, len(self.dataset), self.batch_size):\n            batch = [(idx, entry) for idx, entry in enumerate(self.dataset[i:i + self.batch_size], start=i)]\n            batches.append(batch)\n        \n        return batches\n    \n    def _load_checkpoint(self) -> Dict[str, Any]:","sourceCodeStart":654,"sourceCodeEnd":690,"githubUrl":"https://github.com/NousResearch/hermes-agent/blob/c896c09c42910c584c4c7d2325b58c14713ea42c/batch_runner.py#L654-L690","documentation":"ValueError from BatchRunner._load_dataset() (batch_runner.py:672): the JSONL file was found and read, but after skipping blank lines, lines with invalid JSON, and lines missing the 'prompt' field, zero usable entries remain. The per-line warnings printed just before tell you which lines were dropped and why.","triggerScenarios":"Every line fails one of the two filters: json.loads raises (trailing comma, single quotes, truncated line) or the parsed object lacks a 'prompt' key — e.g. the file is actually JSON (one array), CSV, or a JSONL of a different schema ({'input': ...} instead of {'prompt': ...}).","commonSituations":"Dataset exported from another tool with different field names; a pretty-printed single JSON object per multiple lines (each physical line is not valid JSON); BOM/encoding artifacts breaking the first line; accidentally pointing at the wrong file (statistics.json, checkpoint.json).","solutions":["Re-read the ⚠️ warnings above the exception — they enumerate 'Line N missing prompt' vs 'Invalid JSON on line N', which tells you the failure mode directly.","If fields are named differently, transform the dataset to {'prompt': ...} entries (jq or a one-liner) rather than editing batch_runner.","If the file is one JSON array, flatten it: jq -c '.[]' in.json > out.jsonl.","Validate the file before the run: parse every line and assert a 'prompt' key (see validationCode)."],"exampleFix":"# before — dataset uses 'input' instead of 'prompt'\n{\"input\": \"hello\", \"id\": 1}\n\n# after — flatten/renamed to the expected schema\n{\"prompt\": \"hello\", \"id\": 1}\n\n# one-off conversion:\n# jq -c '{prompt: .input, id: .id}' in.jsonl > fixed.jsonl","handlingStrategy":"validation","validationCode":"import json\nfrom pathlib import Path\n\ndef dataset_is_valid(path: str) -> bool:\n    ok = 0\n    for line in Path(path).read_text(encoding=\"utf-8\").splitlines():\n        line = line.strip()\n        if not line:\n            continue\n        try:\n            entry = json.loads(line)\n        except json.JSONDecodeError:\n            return False\n        if not isinstance(entry, dict) or \"prompt\" not in entry:\n            return False\n        ok += 1\n    return ok > 0","typeGuard":null,"tryCatchPattern":"try:\n    runner.load_dataset()\nexcept ValueError as exc:\n    # per-line warnings above the exception identify the bad lines\n    raise SystemExit(f\"dataset has no usable entries: {exc}\") from exc","preventionTips":["Pre-validate JSONL datasets with a lint step (every line: valid JSON + 'prompt' key).","Flatten single-JSON-array exports with jq -c '.[]' before use.","Write datasets with json.dumps per line, never pretty-printing."],"tags":["batch","dataset","jsonl","schema"],"backgroundTag":null,"analyzedSha":"c896c09c42910c584c4c7d2325b58c14713ea42c","analyzedAt":"2026-08-14T17:18:01.089Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}