NousResearch/hermes-agent · error · FileNotFoundError

Dataset file not found: {self.dataset_file}

Error message

Dataset file not found: {self.dataset_file}

What it means

FileNotFoundError from BatchRunner._load_dataset() (batch_runner.py:652): the JSONL dataset path passed to the runner does not exist on disk, so nothing can be loaded. Raised at dataset-load time, before any batches are created.

Source

Thrown at batch_runner.py:652

        print(f"   Batch size: {self.batch_size}")
        print(f"   Total batches: {len(self.batches)}")
        print(f"   Run name: {self.run_name}")
        print(f"   Distribution: {self.distribution}")
        print(f"   Output directory: {self.output_dir}")
        print(f"   Workers: {self.num_workers}")
        if self.ephemeral_system_prompt:
            prompt_preview = self.ephemeral_system_prompt[:60] + "..." if len(self.ephemeral_system_prompt) > 60 else self.ephemeral_system_prompt
            print(f"   🔒 Ephemeral system prompt: '{prompt_preview}'")
    
    def _load_dataset(self) -> List[Dict[str, Any]]:
        """
        Load dataset from JSONL file.
        
        Returns:
            List[Dict]: List of dataset entries
        """
        if not self.dataset_file.exists():
            raise FileNotFoundError(f"Dataset file not found: {self.dataset_file}")
        
        dataset = []
        with open(self.dataset_file, 'r', encoding='utf-8') as f:
            for line_num, line in enumerate(f, 1):
                line = line.strip()
                if not line:
                    continue
                
                try:
                    entry = json.loads(line)
                    if 'prompt' not in entry:
                        print(f"⚠️  Warning: Line {line_num} missing 'prompt' field, skipping")
                        continue
                    dataset.append(entry)
                except json.JSONDecodeError as e:
                    print(f"⚠️  Warning: Invalid JSON on line {line_num}: {e}")
                    continue
        

View on GitHub (pinned to c896c09c42)

Solutions

1. Check the path exists from the same cwd the runner uses: `ls -l <dataset_file>` — resolve to an absolute path if cwd varies.
2. If a preprocessing step produces the dataset, verify it ran and wrote output before BatchRunner starts.
3. Fix the path in the CLI invocation/config (absolute paths avoid cwd sensitivity in scheduled contexts).

Example fix

# before
runner = BatchRunner(run_name="r1", dataset_file="data/prompts.jsonl", ...)
# run from another cwd -> FileNotFoundError

# after
from pathlib import Path
dataset = Path("data/prompts.jsonl").resolve()
if not dataset.exists():
    raise SystemExit(f"missing dataset: {dataset}")
runner = BatchRunner(run_name="r1", dataset_file=str(dataset), ...)

from pathlib import Path

dataset = Path(dataset_file).expanduser().resolve()
if not dataset.is_file():
    raise SystemExit(f"dataset not found: {dataset}")

try:
    runner = BatchRunner(run_name=r, dataset_file=dataset_file, ...)
except FileNotFoundError as exc:
    raise SystemExit(f"check dataset path/cwd: {exc}") from exc

When it happens

Trigger: Passing a relative dataset path while running from a different working directory; typo in the filename; the file lives on another machine/share that is not mounted; the generating step upstream failed so the file was never written.

Common situations: Cron/CI jobs whose cwd differs from the dev shell; run_name/output paths that imply the dataset should exist but the preprocessing step was skipped; moved or renamed datasets after a refactor.

Related errors

• Remote path must not be empty.
• Remote path must be absolute or start with ~/: "${s}"
• Unsafe SSH key path: contains control characters.
• Unknown distribution: {distribution}. Available: {list(list_
• No valid entries found in dataset file: {self.dataset_file}

AI-assisted analysis of NousResearch/hermes-agent@c896c09c42 (2026-08-14). Data as JSON: /api/errors/e86bd1ed4133a431. Report an issue: GitHub.