aaif-goose/goose · error · FileNotFoundError

Output file not found: {output_path}

Error message

Output file not found: {output_path}

What it means

Python FileNotFoundError raised by load_output_file() when the model-output file the judge is supposed to grade does not exist under the working directory. The output_file name is joined onto working_dir, so both must be correct.

Source

Thrown at scripts/bench-postprocess-scripts/llm-judges/llm_judge.py:181

        print(f"Error evaluating with OpenAI: {str(e)}")
        raise ValueError(f"OpenAI evaluation failed: {str(e)}")


def load_eval_results(working_dir: Path) -> Dict[str, Any]:
    """Load the eval-results.json file from the working directory."""
    eval_results_path = working_dir / "eval-results.json"
    if not eval_results_path.exists():
        raise FileNotFoundError(f"eval-results.json not found in {working_dir}")
    
    with open(eval_results_path, 'r') as f:
        return json.load(f)


def load_output_file(working_dir: Path, output_file: str) -> str:
    """Load the output file to evaluate from the working directory."""
    output_path = working_dir / output_file
    if not output_path.exists():
        raise FileNotFoundError(f"Output file not found: {output_path}")
    
    with open(output_path, 'r') as f:
        return f.read().strip()


def load_evaluation_prompt(working_dir: Path) -> str:
    """Load the evaluation prompt from a file or use a default.
    
    This function looks for a prompt.txt file in the working directory.
    If not found, it returns a default evaluation prompt.
    """
    prompt_file = working_dir / "prompt.txt"
    if prompt_file.exists():
        with open(prompt_file, 'r') as f:
            return f.read().strip()
    
    # Default evaluation prompt
    return """You are an expert evaluator assessing the quality of AI responses.

View on GitHub (pinned to 3810898a74)

Solutions

1. Verify the exact path: ls <working_dir>/<output_file> and compare character-for-character with the value passed to the script
2. If the file is elsewhere, move/copy it into working_dir or fix the working_dir/output_file arguments
3. Re-run the generation stage that is supposed to produce the output file
4. Check the expected filename in the benchmark config you are post-processing

Example fix

# before
text = load_output_file(working_dir, "output.txt")

# after
out_path = working_dir / "output.txt"
if not out_path.exists():
    raise SystemExit(f"missing {out_path}; generate the output first")
text = load_output_file(working_dir, "output.txt")

from pathlib import Path
p = working_dir / output_file
if not p.is_file():
    raise SystemExit(f"missing output file {p}; generate model output first")

try:
    text = load_output_file(working_dir, output_file)
except FileNotFoundError as e:
    raise SystemExit(f"cannot judge: {e}")  # stop the stage, no fallback data

When it happens

Trigger: Calling the judge with an output_file name that is not present in working_dir — wrong filename (typo, different extension like .txt vs .jsonl), output written to a different directory, or the generation step never ran.

Common situations: Output filename convention changed between runs; artifacts in a nested per-model subdirectory while the script looks at the flat working dir; running the judge standalone without the preceding generation stage.

Related errors

• eval-results.json not found in {working_dir}
• --goose-binary does not exist or is not a file: {args.goose_
• OpenAI evaluation failed: {str(e)}
• Update file not found. Please download the update first.
• Job name must start with a letter or number and contain only

AI-assisted analysis of aaif-goose/goose@3810898a74 (2026-08-16). Data as JSON: /api/errors/612094dfcae8dfde. Report an issue: GitHub.