{"record":{"id":"612094dfcae8dfde","repo":"aaif-goose/goose","slug":"output-file-not-found-output-path","errorCode":null,"errorMessage":"Output file not found: {output_path}","messagePattern":"Output file not found: (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"error","filePath":"scripts/bench-postprocess-scripts/llm-judges/llm_judge.py","lineNumber":181,"sourceCode":"        print(f\"Error evaluating with OpenAI: {str(e)}\")\n        raise ValueError(f\"OpenAI evaluation failed: {str(e)}\")\n\n\ndef load_eval_results(working_dir: Path) -> Dict[str, Any]:\n    \"\"\"Load the eval-results.json file from the working directory.\"\"\"\n    eval_results_path = working_dir / \"eval-results.json\"\n    if not eval_results_path.exists():\n        raise FileNotFoundError(f\"eval-results.json not found in {working_dir}\")\n    \n    with open(eval_results_path, 'r') as f:\n        return json.load(f)\n\n\ndef load_output_file(working_dir: Path, output_file: str) -> str:\n    \"\"\"Load the output file to evaluate from the working directory.\"\"\"\n    output_path = working_dir / output_file\n    if not output_path.exists():\n        raise FileNotFoundError(f\"Output file not found: {output_path}\")\n    \n    with open(output_path, 'r') as f:\n        return f.read().strip()\n\n\ndef load_evaluation_prompt(working_dir: Path) -> str:\n    \"\"\"Load the evaluation prompt from a file or use a default.\n    \n    This function looks for a prompt.txt file in the working directory.\n    If not found, it returns a default evaluation prompt.\n    \"\"\"\n    prompt_file = working_dir / \"prompt.txt\"\n    if prompt_file.exists():\n        with open(prompt_file, 'r') as f:\n            return f.read().strip()\n    \n    # Default evaluation prompt\n    return \"\"\"You are an expert evaluator assessing the quality of AI responses.","sourceCodeStart":163,"sourceCodeEnd":199,"githubUrl":"https://github.com/aaif-goose/goose/blob/3810898a7447ec3299be72e223d3570a7aabf0ab/scripts/bench-postprocess-scripts/llm-judges/llm_judge.py#L163-L199","documentation":"Python FileNotFoundError raised by load_output_file() when the model-output file the judge is supposed to grade does not exist under the working directory. The output_file name is joined onto working_dir, so both must be correct.","triggerScenarios":"Calling the judge with an output_file name that is not present in working_dir — wrong filename (typo, different extension like .txt vs .jsonl), output written to a different directory, or the generation step never ran.","commonSituations":"Output filename convention changed between runs; artifacts in a nested per-model subdirectory while the script looks at the flat working dir; running the judge standalone without the preceding generation stage.","solutions":["Verify the exact path: ls <working_dir>/<output_file> and compare character-for-character with the value passed to the script","If the file is elsewhere, move/copy it into working_dir or fix the working_dir/output_file arguments","Re-run the generation stage that is supposed to produce the output file","Check the expected filename in the benchmark config you are post-processing"],"exampleFix":"# before\ntext = load_output_file(working_dir, \"output.txt\")\n\n# after\nout_path = working_dir / \"output.txt\"\nif not out_path.exists():\n    raise SystemExit(f\"missing {out_path}; generate the output first\")\ntext = load_output_file(working_dir, \"output.txt\")","handlingStrategy":"validation","validationCode":"from pathlib import Path\np = working_dir / output_file\nif not p.is_file():\n    raise SystemExit(f\"missing output file {p}; generate model output first\")","typeGuard":null,"tryCatchPattern":"try:\n    text = load_output_file(working_dir, output_file)\nexcept FileNotFoundError as e:\n    raise SystemExit(f\"cannot judge: {e}\")  # stop the stage, no fallback data","preventionTips":["Derive output_file from the same config the generator used","Assert artifacts exist before invoking the judge","Keep generation and judging filenames in one shared constant"],"tags":["python","file-not-found","benchmark","filesystem"],"backgroundTag":null,"analyzedSha":"3810898a7447ec3299be72e223d3570a7aabf0ab","analyzedAt":"2026-08-16T10:14:26.282Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}