{"record":{"id":"6546383738fab485","repo":"google-gemini/gemini-cli","slug":"required-judge-md-prompt-file-missing-from-prompt","errorCode":null,"errorMessage":"Required judge.md prompt file missing from {PROMPT_FILE.parent}","messagePattern":"Required judge\\.md prompt file missing from (.+?)","errorType":"exception","errorClass":"FileNotFoundError","httpStatus":null,"severity":"critical","filePath":"tools/caretaker-agent/evals/triage/judge.py","lineNumber":21,"sourceCode":"\nProvides evaluation functions:\n1. evaluate_categorization: Exact match string evaluation for quality & effort.\n2. judge_workable_spec: LLM-as-a-Judge grading for Workable Specs matching Golden Spec fidelity (0-2 Rubric Scale) via Gemini API.\n\"\"\"\n\nimport os\nimport json\nfrom pathlib import Path\nfrom typing import Any, Dict\nfrom dotenv import load_dotenv\n\nload_dotenv()\n\nfrom google import genai\n\nPROMPT_FILE = Path(__file__).parent / \"judge.md\"\nif not PROMPT_FILE.exists():\n    raise FileNotFoundError(f\"Required judge.md prompt file missing from {PROMPT_FILE.parent}\")\n\nwith open(PROMPT_FILE, \"r\", encoding=\"utf-8\") as f:\n    JUDGE_PROMPT = f.read()\n\n_CLIENT: Any = None\n\n\ndef _get_client() -> genai.Client:\n    \"\"\"Returns thread-safe cached Gemini API client instance.\"\"\"\n    global _CLIENT\n    if _CLIENT is None:\n        api_key = os.environ.get(\"GEMINI_API_KEY\")\n        _CLIENT = genai.Client(api_key=api_key)\n    return _CLIENT\n\n\ndef evaluate_categorization(predicted: Dict[str, Any], expected: Dict[str, Any]) -> Dict[str, Any]:\n    \"\"\"","sourceCodeStart":3,"sourceCodeEnd":39,"githubUrl":"https://github.com/google-gemini/gemini-cli/blob/5024443c7217464a66e98f80d73172a26440bd8f/tools/caretaker-agent/evals/triage/judge.py#L3-L39","documentation":"This FileNotFoundError is raised at module import time in judge.py when the sibling file judge.md is not found next to judge.py. Because the check and file read happen at top-level (outside any function), importing judge.py itself triggers the error, which means any module importing evaluate_categorization or judge_workable_spec fails to load. The file holds the LLM-as-a-judge system prompt.","triggerScenarios":"Any import of evals.triage.judge (directly or via runner.py) when judge.md is missing from the same directory. The check runs once at import, so the exception propagates as an ImportError/ModuleNotFoundError wrapper or a raw FileNotFoundError depending on context.","commonSituations":"judge.md was deleted, renamed, or not included in a Docker image due to .dockerignore excluding .md files. The package was installed from a wheel/sdist that did not include non-Python data files (missing package_data or MANIFEST.in). Running from a shallow clone or worktree that lacks the file. Case mismatch on case-sensitive filesystems.","solutions":["Confirm judge.md exists at evals/triage/judge.md alongside judge.py.","If packaging, ensure .md files are included: add 'include *.md' to MANIFEST.in or package_data={'': ['*.md']} in setup.","Check .dockerignore does not exclude *.md from the build context.","On case-sensitive filesystems, verify exact casing 'judge.md'.","Run: ls evals/triage/judge.md to confirm presence before importing."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"from pathlib import Path\n\nPROMPT_FILE = Path(__file__).parent / 'judge.md'\n\ndef judge_prompt_available() -> bool:\n    return PROMPT_FILE.exists()\n\n# In CI: assert judge_prompt_available() before importing evals.triage.judge.","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Ensure package_data / MANIFEST.in includes *.md so judge.md ships in distributions.","Audit .dockerignore to confirm .md files enter the build context.","Add a CI assertion that judge.md exists in evals/triage/.","Keep prompt files under version control and reviewed with code changes."],"tags":["file-not-found","packaging","import-time","judge","prompt","python"],"backgroundTag":null,"analyzedSha":"5024443c7217464a66e98f80d73172a26440bd8f","analyzedAt":"2026-08-12T06:01:53.711Z","schemaVersion":2},"datasetVersion":"2026-08-12T13:17:24.610Z"}