{"record":{"id":"0baffcad0e83eca0","repo":"aaif-goose/goose","slug":"openai-api-key-environment-variable-is-not-set-bu","errorCode":null,"errorMessage":"OPENAI_API_KEY environment variable is not set, but is needed to run this evaluation.","messagePattern":"OPENAI_API_KEY environment variable is not set, but is needed to run this evaluation\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"scripts/bench-postprocess-scripts/llm-judges/llm_judge.py","lineNumber":52,"sourceCode":"def evaluate_with_openai(prompt: str, text: str, rubric_max_score: int = 2) -> float:\n    \"\"\"Evaluate response using OpenAI's API.\n    \n    Args:\n        prompt: System prompt for evaluation\n        text: Text to evaluate\n        rubric_max_score: Maximum score for the rubric (default: 2.0)\n        \n    Returns:\n        float: Evaluation score (0 to rubric_max_score)\n        \n    Raises:\n        ValueError: If OPENAI_API_KEY environment variable is not set\n    \"\"\"\n    print(\"Starting OpenAI evaluation...\")\n    api_key = os.getenv(\"OPENAI_API_KEY\")\n    if not api_key:\n        print(\"No OpenAI API key found!\")\n        raise ValueError(\"OPENAI_API_KEY environment variable is not set, but is needed to run this evaluation.\")\n        \n    try:\n        client = OpenAI(api_key=api_key)\n        \n        # Append output instructions to system prompt\n        output_instructions = f\"\"\"\nOutput Instructions:\nReturn your evaluation as a JSON object in the following format:\n{{\n    \"reasoning\": \"Your brief reasoning for the score\",\n    \"score\": <integer between 0 and {rubric_max_score}>\n}}\n\nIMPORTANT: \n- Do not use any markdown formatting (no ```json blocks)\n- Do not include any additional text before or after the JSON\n- Return only the raw JSON object\n- The score must be an integer between 0 and {rubric_max_score}\"\"\"","sourceCodeStart":34,"sourceCodeEnd":70,"githubUrl":"https://github.com/aaif-goose/goose/blob/3810898a7447ec3299be72e223d3570a7aabf0ab/scripts/bench-postprocess-scripts/llm-judges/llm_judge.py#L34-L70","documentation":"Raised by evaluate_with_openai in the benchmark LLM-judge script before any API call is made. The judge instantiates an OpenAI client to score responses against a rubric, and it reads the key exclusively from the environment via os.getenv('OPENAI_API_KEY'); an unset (or empty) variable aborts with this message. It is a fail-fast guard so runs do not die mid-batch on the first request.","triggerScenarios":"Running the judge in a fresh shell or CI container without exporting the key; the key present in a .env file this script never loads; the variable name misspelled (OPENAI_APIKEY, OPENAI_KEY); an exported empty value.","commonSituations":"CI jobs with clean environments; local runs where the developer authenticated via a wrapper tool but never exported the variable; rotating keys and the shell still holding nothing.","solutions":["Export the variable in the invoking shell before running the judge","For CI, inject OPENAI_API_KEY as a masked secret into the job environment","Confirm it is set and non-empty without printing it: python -c \"import os; assert os.getenv('OPENAI_API_KEY')\""],"exampleFix":"# before\n# key never exported; judge aborts on first call\npython llm_judge.py ...\n\n# after\nexport OPENAI_API_KEY='<your-key-here>'\npython llm_judge.py ...","handlingStrategy":"validation","validationCode":"import os\n\nassert os.getenv('OPENAI_API_KEY'), (\n    'export OPENAI_API_KEY (non-empty) before running the judge'\n)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Export the key in the invoking shell or inject it as a masked CI secret","Note this script reads only the environment — it does not load .env files itself","Verify presence without printing the value: python -c \"import os; assert os.getenv('OPENAI_API_KEY')\""],"tags":["environment","secrets","openai","llm-judge","benchmark"],"backgroundTag":null,"analyzedSha":"3810898a7447ec3299be72e223d3570a7aabf0ab","analyzedAt":"2026-08-16T10:14:26.282Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}