abi/screenshot-to-code · critical · Exception

Gemini API key not found

Error message

Gemini API key not found

What it means

Exception raised by the eval runner core when model is in GEMINI_MODELS and the module-level GEMINI_API_KEY is empty. Eval runs use Gemini for the evaluated model (and potentially for asset extraction), so the guard fires before the agent runner and recorder are built.

Solutions

  1. Set GEMINI_API_KEY in the environment / backend/.env of the process that executes evals and restart it.
  2. Confirm the exact variable name (GEMINI_API_KEY, not GOOGLE_API_KEY).
  3. Evaluate a different model family if no Gemini key is available.

Example fix

# backend/.env
GEMINI_API_KEY=AIza...
Defensive patterns

Strategy: validation

Validate before calling

import os
if model in GEMINI_MODELS and not os.environ.get("GEMINI_API_KEY"):
    raise SystemExit("GEMINI_API_KEY not set; cannot run evals on Gemini models")

Try / catch

try:
    await run_eval(...)
except Exception as e:
    if "API key not found" in str(e):
        abort_run("Configure GEMINI_API_KEY and restart the eval worker")

Prevention

When it happens

Trigger: Starting an eval run with a Gemini model while GEMINI_API_KEY is unset in the process running the evals.

Common situations: Same key configured for the app but evals run in a separate process/cron without the var; key added to .env after the eval worker started.

Related errors

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/db5d48767f161ad0. Report an issue: GitHub.

Appendix: source

Thrown at backend/evals/core.py:44

    input_mode: str,
    eval_set: str | None,
    eval_session_id: str | None,
    input_file: str | None,
) -> str:
    async def send_message(
        _: str,
        __: str | None,
        ___: int,
        ____: dict[str, Any] | None = None,
        _____: str | None = None,
    ) -> None:
        # Evals do not stream tool/assistant messages to a frontend.
        return None

    if model in ANTHROPIC_MODELS and not ANTHROPIC_API_KEY:
        raise Exception("Anthropic API key not found")
    if model in GEMINI_MODELS and not GEMINI_API_KEY:
        raise Exception("Gemini API key not found")
    if model in OPENAI_MODELS and not OPENAI_API_KEY:
        raise Exception("OpenAI API key not found")

    print(f"[EVALS] Using agent runner for model: {model.value}")

    recorder = AgentRunRecorder(
        generation_id=(
            f"gen_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{uuid.uuid4().hex[:8]}"
        ),
        variant_index=0,
        entry_point="eval",
        stack=str(stack),
        input_mode=input_mode,
        generation_type="create",
        eval_session=eval_session_id,
        eval_set=eval_set,
        input_file=input_file,
    )

View on GitHub (pinned to d026163f58)