What causes "Anthropic API key not found" in abi/screenshot-to-code?

Set ANTHROPIC_API_KEY in backend/.env and restart whatever process runs the evals.

abi/screenshot-to-code · critical · Exception

Anthropic API key not found

Error message

Anthropic API key not found

What it means

Exception raised by the eval runner core when model is in ANTHROPIC_MODELS but the module-level ANTHROPIC_API_KEY (loaded from env at import) is falsy. This is the evals-side duplicate of the factory guard: evals build their own agent runner with a no-op streaming callback, so they check keys before constructing it.

Source

Thrown at backend/evals/core.py:42

    stack: Stack,
    model: Llm,
    input_mode: str,
    eval_set: str | None,
    eval_session_id: str | None,
    input_file: str | None,
) -> str:
    async def send_message(
        _: str,
        __: str | None,
        ___: int,
        ____: dict[str, Any] | None = None,
        _____: str | None = None,
    ) -> None:
        # Evals do not stream tool/assistant messages to a frontend.
        return None

    if model in ANTHROPIC_MODELS and not ANTHROPIC_API_KEY:
        raise Exception("Anthropic API key not found")
    if model in GEMINI_MODELS and not GEMINI_API_KEY:
        raise Exception("Gemini API key not found")
    if model in OPENAI_MODELS and not OPENAI_API_KEY:
        raise Exception("OpenAI API key not found")

    print(f"[EVALS] Using agent runner for model: {model.value}")

    recorder = AgentRunRecorder(
        generation_id=(
            f"gen_{datetime.now().strftime('%Y%m%d_%H%M%S')}_{uuid.uuid4().hex[:8]}"
        ),
        variant_index=0,
        entry_point="eval",
        stack=str(stack),
        input_mode=input_mode,
        generation_type="create",
        eval_session=eval_session_id,
        eval_set=eval_set,

View on GitHub (pinned to d026163f58)

Solutions

Set ANTHROPIC_API_KEY in backend/.env and restart whatever process runs the evals.
Verify the var in the exact process: the check uses the value captured at module import, so a restart is required after editing .env.
Or run the eval set against a model whose key is present.

Example fix

# backend/.env
ANTHROPIC_API_KEY=sk-ant-...

Defensive patterns

Strategy: validation

Validate before calling

import os
required = {Llm.__members__.get(m) for m in []}  # placeholder
if model in ANTHROPIC_MODELS and not os.environ.get("ANTHROPIC_API_KEY"):
    raise SystemExit("ANTHROPIC_API_KEY not set; cannot run evals on Claude models")

Try / catch

try:
    await run_eval(...)
except Exception as e:
    if "API key not found" in str(e):
        abort_run("Configure the provider key and restart the eval worker")

Prevention

Provision all three provider keys in the eval worker's environment even if you eval one family.
Restart the eval worker after .env changes — keys are read at module import.
Fail eval sessions early with a clear message instead of mid-run.

When it happens

Trigger: Starting an eval run (e.g. /run_evals or the eval sessions UI) with a Claude model selected while ANTHROPIC_API_KEY is unset in the backend process.

Common situations: Eval suite launched from a shell without the env var, backend restarted before .env was complete, or evals run in CI without secrets.

Related errors

AI-assisted analysis of abi/screenshot-to-code@d026163f58 (2026-08-14). Data as JSON: /api/errors/b06a6bbae3c8c036. Report an issue: GitHub.