aaif-goose/goose · error · ValueError

Missing env vars for provider '{provider}': {', '.join(missi

Error message

Missing env vars for provider '{provider}': {', '.join(missing_secrets)}. Set them in a .env file (cwd or {HARBOR_DIR}) or your shell.

What it means

Raised by build_harbor_config in the harbor benchmark runner after it splits --model into a provider prefix and looks that provider up in PROVIDER_SECRETS (imported from agent.py). Every listed environment variable (typically the provider API key) must already exist and be non-empty in os.environ; load_dotenv only fills vars from a .env file in the cwd or in evals/harbor/, and only ones not already set. The message names the provider, the exact missing keys, and both .env locations.

Source

Thrown at evals/harbor/runner.py:147

    if args.trials < 1:
        raise ValueError("--trials must be at least 1")
    if args.concurrency < 1:
        raise ValueError("--concurrency must be at least 1")
    if args.timeout_multiplier <= 0:
        raise ValueError("--timeout-multiplier must be positive")

    goose_binary = args.goose_binary.expanduser().resolve()
    if not goose_binary.is_file():
        raise ValueError(f"--goose-binary does not exist or is not a file: {args.goose_binary}")

    config_yaml, extension_entries = render_goose_config(args.extensions)

    provider = args.model.split("/", 1)[0]
    missing_secrets = [
        key for key in PROVIDER_SECRETS.get(provider, []) if not os.environ.get(key)
    ]
    if missing_secrets:
        raise ValueError(
            f"Missing env vars for provider '{provider}': {', '.join(missing_secrets)}. "
            f"Set them in a .env file (cwd or {HARBOR_DIR}) or your shell."
        )

    agent_kwargs: dict[str, Any] = {
        "goose_binary": str(goose_binary),
        "config_yaml": config_yaml,
        "extension_entries": extension_entries,
        "install_goose_runtime_deps": args.install_goose_runtime_deps,
    }
    if args.max_turns is not None:
        agent_kwargs["max_turns"] = args.max_turns

    job_name = (
        validate_job_name(args.job_name)
        if args.job_name
        else default_job_name(args.model, args.dataset)
    )

View on GitHub (pinned to 3810898a74)

Solutions

1. Put the named keys in a .env file in the cwd or in evals/harbor/ (KEY=value lines), or export them in the shell
2. Copy the exact key names from the error message; check spelling and that values are non-empty
3. Confirm the provider prefix in --model is the one PROVIDER_SECRETS expects (anthropic/, openai/, ...)

Example fix

# before
# no .env anywhere; keys only in the desktop app
python runner.py --model anthropic/claude-sonnet-4-6

# after
printf 'ANTHROPIC_API_KEY=<your-key-here>\n' > evals/harbor/.env
python runner.py --model anthropic/claude-sonnet-4-6

import os
from agent import PROVIDER_SECRETS

provider = args.model.split('/', 1)[0]
missing = [k for k in PROVIDER_SECRETS.get(provider, []) if not os.environ.get(k)]
assert not missing, (
    f'missing env vars: {missing}. '
    'Put them in ./.env or evals/harbor/.env, or export them'
)

When it happens

Trigger: Running with --model anthropic/claude-sonnet-4-6 without ANTHROPIC_API_KEY in the environment or either .env file; a .env that exists but misspells the key name; exporting the variable as an empty string (os.environ.get returns '' which counts as missing); using a provider prefix whose secrets map entry differs (e.g. a bare model provider alias).

Common situations: CI shells that start clean without sourcing the developer's env; .env placed in a directory other than cwd or evals/harbor/; rotating keys and forgetting to update .env; assuming the desktop app's stored key is visible to the eval runner.

Related errors

• OPENAI_API_KEY environment variable is not set, but is neede
• GOOSE_SERVER__SECRET_KEY must be set when using GOOSE_EXTERN
• Job name must start with a letter or number and contain only
• --model must be in provider/model form, e.g. anthropic/claud
• --trials must be at least 1

AI-assisted analysis of aaif-goose/goose@3810898a74 (2026-08-16). Data as JSON: /api/errors/ddf53a3683910d63. Report an issue: GitHub.