{"record":{"id":"eed3bc8b19d92697","repo":"aaif-goose/goose","slug":"model-must-be-in-provider-model-form-e-g-anthr","errorCode":null,"errorMessage":"--model must be in provider/model form, e.g. anthropic/claude-sonnet-4-6","messagePattern":"--model must be in provider/model form, e\\.g\\. anthropic/claude-sonnet-4-6","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"evals/harbor/runner.py","lineNumber":128,"sourceCode":"    if index_url is None:\n        return {}\n    return {key: index_url for key in PACKAGE_INDEX_ENV_VARS}\n\n\ndef dataset_config(dataset_ref: str, tasks: list[str]) -> dict[str, Any]:\n    name, sep, ref = dataset_ref.rpartition(\"@\")\n    dataset_name = name if sep else dataset_ref\n    dataset: dict[str, Any] = {\"name\": dataset_name}\n    if sep:\n        dataset[\"ref\" if \"/\" in name else \"version\"] = ref\n    if tasks:\n        dataset[\"task_names\"] = tasks\n    return dataset\n\n\ndef build_harbor_config(args: argparse.Namespace) -> dict[str, Any]:\n    if \"/\" not in args.model:\n        raise ValueError(\"--model must be in provider/model form, e.g. anthropic/claude-sonnet-4-6\")\n    if args.trials < 1:\n        raise ValueError(\"--trials must be at least 1\")\n    if args.concurrency < 1:\n        raise ValueError(\"--concurrency must be at least 1\")\n    if args.timeout_multiplier <= 0:\n        raise ValueError(\"--timeout-multiplier must be positive\")\n\n    goose_binary = args.goose_binary.expanduser().resolve()\n    if not goose_binary.is_file():\n        raise ValueError(f\"--goose-binary does not exist or is not a file: {args.goose_binary}\")\n\n    config_yaml, extension_entries = render_goose_config(args.extensions)\n\n    provider = args.model.split(\"/\", 1)[0]\n    missing_secrets = [\n        key for key in PROVIDER_SECRETS.get(provider, []) if not os.environ.get(key)\n    ]\n    if missing_secrets:","sourceCodeStart":110,"sourceCodeEnd":146,"githubUrl":"https://github.com/aaif-goose/goose/blob/3810898a7447ec3299be72e223d3570a7aabf0ab/evals/harbor/runner.py#L110-L146","documentation":"Raised by build_harbor_config in the harbor benchmark runner when the --model string contains no '/'. The runner requires provider/model form (e.g. anthropic/claude-sonnet-4-6) because it splits on the first slash to get the provider prefix, which it then uses to look up required secrets in PROVIDER_SECRETS and to configure the agent. A bare model name gives it no provider, so it refuses up front rather than failing later with a cryptic provider error.","triggerScenarios":"Running with --model claude-sonnet-4-6; --model gpt-4o; any model id copied without its provider prefix.","commonSituations":"Copying the model id column of a leaderboard rather than the provider/model column; switching from another harness that takes bare model names; defaulting mentally to just the model because the provider is implied by the API key in use.","solutions":["Pass the provider-qualified id: --model anthropic/claude-sonnet-4-6","Check the provider prefix matches one PROVIDER_SECRETS knows (anthropic, openai, ...) so the secret lookup in error 53 does not silently find nothing","If wrapping the runner in scripts, validate model strings with a '/' check before invoking it"],"exampleFix":"# before\npython runner.py --model claude-sonnet-4-6\n\n# after\npython runner.py --model anthropic/claude-sonnet-4-6","handlingStrategy":"validation","validationCode":"def is_provider_model(model: str) -> bool:\n    provider, sep, name = model.partition('/')\n    return bool(sep) and bool(provider) and bool(name)\n\nassert is_provider_model(args.model), '--model must be provider/model, e.g. anthropic/claude-sonnet-4-6'","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Always copy model ids as provider/model from provider docs or leaderboards","Validate the slash form in wrapper scripts before invoking the runner","Keep the provider prefix consistent with the secrets map so later env checks are meaningful"],"tags":["cli","validation","model-config","harbor","benchmark"],"backgroundTag":null,"analyzedSha":"3810898a7447ec3299be72e223d3570a7aabf0ab","analyzedAt":"2026-08-16T10:14:26.282Z","schemaVersion":2},"datasetVersion":"2026-08-16T13:17:31.715Z"}