aaif-goose/goose · error · ValueError

Job name must start with a letter or number and contain only

Error message

Job name must start with a letter or number and contain only letters, numbers, dots, underscores, and hyphens

What it means

Raised by validate_job_name in the harbor benchmark runner before any job is launched. A job name must match ^[A-Za-z0-9][A-Za-z0-9._-]*$: start with a letter or digit, then only letters, digits, dots, underscores, hyphens. The default name built by default_job_name always passes because it sanitizes model and dataset with re.sub; the error only fires on a user-supplied --job-name that violates the charset rule (Kubernetes-style name constraints).

Source

Thrown at evals/harbor/runner.py:91

    for name, entry in available.items():
        entry["enabled"] = name in extensions

    recipe_entries = [
        {"type": available[name]["type"], "name": name} for name in extensions
    ]
    return yaml.dump(template, sort_keys=False), recipe_entries


def default_job_name(model: str, dataset: str) -> str:
    safe_model = re.sub(r"[^A-Za-z0-9._-]+", "-", model).strip("-")
    safe_dataset = re.sub(r"[^A-Za-z0-9._-]+", "-", dataset).strip("-")
    timestamp = datetime.now().strftime("%Y-%m-%d__%H-%M-%S")
    return f"goose-{safe_dataset}-{safe_model}-{timestamp}"


def validate_job_name(job_name: str) -> str:
    if not re.match(r"^[A-Za-z0-9][A-Za-z0-9._-]*$", job_name):
        raise ValueError(
            "Job name must start with a letter or number and contain only "
            "letters, numbers, dots, underscores, and hyphens"
        )
    return job_name


def parse_csv(value: str) -> list[str]:
    return [item.strip() for item in value.split(",") if item.strip()]


PACKAGE_INDEX_ENV_VARS = ("UV_DEFAULT_INDEX", "PIP_INDEX_URL", "UV_INDEX_URL")


def package_index_env() -> dict[str, str]:
    index_url = next(
        (os.environ[key] for key in PACKAGE_INDEX_ENV_VARS if os.environ.get(key)),
        None,
    )

View on GitHub (pinned to 3810898a74)

Solutions

1. Use a name matching the pattern, e.g. --job-name run.clb-2_baseline
2. Omit --job-name entirely and let default_job_name build a safe goose-<dataset>-<model>-<timestamp> name
3. If generating names programmatically, sanitize first: re.sub(r'[^A-Za-z0-9._-]+', '-', name).strip('-') and prefix 'goose-' if it no longer starts alphanumerically

Example fix

# before
name = args.job_name or default_job_name(args.model, args.dataset)
validate_job_name(name)  # raises on 'my run!!'

# after
name = args.job_name or default_job_name(args.model, args.dataset)
name = re.sub(r'[^A-Za-z0-9._-]+', '-', name).strip('-')
if not re.match(r'^[A-Za-z0-9]', name):
    name = f'goose-{name}'
validate_job_name(name)

import re

def is_valid_job_name(name: str) -> bool:
    return bool(re.match(r'^[A-Za-z0-9][A-Za-z0-9._-]*$', name))

assert is_valid_job_name(args.job_name), 'job name must match ^[A-Za-z0-9][A-Za-z0-9._-]*$'

When it happens

Trigger: Passing --job-name 'my run' (space), 'run/claude' (slash), 'run#1' (hash); a name starting with '-' or '_' such as --job-name '_baseline'; a name that is empty after shell quoting mishaps.

Common situations: Using human-readable labels with spaces or colons as job names; prefixing names with dashes for sorting; copy-pasting names from issue titles containing punctuation.

Related errors

• --model must be in provider/model form, e.g. anthropic/claud
• --trials must be at least 1
• --concurrency must be at least 1
• --timeout-multiplier must be positive
• --goose-binary does not exist or is not a file: {args.goose_

AI-assisted analysis of aaif-goose/goose@3810898a74 (2026-08-16). Data as JSON: /api/errors/c8868a32145cd1ca. Report an issue: GitHub.